A method and system for displaying data fusion results, a vehicle

Abstract

The present invention discloses a method and system for displaying data fusion results, and a vehicle. The invention includes: receiving and parsing data messages sent by a vehicle in real time through a message parsing thread, and obtaining fused data; writing the fused data into a protected area; after detecting that the display thread has received a preset signal, controlling the display thread to read the fused data in the protected area; determining the fusion result based on the fusion data, and displaying the fusion result, wherein the fusion result at least includes obstacle information at a preset distance from the vehicle, and the obstacle information at least includes the location information of the obstacle and the size information of the obstacle. The present invention solves the technical problem in the related art that the obstacle information collected by the vehicle during driving cannot be fused in real time and displayed intuitively.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving, and in particular to a method and system for displaying data fusion results, and a vehicle. Background Art

[0002] Nowadays, autonomous driving technology is developing rapidly at home and abroad, and various OEMs and new car manufacturers are stepping up the research and development of vehicle autonomous driving functions. As the level of intelligent driving increases, the types and quantities of sensors are also increasing, so multi-sensor fusion technology is an inevitable trend. In the multi-sensor fusion technology link, an important issue is how to quickly and intuitively obtain and analyze the fusion results in different road test scenarios.

[0003] The commonly used method to view fusion results is to analyze them through the CANoe tool, which uses the DBC tool to parse the CAN frame signal transmitted on the actual vehicle into an application layer signal and displays the signal in the form of a chart. The advantages of this method of observing fusion results are easy operation, good real-time performance, and strong versatility. However, the fusion results can only be fed back through curves, and the position and speed relationship of each fusion target cannot be directly obtained. In addition, the distance jump of the fused target and the target misfusion scenario cannot be intuitively analyzed and displayed.

[0004] With regard to the technical problem in related technologies that obstacle information collected by vehicles during driving cannot be integrated in real time and displayed intuitively, no effective solution has been proposed so far. Summary of the invention

[0005] The main purpose of the present invention is to provide a method and system for displaying data fusion results and a vehicle to solve the technical problem in the related art that obstacle information collected by the vehicle during driving cannot be fused in real time and intuitively displayed.

[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present invention, a method for displaying data fusion results is provided. The invention comprises: receiving and parsing data messages sent by vehicles in real time through a message parsing thread, and obtaining fused data; writing the fused data into a protected area; after detecting that the display thread has received a preset signal, controlling the display thread to read the fused data in the protected area; determining the fusion result based on the fusion data, and displaying the fusion result, wherein the fusion result at least includes obstacle information at a preset distance from the vehicle, and the obstacle information at least includes the location information of the obstacle and the size information of the obstacle.

[0007] Furthermore, before receiving and parsing the data message sent by the vehicle in real time through the message parsing thread and obtaining the fused data, the method also includes: obtaining the vehicle's driving information and obstacle information, wherein the driving information includes at least the vehicle's speed and acceleration; fusing the driving information and obstacle information into fused data, and packaging the fused data into a data message.

[0008] Furthermore, the vehicle's driving information and obstacle information are obtained, including: obtaining the vehicle's driving information transmitted through the vehicle's CAN bus; obtaining radar information sent by the laser radar from the private CAN corresponding to the laser radar, wherein the radar information at least includes obstacle information; obtaining visual information collected by the vehicle-mounted camera, wherein the visual information at least includes obstacle information.

[0009] Furthermore, before fusing the driving information and obstacle information into fused data and packaging the fused data into a data message, the method includes: calling a preset underlying library to initialize the channels corresponding to multiple buses and the ports corresponding to multiple buses; after the channels and ports are initialized, receiving the fused data by registering the system kernel.

[0010] Furthermore, after detecting that the display thread has received a preset signal, before controlling the display thread to read the fusion data in the protection zone, the method includes: requesting a preset semaphore from the protection zone; if the request fails, controlling the display thread to enter a sleep state; if the display thread is in a sleep state, requesting the preset semaphore in real time until the request succeeds; if the preset semaphore is successfully requested, controlling the display thread to exit the sleep state.

[0011] Furthermore, a fusion result is determined based on the fused data, including: drawing a bird's-eye view including obstacles based on the location information of the obstacles contained in the fused data; determining a table including obstacle size information and obstacle location information based on the size information of the obstacles and the location information of the obstacles contained in the fused data; and determining the bird's-eye view and the table as the fusion result.

[0012] Furthermore, displaying the fusion result includes: sending the fusion data to a host computer, and controlling the host computer to display the fusion result.

[0013] In order to achieve the above-mentioned purpose, according to another aspect of the present invention, a display system for data fusion results is provided. The device includes: a message parsing unit, which is used to receive and parse the data message sent by the vehicle in real time through the message parsing thread, and obtain fusion data; a writing unit, which is used to write the fusion data into the protection zone; a first reading unit, which is used to control the display thread to read the fusion data in the protection zone after detecting that the display thread receives a preset signal; a data fusion unit, which is used to determine the fusion result based on the fusion data and display the fusion result, wherein the fusion result at least includes the obstacle information at a preset distance from the vehicle, and the obstacle information at least includes the location information of the obstacle and the size information of the obstacle.

[0014] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein the program executes any one of the above-mentioned data collection methods based on a policy sandbox.

[0015] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a vehicle is provided, characterized in that it includes: a vehicle-mounted radar; a vehicle-mounted camera; a display system for data fusion results, which is used to execute the above-mentioned method for displaying data fusion results.

[0016] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a processor is provided, and the processor is used to run a program, wherein the program executes any one of the above-mentioned data collection methods based on a policy sandbox.

[0017] Through the present invention, the following steps are adopted: receiving and parsing the data message sent by the vehicle in real time through the message parsing thread, and obtaining the fused data; writing the fused data into the protected area; after detecting that the display thread receives the preset signal, controlling the display thread to read the fused data in the protected area; determining the fusion result based on the fusion data, and displaying the fusion result, wherein the fusion result at least includes the obstacle information at the preset distance from the vehicle, and the obstacle information at least includes the location information of the obstacle and the size information of the obstacle, which solves the technical problem in the related art that the obstacle information collected by the vehicle during driving cannot be fused in real time and displayed intuitively. Thus, the technical effect of quickly and intuitively obtaining and analyzing the fusion results in different road test scenarios is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0019] Figure 1 is a flow chart of a method for displaying data fusion results provided according to an embodiment of the present invention;

[0020] Figure 2 A schematic diagram of the structure of a data fusion result display system 1 provided in an embodiment of the present application; and

[0021] Figure 3 4 is a schematic diagram of a second display system for data fusion results according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so as to describe the embodiments of the present invention described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] According to an embodiment of the present invention, a method for displaying data fusion results is provided.

[0026] Figure 1 FIG. 1 is a flow chart of a method for displaying data fusion results according to an embodiment of the present invention. Figure 1 As shown, the invention comprises the following steps:

[0027] Step S101, receiving and parsing data messages sent by vehicles in real time through a message parsing thread, and obtaining fused data;

[0028] Step S102, writing the fused data into the protection area;

[0029] Step S103, after detecting that the display thread receives a preset signal, controlling the display thread to read the fusion data in the protection zone;

[0030] Step S104, determining a fusion result based on the fusion data, and displaying the fusion result, wherein the fusion result at least includes obstacle information at a preset distance from the vehicle, and the obstacle information at least includes location information of the obstacle and size information of the obstacle.

[0031] The embodiment of the present invention provides a method for displaying data fusion results, which receives and analyzes data messages sent by vehicles in real time through a message parsing thread, and obtains fused data; writes the fused data into a protection zone; after detecting that the display thread has received a preset signal, controls the display thread to read the fused data in the protection zone; determines the fusion result based on the fusion data, and displays the fusion result, wherein the fusion result at least includes obstacle information at a preset distance from the vehicle, and the obstacle information at least includes the location information of the obstacle and the size information of the obstacle. The technical problem in the related art that the obstacle information collected by the vehicle during driving cannot be fused in real time and displayed intuitively is solved. Thus, the technical effect of quickly and intuitively obtaining and analyzing the fusion results in different road test scenarios is achieved.

[0032] As mentioned above, the current autonomous driving cannot perform online analysis when acquiring and analyzing fusion results. The reason is that in the process of developing the algorithm, two methods are mainly used to verify the fusion results. One is to analyze the fusion data online through CANoe / CANape, but this method is very unintuitive, and it is difficult to observe all the fusion results at the current moment due to the limitation of the bus load; the other is to send the fusion results from the bus, collect the data through tools such as CANoe, and analyze the offline asc file through software such as MATLAB. This method intuitively displays the information of all fusion results, but cannot perform online analysis.

[0033] The present application provides a method for real-time display of data fusion results by multi-threaded collaborative operation, wherein the multi-thread includes a message parsing thread, a display thread and a drawing thread.

[0034] Specifically, the signal transmitted by the real vehicle is analyzed online based on the XL underlying driver library, and the signal mechanism is used to pass it to another drawing thread, and the fusion result information is visualized in the drawing thread using the Easyx interface. This method can intuitively and quickly observe the fusion results, solve the problems of traditional methods, and has a positive effect on fusion algorithm parameter adjustment, defect analysis and version iteration.

[0035] In an optional embodiment, before receiving and parsing the data message sent by the vehicle in real time through the message parsing thread and obtaining the fused data, the method also includes: obtaining the vehicle's driving information and obstacle information, wherein the driving information at least includes the vehicle's speed and acceleration; fusing the driving information and obstacle information into fused data, and packaging the fused data into a data message. Obtaining the vehicle's driving information and obstacle information includes: obtaining the vehicle's driving information transmitted through the vehicle's CAN bus from the vehicle's CAN bus; obtaining radar information sent by the laser radar from the private CAN corresponding to the laser radar, wherein the radar information at least includes obstacle information; obtaining visual information collected by the vehicle-mounted camera, wherein the visual information at least includes obstacle information.

[0036] like Figure 2 As shown, Figure 2 A structural diagram of a display system for a data fusion result provided in an embodiment of the present application, the system includes four parts, first of all, a basic body part, a vehicle-mounted radar, a vehicle-mounted camera and a host computer part. The basic body sends the vehicle body speed, acceleration and other vehicle information to the vehicle-mounted camera part through the vehicle CAN, and the vehicle-mounted radar sends the detected radar information to the vehicle-mounted camera through the private CAN of the radar. At the same time, the SOC unit on the vehicle-mounted camera will perceive the visual target (that is, the obstacle, the obstacle includes the front vehicle or pedestrian), and will transmit the visual target to the fusion module through the internal bus. When the fusion module receives the vehicle information, visual information, and radar information within the specified time, it will perform a fusion, and send the fusion result to the host computer through the debugging CAN. The message parsing thread in the host computer can perform data parsing, data display and fusion result verification according to the fusion result. The type, speed, acceleration, route, and position of the obstacle can be displayed online in real time through this fusion system structure, which ensures the real-time nature of the fusion result.

[0037] It should be noted that in the above display system, the data fusion unit is arranged in the vehicle-mounted camera. In another optional embodiment of the present application, the data fusion unit may be an independent unit for fusing multiple data after receiving the multiple data.

[0038] In an optional embodiment, before fusing driving information and obstacle information into fused data and packaging the fused data into a data message, the method includes: calling a preset underlying library to initialize channels corresponding to multiple buses and ports corresponding to multiple buses; after the channels and ports are initialized, receiving the fused data by registering with the system kernel.

[0039] As mentioned above, due to the real-time and uncertainty of sending messages, the message receiving and parsing part needs to be decoupled from the result visualization part. The message is parsed through a separate thread. First, the XL underlying library is called to complete the initialization of the bus channel and port. Secondly, the data on the bus is received by registering the system kernel. This method can ensure that when no data is received on the bus at the current moment, it can be switched to other threads through kernel scheduling, ensuring the real-time display of fusion data. Since in the current system, the cycle of data transmission on the debug bus is 50ms, when the time difference between the received data and the previous time exceeds 50ms, a new round of data transmission will be started, so the data in the previous round needs to be cleared. When the expected number of data frames are received within the unit period, they need to be parsed and data extracted through the protocol content. The method of parsing messages in DBC files adopts multi-threaded collaborative operation, so a protection mechanism is needed. The fusion result is written into the protection zone through the read and write interface of the protection zone. After writing, the drawing signal is sent to the display thread through the signal mechanism of the system kernel, and finally enters the next cycle to wait for the next round of data on the debug bus.

[0040] In an optional embodiment, after detecting that the display thread has received a preset signal, before controlling the display thread to read the fusion data in the protection zone, the method includes: requesting a preset semaphore from the protection zone; if the request fails, controlling the display thread to enter a sleep state; if the display thread is in a sleep state, requesting a preset semaphore in real time until the request succeeds; if the preset semaphore is successfully requested, controlling the display thread to exit the sleep state.

[0041] As mentioned above, since the verification method operates through the coordinated operation of three parts, data may be read and written at the same time. Therefore, this part needs to protect the reading and writing of data stored in the protected area. This method uses a semaphore mechanism to protect it. When performing a read or write operation, the display thread first needs to request a read-write semaphore. When the semaphore is occupied, it means that other threads have operations on the data area at this time, so the display thread needs to enter a dormant state and wait for operations from other threads. When the semaphore is not occupied, it means that the thread can monopolize the data area at this time, and then the data can be read and written normally. When the read and write operations are completed, the display thread needs to release the semaphore so that other threads can continue to operate.

[0042] In an optional embodiment, a fusion result is determined based on the fused data, including: drawing a bird's-eye view including obstacles based on the location information of the obstacles included in the fused data; determining a table including obstacle size information and obstacle location information based on the size information of the obstacles and the location information of the obstacles included in the fused data; and determining the bird's-eye view and the table as the fusion result.

[0043] In an optional embodiment, displaying the fusion result includes: sending the fusion data to a host computer, and controlling the host computer to display the fusion result.

[0044] As mentioned above, the last one is the data visualization display module. When receiving the drawing signal sent by the message module, this module is activated and switches from the sleep state to the running state. Every time a signal is received, it will request the fusion result data from the protection zone. When the normal result is received, the final fusion result can be displayed on the host computer by calling the Easyx interface.

[0045] As mentioned above, the fusion result displayed by the fusion result display module is a bird's-eye view including obstacles and vehicles, and the bird's-eye view includes the location information, size information and relative position information of the obstacle and the vehicle of this application. At the same time, in an optional embodiment, the fusion result also includes a table containing obstacle information, and the table contains at least the location coordinate information, size information, and status information (divided into static and dynamic) of the obstacle. When the state of the obstacle is dynamic, the speed information of the obstacle is also displayed in the table.

[0046] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0047] The embodiment of the present invention further provides a data fusion result display system. It should be noted that the data fusion result display system of the embodiment of the present invention can be used to execute the data fusion result display method provided by the embodiment of the present invention. The following introduces a data fusion result display system provided by the embodiment of the present invention.

[0048] The present application also provides a vehicle, characterized in that it includes: a vehicle-mounted radar; a vehicle-mounted camera; and a data fusion result display system, which is used to execute the above-mentioned method for displaying data fusion results.

[0049] Figure 3 FIG. 1 is a schematic diagram of a second display system for data fusion results according to an embodiment of the present invention. Figure 3As shown, the device includes: a message parsing unit 301, which is used to receive and parse the data message sent by the vehicle in real time through the message parsing thread, and obtain fused data; a writing unit 302, which is used to write the fused data into the protection zone; a first reading unit 303, which is used to control the display thread to read the fused data in the protection zone after detecting that the display thread has received a preset signal; a data fusion unit 304, which is used to determine the fusion result based on the fusion data and display the fusion result, wherein the fusion result at least includes the obstacle information at a preset distance from the vehicle, and the obstacle information at least includes the position information of the obstacle and the size information of the obstacle.

[0050] In an optional embodiment, the second receiving unit is used to receive and parse the data message sent by the vehicle in real time through the message parsing thread, and before obtaining the fused data, the device also includes: a first acquisition subunit, used to obtain the vehicle's driving information and obstacle information, wherein the driving information includes at least the vehicle's speed and acceleration; a first fusion subunit, used to fuse the driving information and obstacle information into fused data, and pack the fused data into a data message.

[0051] In an optional embodiment, the data fusion unit 304 includes: a second acquisition subunit, used to acquire the driving information of the vehicle transmitted through the vehicle CAN bus from the vehicle CAN bus; a third acquisition subunit, used to acquire radar information sent by the laser radar from the private CAN corresponding to the laser radar, wherein the radar information at least includes obstacle information; a fourth acquisition subunit, used to acquire visual information collected by the vehicle-mounted camera, wherein the visual information at least includes obstacle information.

[0052] In an optional embodiment, the second fusion sub-unit is used to fuse the driving information and obstacle information into fused data, and before packaging the fused data into a data message, the device includes: a calling module, used to call a preset underlying library to initialize the channels corresponding to multiple buses and the ports corresponding to multiple buses; a receiving module, used to receive the fused data by registering the system kernel after the channels and ports are initialized.

[0053] In an optional embodiment, the second reading unit is used to control the display thread to read the fusion data in the protection zone after detecting that the display thread has received a preset signal, and the device includes: a first request sub-unit, used to request a preset semaphore from the protection zone; a first control sub-unit, used to control the display thread to enter a sleep state if the request fails; a second request sub-unit, used to request the preset semaphore in real time when the display thread is in the sleep state until the request succeeds; and a second control sub-unit, used to control the display thread to exit the sleep state if the preset semaphore is successfully requested.

[0054] In an optional embodiment, the data fusion unit 304 includes: a drawing subunit, used to draw a bird's-eye view containing obstacles based on the location information of the obstacles contained in the fused data; a first determination subunit, used to determine a table containing obstacle size information and obstacle location information based on the size information of the obstacles and the location information of the obstacles contained in the fused data; and a second determination subunit, used to determine the bird's-eye view and the table as the fusion result.

[0055] In an optional embodiment, the data fusion unit 304 includes: a sending subunit, which is used to send the fused data to the host computer and control the host computer to display the fusion result.

[0056] The embodiment of the present invention provides a display system for data fusion results, which is used to receive and parse data messages sent by vehicles in real time through message parsing threads through message parsing units 301, and obtain fusion data; a writing unit 302, which is used to write fusion data into a protection zone; a first reading unit 303, which is used to control the display thread to read the fusion data in the protection zone after detecting that the display thread receives a preset signal; a data fusion unit 304, which is used to determine the fusion result according to the fusion data and display the fusion result, wherein the fusion result at least includes obstacle information at a preset distance from the vehicle, and the obstacle information at least includes the position information of the obstacle and the size information of the obstacle. The technical problem that the obstacle information collected by the vehicle during driving cannot be fused and displayed intuitively in real time in the related art is solved, thereby achieving the technical effect of quickly and intuitively obtaining and analyzing the fusion results in different road test scenarios.

[0057] A data fusion result display system includes a processor and a memory. The message parsing unit 301 and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions.

[0058] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the technical problem that the obstacle information collected by the vehicle during driving cannot be integrated and displayed intuitively in real time in the related technology can be solved by adjusting the kernel parameters.

[0059] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0060] An embodiment of the present invention provides a storage medium on which a program is stored. When the program is executed by a processor, a method for displaying a data fusion result is implemented.

[0061] An embodiment of the present invention provides a processor, which is used to run a program, wherein a method for displaying data fusion results is executed when the program is run.

[0062] An embodiment of the present invention provides a device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the following steps are implemented: receiving and parsing data messages sent by a vehicle in real time through a message parsing thread, and obtaining fused data; writing the fused data into a protected area; after detecting that a display thread has received a preset signal, controlling the display thread to read the fused data in the protected area; determining a fusion result based on the fusion data, and displaying the fusion result, wherein the fusion result at least includes obstacle information at a preset distance from the vehicle, and the obstacle information at least includes location information of the obstacle and size information of the obstacle.

[0063] Furthermore, before receiving and parsing the data message sent by the vehicle in real time through the message parsing thread and obtaining the fused data, the method also includes: obtaining the vehicle's driving information and obstacle information, wherein the driving information includes at least the vehicle's speed and acceleration; fusing the driving information and obstacle information into fused data, and packaging the fused data into a data message.

[0064] Furthermore, the vehicle's driving information and obstacle information are obtained, including: obtaining the vehicle's driving information transmitted through the vehicle's CAN bus; obtaining radar information sent by the laser radar from the private CAN corresponding to the laser radar, wherein the radar information at least includes obstacle information; obtaining visual information collected by the vehicle-mounted camera, wherein the visual information at least includes obstacle information.

[0065] Furthermore, before fusing the driving information and obstacle information into fused data and packaging the fused data into a data message, the method includes: calling a preset underlying library to initialize the channels corresponding to multiple buses and the ports corresponding to multiple buses; after the channels and ports are initialized, receiving the fused data by registering the system kernel.

[0066] Furthermore, after detecting that the display thread has received a preset signal, before controlling the display thread to read the fusion data in the protection zone, the method includes: requesting a preset semaphore from the protection zone; if the request fails, controlling the display thread to enter a sleep state; if the display thread is in a sleep state, requesting the preset semaphore in real time until the request succeeds; if the preset semaphore is successfully requested, controlling the display thread to exit the sleep state.

[0067] Furthermore, a fusion result is determined based on the fused data, including: drawing a bird's-eye view including obstacles based on the location information of the obstacles contained in the fused data; determining a table including obstacle size information and obstacle location information based on the size information of the obstacles and the location information of the obstacles contained in the fused data; and determining the bird's-eye view and the table as the fusion result.

[0068] Furthermore, displaying the fusion result includes: sending the fusion data to a host computer, and controlling the host computer to display the fusion result.

[0069] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0070] The present invention also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: receiving and parsing data messages sent by a vehicle in real time through a message parsing thread, and obtaining fused data; writing the fused data into a protected area; after detecting that a display thread has received a preset signal, controlling the display thread to read the fused data in the protected area; determining a fusion result based on the fusion data, and displaying the fusion result, wherein the fusion result at least includes obstacle information at a preset distance from the vehicle, and the obstacle information at least includes the location information of the obstacle and the size information of the obstacle.

[0071] Furthermore, before receiving and parsing the data message sent by the vehicle in real time through the message parsing thread and obtaining the fused data, the method also includes: obtaining the vehicle's driving information and obstacle information, wherein the driving information includes at least the vehicle's speed and acceleration; fusing the driving information and obstacle information into fused data, and packaging the fused data into a data message.

[0072] Furthermore, the vehicle's driving information and obstacle information are obtained, including: obtaining the vehicle's driving information transmitted through the vehicle's CAN bus; obtaining radar information sent by the laser radar from the private CAN corresponding to the laser radar, wherein the radar information at least includes obstacle information; obtaining visual information collected by the vehicle-mounted camera, wherein the visual information at least includes obstacle information.

[0073] Furthermore, before fusing the driving information and obstacle information into fused data and packaging the fused data into a data message, the method includes: calling a preset underlying library to initialize the channels corresponding to multiple buses and the ports corresponding to multiple buses; after the channels and ports are initialized, receiving the fused data by registering the system kernel.

[0074] Furthermore, after detecting that the display thread has received a preset signal, before controlling the display thread to read the fusion data in the protection zone, the method includes: requesting a preset semaphore from the protection zone; if the request fails, controlling the display thread to enter a sleep state; if the display thread is in a sleep state, requesting the preset semaphore in real time until the request succeeds; if the preset semaphore is successfully requested, controlling the display thread to exit the sleep state.

[0075] Furthermore, a fusion result is determined based on the fused data, including: drawing a bird's-eye view including obstacles based on the location information of the obstacles contained in the fused data; determining a table including obstacle size information and obstacle location information based on the size information of the obstacles and the location information of the obstacles contained in the fused data; and determining the bird's-eye view and the table as the fusion result.

[0076] Furthermore, displaying the fusion result includes: sending the fusion data to a host computer, and controlling the host computer to display the fusion result.

[0077] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0078] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0079] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0080] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0081] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0082] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0083] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transit media), such as modulated data signals and carrier waves.

[0084] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0085] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0086] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for displaying data fusion results, characterized in that: include: Receive and parse the data messages sent by the vehicle in real time through the message parsing thread, and obtain fused data; Before receiving and parsing the data message sent by the vehicle in real time through the message parsing thread and obtaining the fused data, the method further includes: obtaining the driving information of the vehicle and the information of the obstacle, wherein the driving information at least includes the speed and acceleration of the vehicle; fusing the driving information and the obstacle information into the fused data, and packaging the fused data into the data message; Writing the fused data into the protected area; After detecting that the display thread receives a preset signal, controlling the display thread to read the fused data in the protection zone; A fusion result is determined based on the fusion data, and the fusion result is displayed, wherein the fusion result at least includes the obstacle information at a preset distance from the vehicle, and the obstacle information at least includes the position information of the obstacle and the size information of the obstacle. The fusion result is determined based on the fusion data, including: drawing a bird's-eye view including the obstacle based on the position information of the obstacle contained in the fusion data; determining a table including the obstacle size information and the obstacle position information based on the size information of the obstacle and the obstacle position information contained in the fusion data; and determining the bird's-eye view and the table as the fusion result.

2. The method according to claim 1, characterized in that Acquiring the driving information of the vehicle and the information of the obstacle, including: Acquire the driving information of the vehicle transmitted through the vehicle CAN bus from the vehicle CAN bus; Acquire radar information sent by the laser radar from a private CAN corresponding to the laser radar, wherein the radar information at least includes the obstacle information; Obtain visual information collected by a vehicle-mounted camera, wherein the visual information at least includes the obstacle information.

3. The method according to claim 2, characterized in that Before fusing the driving information and the obstacle information into the fused data and packaging the fused data into the data message, the method includes: Calling a preset underlying library to perform initialization operations on channels corresponding to a plurality of buses and ports corresponding to a plurality of the buses; After the channel and the port are initialized, the fused data is received by registering the system kernel.

4. The method according to claim 1, characterized in that: After detecting that the display thread receives a preset signal, before controlling the display thread to read the fused data in the protection zone, the method includes: Requesting a preset semaphore from the protection zone; In case of request failure, controlling the display thread to enter a dormant state; When the display thread is in the dormant state, requesting the preset semaphore in real time until the request is successful; In the case where the preset semaphore is requested successfully, the display thread is controlled to exit the sleep state.

5. The method according to claim 1, characterized in that Display the fusion result, including: The fusion data is sent to a host computer, and the host computer is controlled to display the fusion result.

6. A data fusion result display system, characterized in that: include: A message parsing unit, used to receive and parse data messages sent by vehicles in real time through a message parsing thread, and obtain fused data; The second receiving unit is used to receive and parse the data message sent by the vehicle in real time through the message parsing thread, and before obtaining the fused data, the display system further includes: a first obtaining subunit, used to obtain the driving information of the vehicle and the information of the obstacle, the driving information at least including the speed and acceleration of the vehicle; a first fusion subunit, used to fuse the driving information and the obstacle information into the fused data, and pack the fused data into the data message; A writing unit, used for writing the fused data into the protection area; A first reading unit, configured to control the display thread to read the fused data in the protection zone after detecting that the display thread receives a preset signal; A data fusion unit is used to determine a fusion result based on the fusion data and display the fusion result, wherein the fusion result at least includes the obstacle information at a preset distance from the vehicle, and the obstacle information at least includes the position information of the obstacle and the size information of the obstacle. The data fusion unit includes: a drawing subunit, used to draw a bird's-eye view containing the obstacle based on the position information of the obstacle contained in the fusion data; a first determination subunit, used to determine a table containing the obstacle size information and the obstacle position information based on the size information of the obstacle and the obstacle position information contained in the fusion data; a second determination subunit, used to determine the bird's-eye view and the table as the fusion result.

7. A vehicle, characterized in that: include: Vehicle-mounted radar; Car camera; A data fusion result display system, used to execute a data fusion result display method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute a method for displaying a data fusion result as described in any one of claims 1 to 5.

Images