Visualization methods, devices, and equipment based on multi-type vehicle data
By converting intelligent driving data into a structure format compatible with ROS2 and displaying it in the Rviz2 interface using a plugin, the problem of poor data compatibility was solved, and a unified display of multiple types of data was achieved, improving software debugging efficiency and user experience.
Patent Information
- Application Number
- CN202210698203.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-20
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2042-06-20
AI Technical Summary
In existing technologies, visualization tools for intelligent driving data have poor compatibility, making it impossible to effectively present multiple types of data on the same interface, which affects the software debugging efficiency of developers and the user experience.
By converting the preset structure format, various types of vehicle data are converted into data formats compatible with ROS2, and then displayed in the Rviz2 interface using a plugin, achieving a unified display of various types of data.
It improves the compatibility of data visualization tools, enabling the presentation of sufficient data on a single interface, thereby enhancing developers' software debugging efficiency and user experience.
Smart Images

Figure CN115145985B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to intelligent driving technology, and more particularly to a visualization display method, device, and equipment based on multi-type vehicle data. Background Technology
[0002] Currently, data in the field of intelligent driving has two main characteristics: large quantity and diverse types. How to process this data effectively and present it in a user-friendly manner directly impacts the user experience and software improvements.
[0003] In existing technologies, intelligent driving data can be displayed using an intelligent driving message visualization solution based on Rviz2. Rviz2 is a very important and commonly used data visualization tool in Robot Operating System 2 (ROS2). Most methods involve collecting the data, publishing it directly through ROS2, and then using Rviz2 to display it.
[0004] However, the above method has poor data compatibility, resulting in insufficient data presentation, which to some extent affects the software debugging efficiency of developers. Summary of the Invention
[0005] This disclosure provides a visualization method, apparatus, and device based on multi-type vehicle data to improve data compatibility, enabling data visualization tools to present sufficient data, thereby improving the software debugging efficiency of developers to a certain extent.
[0006] According to a first aspect of this disclosure, a visualization method based on multi-type vehicle data is provided, applied to an electronic device, the method comprising:
[0007] Receive vehicle data of various data types sent by the vehicle;
[0008] Vehicle data whose data type conforms to the first data type condition is converted into first data in a preset structure format; wherein, the preset structure format is a data format for visual conversion of vehicle data;
[0009] The first data and the second data from the various types of vehicle data are displayed on the same interface; wherein the data type of the second data conforms to the conditions of the second data type.
[0010] Wherein, the first data type condition includes at least one data type, and the second data type condition includes at least one data type; the data types included in the first data type condition are all different from the data types included in the second data type condition, or, some data types in the first data type condition are the same as some data types in the second data type condition.
[0011] According to a second aspect of this disclosure, a visualization method based on multi-type vehicle data is provided, applied to vehicles, the method comprising:
[0012] Acquire vehicle data of various data types and send the vehicle data of various data types to electronic devices;
[0013] Among them, vehicle data of the various data types that conform to the first data type condition are used to be converted into first data in a preset structure format; the preset structure format is a data format for visual conversion of vehicle data.
[0014] The first data and the second data from the various types of vehicle data are used to display on the same interface; the data type of the second data conforms to the second data type condition; the first data type condition includes at least one data type, and the second data type condition includes at least one data type; the data types included in the first data type condition are different from the data types included in the second data type condition, or, some data types in the first data type condition are the same as some data types in the second data type condition.
[0015] According to a third aspect of this disclosure, a visualization display device based on multi-type vehicle data is provided, applied to an electronic device, the device comprising:
[0016] The receiving unit is used to receive vehicle data of various data types sent by the vehicle.
[0017] The format conversion unit is used to convert vehicle data whose data type conforms to the first data type condition into first data in a preset structure format; wherein, the preset structure format is a data format for visual conversion of vehicle data;
[0018] The display unit is used to display first data and second data from the multiple types of vehicle data on the same interface; wherein the data type of the second data conforms to the conditions of a second data type.
[0019] Wherein, the first data type condition includes at least one data type, and the second data type condition includes at least one data type; the data types included in the first data type condition are all different from the data types included in the second data type condition, or, some data types in the first data type condition are the same as some data types in the second data type condition.
[0020] According to a fourth aspect of this disclosure, a visualization display device based on multi-type vehicle data is provided, applied to a vehicle, the device comprising:
[0021] An acquisition unit is used to acquire vehicle data of various data types and send the vehicle data of various data types to an electronic device;
[0022] Among them, vehicle data of the various data types that conform to the first data type condition are used to be converted into first data in a preset structure format; the preset structure format is a data format for visual conversion of vehicle data.
[0023] The first data and the second data from the various types of vehicle data are used to display on the same interface; the data type of the second data conforms to the second data type condition; the first data type condition includes at least one data type, and the second data type condition includes at least one data type; the data types included in the first data type condition are different from the data types included in the second data type condition, or, some data types in the first data type condition are the same as some data types in the second data type condition.
[0024] According to a fifth aspect of this disclosure, an electronic device is provided, including a memory and a processor; wherein,
[0025] The memory is used to store computer programs;
[0026] The processor is configured to read a computer program stored in the memory and execute the method as described in the first aspect according to the computer program in the memory.
[0027] According to a sixth aspect of this disclosure, a vehicle is provided, including a memory and a processor; wherein,
[0028] The memory is used to store computer programs;
[0029] The processor is configured to read a computer program stored in the memory and execute the method as described in the second aspect according to the computer program in the memory.
[0030] According to a seventh aspect of this disclosure, a visualization display system based on multi-type vehicle data is provided, including electronic equipment as described in the fifth aspect and a vehicle as described in the sixth aspect.
[0031] According to the eighth aspect of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the methods described in the first and second aspects.
[0032] According to a ninth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the methods described in the first and second aspects.
[0033] The present disclosure provides a visualization display method, apparatus, and device based on multiple types of vehicle data, comprising: receiving vehicle data of multiple data types sent by vehicles; converting vehicle data whose data types conform to a first data type condition into first data in a preset structure format; wherein the preset structure format is a data format for visualizing vehicle data; displaying the first data and second data from the multiple data types of vehicle data on the same interface; wherein the data type of the second data conforms to a second data type condition; wherein the first data type condition includes at least one data type, and the second data type condition includes at least one data type; the data types included in the first data type condition are all different from the data types included in the second data type condition, or some data types in the first data type condition are the same as some data types in the second data type condition. The visualization display method, apparatus, and device based on multiple types of vehicle data provided in this disclosure can make the data visualization tool compatible with multiple data types through the conversion of the preset structure format; and can use plugins to display multiple data types in the data visualization tool; enabling the data visualization tool to present sufficient data, thereby improving the software debugging efficiency of developers and enhancing the user experience to a certain extent. Attached Figure Description
[0034] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0035] Figure 1 This is a flowchart illustrating an exemplary embodiment of the present disclosure of a visualization method based on multiple types of vehicle data.
[0036] Figure 2 A flowchart illustrating a visualization method based on multi-type vehicle data, which is another exemplary embodiment of this disclosure;
[0037] Figure 3 A schematic diagram of an Rviz2 display interface based on multi-type vehicle data, illustrating an exemplary embodiment of this disclosure;
[0038] Figure 4 This is a flowchart illustrating a visualization method based on multiple types of vehicle data, which is yet another exemplary embodiment of this disclosure.
[0039] Figure 5 A flowchart illustrating a visualization method based on multiple types of vehicle data, which is yet another exemplary embodiment of this disclosure;
[0040] Figure 6 A structural diagram of a visualization display device based on multi-type vehicle data, illustrating an exemplary embodiment of this disclosure;
[0041] Figure 7 A structural diagram of a visualization display device based on multiple types of vehicle data, as shown in another exemplary embodiment of this disclosure;
[0042] Figure 8 This is a structural diagram of an electronic device illustrated in an exemplary embodiment of the present disclosure.
[0043] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0044] Currently, data in the field of intelligent driving has two main characteristics: large quantity and diverse types. How to process this data effectively and present it in a user-friendly manner directly impacts the user experience and software improvements.
[0045] In existing technologies, intelligent driving data can be displayed using an intelligent driving message visualization solution based on Rviz2. Rviz2 is a very important and commonly used data visualization tool in ROS2. Rviz2 is a 3D visualization tool that is highly compatible with robot platforms based on the ROS2 software framework. In Rviz2, Extensible Markup Language (XML) can be used to describe the size, mass, position, material, joints, and other attributes of any physical object, such as the robot and surrounding objects, and present them in the interface. Simultaneously, Rviz2 can also graphically display real-time information from robot sensors, the robot's motion status, and changes in the surrounding environment. In short, Rviz2 provides users with a graphical display of all monitorable information through robot model parameters, sensor information published by the robot, and other data. Users and developers can also control the robot's behavior through buttons, sliders, and numerical values in the Rviz2 control interface. Most of the time, after data collection is complete, the data is directly published through ROS2 and then displayed using Rviz2.
[0046] However, the existing Rviz2 suffers from poor data compatibility and limited functionality, rarely utilizing Qt plugins to improve extensibility and interface aesthetics. Qt is a cross-platform graphical user interface application development framework. Furthermore, current technologies rarely present video data, point cloud data, and key parameters of interest to professionals (such as vehicle driving data) simultaneously within a single software interface. This prevents the software from meeting the needs of both professionals and general users viewing the data concurrently. Especially during developer debugging, insufficient data presentation hinders software improvement and the ability to respond to user feedback, impacting both developer efficiency and user experience.
[0047] To address the aforementioned technical issues, the solution provided in this disclosure allows data visualization tools to be compatible with multiple data types through the conversion of preset structure formats; and allows the display of multiple data types within the data visualization tool using plugins; thus enabling the data visualization tool to present sufficient data, thereby improving the software debugging efficiency of developers and enhancing the user experience to a certain extent.
[0048] The technical solutions of this disclosure and how they solve the aforementioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this disclosure will now be described with reference to the accompanying drawings.
[0049] Figure 1 This is a schematic flowchart illustrating a visualization method based on multi-type vehicle data, as shown in an exemplary embodiment of this disclosure. The visualization method based on multi-type vehicle data provided in this embodiment can be applied to electronic devices.
[0050] like Figure 1 As shown, the visualization method based on multi-type vehicle data provided in this embodiment includes:
[0051] Step 101: Receive vehicle data of various data types sent by the vehicle.
[0052] The method provided in this disclosure can be executed by an electronic device with computing capabilities. This electronic device is capable of acquiring vehicle data sent by the vehicle.
[0053] Specifically, vehicles can collect vehicle data and send it to electronic devices. Electronic devices can receive the vehicle data sent by the vehicles.
[0054] Specifically, data can be transmitted between vehicles and electronic devices via the Transmission Control Protocol / Internet Protocol (TCP / IP).
[0055] Vehicle data can include various data types, such as different types of video data, point cloud data, and text data. Video data can be acquired through cameras installed in the vehicle. Point cloud data can be acquired through radar installed in the vehicle. Text data can include vehicle driving data, which can be collected from the vehicle's bus system.
[0056] Step 102: Convert vehicle data whose data type conforms to the first data type condition into first data in a preset structure format; wherein, the preset structure format is a data format for visual conversion of vehicle data.
[0057] The first data type condition can include vehicle data of various types, such as multiple point cloud data, including lane line data and odometer data. Lane line data is used to represent lane lines in the vehicle's external environment. Odometer data is used to represent the vehicle's historical driving trajectory. Specifically, both lane line data and odometer data can be presented in point cloud data format.
[0058] The preset structure format is the data format used for visualizing vehicle data. The preset structure format can be a structure format compatible with ROS2.
[0059] Specifically, vehicle data whose data type conforms to the first data type condition can be converted into first data in a structure format adapted to ROS2.
[0060] Step 103: Display the first data and the second data from multiple types of vehicle data on the same interface; wherein the data type of the second data conforms to the second data type condition; wherein the first data type condition includes at least one data type, and the second data type condition includes at least one data type; all data types included in the first data type condition are different from all data types included in the second data type condition, or some data types in the first data type condition are the same as some data types in the second data type condition.
[0061] Specifically, Rviz2 can be used to display the first data and the second data from multiple types of vehicle data within the same Rviz2 interface.
[0062] Specifically, Rviz2 can recognize and adapt to ROS2 structure formats.
[0063] The second data type condition can include vehicle data of various types, such as video data, point cloud data, and text data.
[0064] Specifically, the first data type condition includes at least one data type, such as at least one point cloud data, like lane line data, odometer data, etc.
[0065] Specifically, the second data type condition includes at least one data type, such as at least one of multiple types of video data, multiple types of point cloud data, and text data. Video data can include image data and lane line data; point cloud data can include lane line data and odometer data; and text data can include vehicle driving data. Image data represents images of the vehicle's external environment acquired by cameras installed in the vehicle; vehicle driving data can include information such as speed and acceleration during vehicle movement. Specifically, some data types can be presented in either video data or point cloud data form, such as lane line data.
[0066] Specifically, the data types included in the first data type condition can all be different from the data types included in the second data type condition, or some data types can be the same; these can be preset according to actual needs.
[0067] Specifically, you can use Rviz2 to display the first data.
[0068] Specifically, pre-configured plugins can be used to display video data, text data, etc., that cannot be processed by Rviz2. Specifically, plugins can be used to invoke Rviz2, displaying the secondary data in the plugin window within the Rviz2 interface.
[0069] Specifically, Rviz2 can include multiple windows on the same interface, such as displaying first data in one window and second data in another. Specifically, the same Rviz2 interface can display first data and second data from multiple types of vehicle data. The visualization display method based on multiple types of vehicle data provided in this disclosure includes: receiving vehicle data of multiple data types sent by a vehicle; converting vehicle data whose data type conforms to a first data type condition into first data in a preset structure format; wherein the preset structure format is the data format for visual conversion of vehicle data; displaying the first data and second data from multiple types of vehicle data on the same interface; wherein the data type of the second data conforms to a second data type condition; wherein the first data type condition includes at least one data type, and the second data type condition includes at least one data type; the data types included in the first data type condition are all different from the data types included in the second data type condition, or some data types in the first data type condition are the same as some data types in the second data type condition. The method disclosed herein allows data visualization tools to be compatible with various data types through preset structure format conversion; it also enables the display of multiple data types within the data visualization tool using plugins; this allows the data visualization tool to present sufficient data, thereby improving software debugging efficiency for developers and enhancing user experience to some extent. Furthermore, this solution can unify video data, point cloud data, and key parameters of interest to professionals (such as vehicle driving data) and present them to users simultaneously within a single software interface. This allows the software to meet the needs of both professionals and general users viewing the data concurrently.
[0070] Figure 2 This is a schematic flowchart illustrating a visualization method based on multi-type vehicle data, as shown in another exemplary embodiment of this disclosure. The visualization method based on multi-type vehicle data provided in this embodiment can be applied to electronic devices.
[0071] like Figure 2 As shown, the visualization method based on multi-type vehicle data provided in this embodiment includes:
[0072] Step 201: Receive vehicle data of various data types sent by the vehicle.
[0073] Specifically, the principle and implementation of step 201 are similar to those of step 101, and will not be repeated here.
[0074] Specifically, after step 201, you can execute either step 202 or step 204.
[0075] Step 202: Cache vehicle data whose data type meets the first data type condition into the first queue.
[0076] The first data type condition can include vehicle data of various types, such as multiple point cloud data, lane line data, odometer data, etc.
[0077] The first queue can be a queue that is pre-set according to the actual situation.
[0078] Specifically, vehicle data whose data type matches the first data type condition can be cached in the first queue.
[0079] In one possible implementation, there is a one-to-one correspondence between each data type that meets the conditions of the first data type and each first queue.
[0080] Specifically, the first data type condition can include vehicle data of various types. Multiple first queues can be preset. Each data type can correspond one-to-one with each first queue.
[0081] Specifically, the method adopted in this solution can directly cache vehicle data that meets the first data type condition in the first queue, instead of storing it in shared memory, thus avoiding the risk of data being overwritten if stored in shared memory.
[0082] Step 203: Read vehicle data whose data type conforms to the first data type condition from the first queue, and convert the vehicle data whose data type conforms to the first data type condition into first data in a preset structure format; wherein, the preset structure format is a data format for visual conversion of vehicle data.
[0083] Specifically, the preset structure format is a data format for visualizing and converting vehicle data.
[0084] In one possible implementation, the preset structure format is a structure format that conforms to the data format of the robot operating system.
[0085] Specifically, vehicle data whose data type conforms to the first data type condition can be read from each first queue first, and then the read vehicle data whose data type conforms to the first data type condition can be converted into first data in a structure format adapted to ROS2.
[0086] In one possible implementation, vehicle data whose data type conforms to the first data type condition is read from a first queue in parallel by multiple threads.
[0087] Specifically, multiple threads can be pre-configured according to the actual situation. These threads then read vehicle data that conforms to the first data type condition from each first queue in parallel.
[0088] Optionally, a thread can be pre-set to read vehicle data that conforms to the first data type condition from each first queue in an asynchronous manner.
[0089] After reading data from the first queue, the various types of vehicle data that conform to the first data type condition can be converted into the first data in a preset structure format by multiple threads in parallel.
[0090] Specifically, multiple threads can be pre-set according to the actual situation. Based on these multiple threads, vehicle data of various types that conform to the first data type condition can be converted into first data in a preset structure format in parallel.
[0091] Optionally, a thread can be pre-set to asynchronously convert vehicle data of various types that conform to the first data type condition into the first data in a preset structure format.
[0092] Specifically, vehicle data whose data type conforms to the first data type condition is read from each first queue in parallel; the vehicle data of each type whose data type conforms to the first data type condition is converted into first data in a preset structure format in parallel; this can improve data processing efficiency compared to the single-line method.
[0093] In one possible implementation, the first data includes at least one of the following: lane line data, odometer data, and target obstacle data.
[0094] Lane line data is used to represent lane lines in the vehicle's external environment. Odometer data is used to represent the vehicle's historical driving trajectory. Target obstacle data can be used to represent obstacles ahead of the vehicle; these obstacles may include pedestrians, vehicles, etc.
[0095] Specifically, lane line data, odometer data, and target obstacle data can all be presented in the form of point cloud data.
[0096] Furthermore, after converting vehicle data conforming to the first data type condition into first data in a preset structure format, a publisher (pub) mechanism can be used to publish the data. Rviz2 can then act as a subscriber (Sub) to receive the published first data.
[0097] Specifically, step 206 can be executed after step 203.
[0098] Step 204: Store the second data in the shared memory of the electronic device; wherein the data type of the second data conforms to the conditions of the second data type.
[0099] The second data type condition can include vehicle data of various types, such as video data, point cloud data, and text data.
[0100] Specifically, shared memory can be pre-allocated. Second data whose data type conforms to the second data type condition is stored in the shared memory of the electronic device.
[0101] In one possible implementation, the second data includes at least one of the following: image data, drivable area data, lane line data, target detection box data, odometer data, target obstacle data, speed limit sign data, and vehicle driving data.
[0102] Among them, image data can be used to characterize image information of the vehicle's external environment acquired by cameras installed in the vehicle; drivable area data can be used to characterize the vehicle's drivable area information; target detection box data may include, for example, information related to billboards in the detected external environment of the vehicle; speed limit sign data may include, for example, information about speed limit signs.
[0103] Specifically, the second data may include text data, which can include vehicle driving data and vehicle attribute information. Vehicle attribute information may include, for example, the vehicle's weight, volume, and center point. The text data may also include text information corresponding to the image data, drivable area data, lane line data, target detection box data, odometer data, target obstacle data, and speed limit sign data in the second data. For example, it may include pixel position data representing the location of target obstacles.
[0104] Specifically, the second data can be displayed in the form of point cloud data, video data, or text data, according to preset settings.
[0105] Step 205: Read the second data from the shared memory and cache the read second data in the second queue.
[0106] Specifically, the data in the shared memory is refreshed in real time. In order to avoid data loss, the second data can be read from the shared memory in real time and cached in the second queue.
[0107] Specifically, the second data can include various data types, and multiple second queues can be pre-set, with each data type corresponding to one second queue.
[0108] Step 206: Read the second data from the second queue in parallel using multiple threads, and display the first data and the currently read second data on the same interface; wherein, the first data type condition includes at least one data type, and the second data type condition includes at least one data type; the data types included in the first data type condition are all different from the data types included in the second data type condition, or, some data types in the first data type condition are the same as some data types in the second data type condition.
[0109] Specifically, multiple threads can be pre-set, and second data can be read from each second queue in parallel based on these threads. Alternatively, a single thread can be pre-set to read second data from each second queue asynchronously.
[0110] After reading the second data from each of the second queues, a pre-defined plugin can be used to display the currently read second data in a window within the Rviz2 interface. Simultaneously, the first data can be displayed in other windows within the same Rviz2 interface.
[0111] Specifically, the first data type condition includes at least one data type, and the second data type condition includes at least one data type. Furthermore, the data types included in the first data type condition and the data types included in the second data type condition can all be different, or some data types can be the same; these can be preset according to actual needs.
[0112] like Figure 3 The image shown is a display interface of Rviz2. Figure 3 The left window displays the second data; the right window displays the first data. Figure 3 The bottom of the two windows on the left and right displays text data, which can be used to display vehicle driving information, vehicle attribute information, and text data corresponding to the data displayed in the left window. For example, Figure 3 As shown, the second data may include image data, lane line data, target obstacle data, and text data. For example... Figure 3 As shown, the first data may include lane line data and target obstacle data.
[0113] In one possible implementation, based on a preset correspondence between controls and data, the data corresponding to each control on the interface is determined from the first data and the currently read second data.
[0114] Specifically, controls can be pre-defined on the RViz2 interface. The correspondence between controls and data can also be pre-defined. Specifically, the data corresponding to each control on the interface can be determined from the first data and the currently read second data. This, in turn, improves the aesthetics of the interface.
[0115] In response to the user's display command, the data indicated by the display command is displayed on the controls of the interface according to the data corresponding to each control on the interface; wherein, the display command is used to indicate the data to be displayed selected by the user.
[0116] Specifically, users can select the data to be displayed based on their actual needs and generate display commands. The electronic device can respond to the user's display commands and display the data indicated by the display commands on the controls of the interface, based on the data corresponding to each control on the interface.
[0117] Figure 4 This is a flowchart illustrating a visualization method based on multi-type vehicle data, as another exemplary embodiment of this disclosure. The visualization method based on multi-type vehicle data provided in this embodiment can be applied to vehicles.
[0118] like Figure 4 As shown, the visualization method based on multi-type vehicle data provided in this embodiment includes:
[0119] Step 401: Obtain vehicle data of multiple data types; wherein, vehicle data whose data type conforms to the first data type condition is used to convert into first data in a preset structure format; the preset structure format is a data format for visual conversion of vehicle data; the first data and the second data from the multiple data types are used to display on the same interface; the data type of the second data conforms to the second data type condition; the first data type condition includes at least one data type, and the second data type condition includes at least one data type; the data types included in the first data type condition are all different from the data types included in the second data type condition, or, some data types in the first data type condition are the same as some data types in the second data type condition.
[0120] Specifically, step 401 can be referred to in the above embodiments, and will not be repeated here.
[0121] Step 402: Send vehicle data of various data types to the electronic device.
[0122] Specifically, vehicles can send vehicle data of various types collected to electronic devices.
[0123] Figure 5This is a schematic flowchart illustrating a visualization method based on multiple types of vehicle data, as another exemplary embodiment of this disclosure. The visualization method based on multiple types of vehicle data provided in this embodiment can be applied to vehicles.
[0124] like Figure 5 As shown, the visualization method based on multi-type vehicle data provided in this embodiment includes:
[0125] Step 501: Obtain the initial vehicle data collected by the acquisition unit in the vehicle, and convert the initial vehicle data into data that conforms to the preset transmission protocol to obtain the converted initial vehicle data.
[0126] Specifically, initial vehicle data can include various types of data, such as video data, point cloud data, and text data. Video data can be acquired through cameras installed in the vehicle. Point cloud data can be acquired through radar installed in the vehicle. Text data can include vehicle driving data, which can be collected from the vehicle's bus system.
[0127] Specifically, middleware technology can be used to convert the initial vehicle data into data that conforms to a preset transmission protocol, thereby obtaining the converted initial vehicle data.
[0128] Step 502: Filter the converted initial vehicle data to remove abnormal data, resulting in vehicle data of various data types. Vehicle data whose data types conform to the first data type condition is used to convert into first data in a preset structure format. The preset structure format is a data format for visual conversion of vehicle data. The first data and second data from the various data types are displayed on the same interface. The data type of the second data conforms to the second data type condition. The first data type condition includes at least one data type, and the second data type condition includes at least one data type. All data types included in the first data type condition are different from all data types included in the second data type condition, or some data types in the first data type condition are the same as some data types in the second data type condition.
[0129] For example, if the converted initial vehicle data includes temperature data, this temperature data can be filtered to remove outlier data with temperatures exceeding 1000 degrees Celsius. Similarly, other types of data included in the converted initial vehicle data can be filtered to remove outlier data.
[0130] In one possible implementation, vehicle data whose data type conforms to the first data type condition is cached in a first queue and then read to be converted into first data in a preset structure format.
[0131] In one possible implementation, there is a one-to-one correspondence between each data type that meets the conditions of the first data type and each first queue.
[0132] In one possible implementation, the second data is stored in the shared memory of the electronic device and then read to be cached in a second queue.
[0133] In one possible implementation, the preset structure format is a structure format that conforms to the data format of the robot operating system.
[0134] In one possible implementation, the first data includes at least one of the following: lane line data, odometer data, and target obstacle data; the second data includes at least one of the following: image data, drivable area data, lane line data, target detection box data, odometer data, target obstacle data, speed limit sign data, and vehicle driving data.
[0135] Specifically, the detailed process of step 502 can be found in the above embodiments and will not be repeated here.
[0136] Step 503: Send vehicle data of various data types to the electronic device.
[0137] Specifically, the principles and implementation methods of steps 503 and 402 are similar, and will not be repeated here.
[0138] Figure 6 This is a structural diagram illustrating a visualization display device based on multi-type vehicle data, as shown in an exemplary embodiment of this disclosure. The visualization display device based on multi-type vehicle data provided in this embodiment can be applied to electronic devices.
[0139] like Figure 6 As shown, the visualization display device 500 based on multi-type vehicle data provided in this disclosure includes:
[0140] The receiving unit 610 is used to receive vehicle data of various data types sent by the vehicle;
[0141] The format conversion unit 620 is used to convert vehicle data whose data type conforms to the first data type condition into first data in a preset structure format; wherein, the preset structure format is a data format for visual conversion of vehicle data;
[0142] Display unit 630 is used to display first data and second data from multiple types of vehicle data on the same interface; wherein the data type of the second data conforms to the conditions of a second data type.
[0143] The first data type condition includes at least one data type, and the second data type condition includes at least one data type; the data types included in the first data type condition are all different from the data types included in the second data type condition, or some data types in the first data type condition are the same as some data types in the second data type condition.
[0144] The format conversion unit 620 is specifically used to cache vehicle data whose data type conforms to the first data type condition into the first queue;
[0145] Read vehicle data whose data type conforms to the first data type condition from the first queue, and convert the vehicle data whose data type conforms to the first data type condition into the first data in the preset structure format.
[0146] In one possible implementation, there is a one-to-one correspondence between each data type that meets the conditions of the first data type and each first queue.
[0147] The format conversion unit 620 is specifically used to read vehicle data that conforms to the first data type condition from the first queue in parallel based on multiple threads;
[0148] Vehicle data whose data type conforms to the first data type condition is converted into first data in a preset structure format by multiple threads in parallel.
[0149] The format conversion unit 620 is also used to store the second data into the shared memory of the electronic device;
[0150] Read the second data from shared memory and cache the read second data in the second queue.
[0151] Display unit 630 is specifically used to read second data from the second queue in parallel based on multiple threads, and display the first data and the currently read second data on the same interface.
[0152] The display unit 630 is specifically used to determine the data corresponding to each control on the interface from the first data and the currently read second data, based on the preset correspondence between controls and data.
[0153] In response to the user's display command, the data indicated by the display command is displayed on the controls of the interface according to the data corresponding to each control on the interface; wherein, the display command is used to indicate the data to be displayed selected by the user.
[0154] In one possible implementation, the preset structure format is a structure format that conforms to the data format of the robot operating system.
[0155] In one possible implementation, the first data includes at least one of the following: lane line data, odometer data, and target obstacle data;
[0156] The second data includes at least one of the following: image data, drivable area data, lane line data, target detection box data, odometer data, target obstacle data, speed limit sign data, and vehicle driving data.
[0157] Figure 7 This is a structural diagram illustrating a visualization display device based on multi-type vehicle data, as shown in another exemplary embodiment of this disclosure. The visualization display device based on multi-type vehicle data provided in this embodiment can be applied to vehicles.
[0158] like Figure 7 As shown, the visualization display device 600 based on multi-type vehicle data provided in this disclosure includes:
[0159] The data acquisition unit 710 is used to acquire vehicle data of various types and send the vehicle data of various types to electronic devices.
[0160] Among them, vehicle data of various data types that conform to the conditions of the first data type are used to convert into first data in a preset structure format; the preset structure format is a data format for visual conversion of vehicle data;
[0161] The first data and the second data from various types of vehicle data are used to display on the same interface; the data type of the second data conforms to the second data type condition; the first data type condition includes at least one data type, and the second data type condition includes at least one data type; the data types included in the first data type condition are all different from the data types included in the second data type condition, or, some data types in the first data type condition are the same as some data types in the second data type condition.
[0162] The data acquisition unit 710 is specifically used to acquire the initial vehicle data collected by the acquisition unit in the vehicle, and convert the initial vehicle data into data that conforms to a preset transmission protocol to obtain the converted initial vehicle data.
[0163] The converted initial vehicle data is filtered to remove abnormal data, resulting in vehicle data of various data types.
[0164] In one possible implementation, vehicle data whose data type conforms to the first data type condition is cached in a first queue and then read to be converted into first data in a preset structure format.
[0165] In one possible implementation, there is a one-to-one correspondence between each data type that meets the conditions of the first data type and each first queue.
[0166] In one possible implementation, the second data is stored in the shared memory of the electronic device and then read to be cached in a second queue.
[0167] In one possible implementation, the preset structure format is a structure format that conforms to the data format of the robot operating system.
[0168] In one possible implementation, the first data includes at least one of the following: lane line data, odometer data, and target obstacle data;
[0169] The second data includes at least one of the following: image data, drivable area data, lane line data, target detection box data, odometer data, target obstacle data, speed limit sign data, and vehicle driving data.
[0170] Figure 8 This is a structural diagram of an electronic device illustrated in an exemplary embodiment of the present disclosure.
[0171] Figure 8 This is a structural diagram of an electronic device illustrated in an exemplary embodiment of the present disclosure.
[0172] like Figure 8 As shown, the electronic device provided in this embodiment includes:
[0173] Memory 801;
[0174] Processor 802; and
[0175] Computer programs;
[0176] The computer program is stored in memory 801 and configured to be executed by processor 802 to implement any of the above-mentioned visualization display methods based on multi-type vehicle data.
[0177] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement any of the above-described visualization display methods based on multi-type vehicle data.
[0178] This embodiment also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described visualization display methods based on multi-type vehicle data.
[0179] A visualization system based on multi-type vehicle data includes electronic equipment and vehicles; the electronic equipment is used to perform... Figure 1 , Figure 2 Any of the methods; the vehicle is used to perform Figure 4 , Figure 5 Any of the methods.
[0180] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A visualization method based on multi-type vehicle data, characterized in that, Applied to electronic devices, the method includes: Receive vehicle data of various data types sent by the vehicle; Vehicle data whose data type conforms to the first data type condition is converted into first data in a preset structure format. The preset structure format is a structure format that conforms to the data format of the robot operating system. The preset structure format is a data format for visual conversion of vehicle data. The second data is stored in the shared memory of the electronic device; wherein the data type of the second data conforms to the second data type condition; the second data is displayed using a pre-set plug-in; Read the second data from the shared memory and cache the read second data in the second queue; Vehicle data whose data type matches the first data type condition is cached in the first queue; Read vehicle data whose data type conforms to the first data type condition from the first queue, and convert the vehicle data whose data type conforms to the first data type condition into first data in a preset structure format; Each data type that meets the first data type condition corresponds one-to-one with each first queue; The second data is read from the second queue in parallel by multiple threads, and the first data and the currently read second data are displayed on the same interface; wherein the first data type condition includes at least one data type, and the second data type condition includes at least one data type; the data types included in the first data type condition are different from the data types included in the second data type condition, or some data types in the first data type condition are the same as some data types in the second data type condition.
2. The method according to claim 1, characterized in that, Read vehicle data whose data type conforms to the first data type condition from the first queue, and convert the vehicle data whose data type conforms to the first data type condition into first data in a preset structure format, including: Vehicle data that conforms to the first data type condition is read from the first queue in parallel by multiple threads; Vehicle data whose data type conforms to the first data type condition is converted into first data in a preset structure format by multiple threads in parallel.
3. The method according to claim 1, characterized in that, Display the first data and the currently read second data on the same interface, including: Based on the preset correspondence between controls and data, determine the data corresponding to each control on the interface from the first data and the currently read second data; In response to a user's display command, the data indicated by the display command is displayed on the controls of the interface according to the data corresponding to each control on the interface; wherein the display command is used to indicate the data to be displayed selected by the user.
4. The method according to any one of claims 1-3, characterized in that, The first data includes at least one of the following: lane line data, odometer data, and target obstacle data; The second data includes at least one of the following: image data, drivable area data, lane line data, target detection box data, odometer data, target obstacle data, speed limit sign data, and vehicle driving data.
5. A visualization method based on multi-type vehicle data, characterized in that, Applied to vehicles, the method includes: The system acquires vehicle data of various data types and sends the vehicle data of various data types to an electronic device, so that the electronic device stores the second data in the shared memory of the electronic device; reads the second data from the shared memory and caches the read second data in a second queue; displays the first data and the second data from the vehicle data of various data types on the same interface, including: reading the second data from the second queue in parallel based on multiple threads, and displaying the first data and the currently read second data on the same interface; Among them, vehicle data of the various data types that conform to the first data type condition are used to convert into first data in a preset structure format; the preset structure format is a structure format that conforms to the data format of the robot operating system; the preset structure format is a data format for visual conversion of vehicle data; The data type of the second data conforms to the second data type condition; the second data is displayed using a pre-set plugin; the first data type condition includes at least one data type, and the second data type condition includes at least one data type; the data types included in the first data type condition are different from the data types included in the second data type condition, or, some data types in the first data type condition are the same as some data types in the second data type condition; Vehicle data whose data type conforms to the first data type condition is cached in the first queue and then read to be converted into first data in a preset structure format. Each data type that meets the first data type condition corresponds one-to-one with each first queue.
6. The method according to claim 5, characterized in that, The acquisition of vehicle data of various data types includes: The vehicle initial data collected by the acquisition unit in the vehicle is obtained, and the vehicle initial data is converted into data that conforms to a preset transmission protocol to obtain the converted vehicle initial data; The converted initial vehicle data is filtered to remove abnormal data, resulting in vehicle data of various data types.
7. The method according to claim 5 or 6, characterized in that, The first data includes at least one of the following: lane line data, odometer data, and target obstacle data; The second data includes at least one of the following: image data, drivable area data, lane line data, target detection box data, odometer data, target obstacle data, speed limit sign data, and vehicle driving data.
8. A visualization display device based on multi-type vehicle data, used to implement the visualization display method based on multi-type vehicle data as described in claim 1, characterized in that, Applied to electronic devices, the device includes: The receiving unit is used to receive vehicle data of various data types sent by the vehicle. A format conversion unit is used to convert vehicle data whose data type conforms to a first data type condition into first data in a preset structure format, wherein the preset structure format is a structure format conforming to the data format of a robot operating system; wherein the preset structure format is a data format for visual conversion of vehicle data; a display unit is used to display the first data and second data from the vehicle data of the multiple data types on the same interface; wherein the data type of the second data conforms to a second data type condition; wherein the first data type condition includes at least one data type, and the second data type condition includes at least one data type; the data types included in the first data type condition are all different from the data types included in the second data type condition, or, some data types in the first data type condition are the same as some data types in the second data type condition.
9. A visualization display device based on multi-type vehicle data, used to implement the visualization display method based on multi-type vehicle data as described in claim 5, characterized in that, Applied to vehicles, the device includes: A data acquisition unit is used to acquire vehicle data of various data types and send the vehicle data of various data types to electronic devices; Among them, vehicle data of various data types that conform to the first data type condition are used to convert into first data in a preset structure format; the preset structure format is a structure format that conforms to the data format of the robot operating system; the first data and the second data of the various data types are used to display on the same interface; the data type of the second data conforms to the second data type condition; the first data type condition includes at least one data type, and the second data type condition includes at least one data type; the data types included in the first data type condition are different from the data types included in the second data type condition, or, some data types in the first data type condition are the same as some data types in the second data type condition.
10. An electronic device, characterized in that, include: Memory and processor; among which, The memory is used to store computer programs; The processor is configured to read a computer program stored in the memory and execute the method described in any one of claims 1-4 according to the computer program in the memory.
11. A vehicle, characterized in that, include: Memory and processor; among which, The memory is used to store computer programs; The processor is configured to read a computer program stored in the memory and execute the method described in any one of claims 5-7 according to the computer program in the memory.
12. A visualization display system based on multi-type vehicle data, characterized in that, This includes the electronic device as described in claim 10, or the vehicle as described in claim 11.
13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method described in any one of claims 1-4 or 5-7.
14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-4 or 5-7.
Citation Information
Patent Citations
A multi-dimensional decision analysis and visualization system for multi-source heterogeneous data
CN109947737A
Multi-data-source data visualization system
CN112434102A
Automatic driving log data processing method and device
CN113987303A
