Intelligent driving data processing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to an intelligent driving data processing method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: under the condition that a data conversion unit in the vehicle-mounted intelligent driving system receives intelligent driving data pushed by a target function unit in the intelligent driving system, performing conversion processing on the intelligent driving data according to a target data conversion strategy matched with the data category of the intelligent driving data, obtaining a data frame in a target data format according to the converted data; pushing the data frame to a display device according to the first pushing frequency; the data frame is used for indicating the display equipment to adopt a visualization processing strategy matched with the target data format to carry out visualization processing on the data frame to obtain visualization data, and the visualization data is displayed through the browser page. By adopting the method, the data visualization cost can be reduced.

Description

Technical Field

[0001] The present application relates to the field of data visualization technology, and in particular to an intelligent driving data processing method, device, computer equipment, computer-readable storage medium and computer program product. Background Art

[0002] Intelligent driving refers to the technology of machines helping people to drive, and completely replacing human driving in special circumstances. With the continuous development of intelligent driving technology, more and more assisted driving functions and even intelligent driving functions are coming out. Due to the time series and complexity of intelligent driving data, visual analysis and display of these data is an extremely important part of intelligent driving development and testing, which can greatly improve the efficiency of intelligent driving data analysis and accelerate the problem location and function iteration of intelligent driving algorithms. Due to the time series, complexity and diversity of intelligent driving data, how to efficiently and accurately visualize these data has also become a matter of great concern to developers and testers in the field of intelligent driving.

[0003] ROS (Robot Operating System) is an open source robot operating system that provides a series of software libraries and tools to help developers create, deploy and manage robot applications. ROS is widely used in various robotics fields, including industrial robots, service robots, unmanned vehicles, etc. In ROS, the most commonly used visualization tool is RVIZ (ROS Visualization) that comes with the ROS toolbox. RVIZ is a 3D visualization-based tool in ROS that is used to visualize the perception, status, and operation of the robot. It provides a user interface that can display the robot model, sensor data, motion planning results, etc. in real time.

[0004] In traditional technology, RVIZ is usually used for data visualization. The biggest problem is its strong platform dependency. It relies on a complete set of ROS environment, which can only be installed in the Linux operating system and has specific requirements for the Linux version. The installation is difficult, time-consuming, and resource-intensive. This results in data visualization being restricted by the system and software and being non-portable. Users need to install the corresponding system and software to achieve data visualization, and the cost of data visualization is high. Summary of the invention

[0005] Based on this, it is necessary to provide a data visualization method, device, computer equipment, computer-readable storage medium and computer program product that can improve the efficiency of intelligent driving data visualization in response to the above-mentioned technical problems.

[0006] In a first aspect, the present application provides an intelligent driving data processing method, which is applied to a data conversion unit in a vehicle-mounted intelligent driving system; the method comprises:

[0007] When receiving the intelligent driving data pushed by the target functional unit in the intelligent driving system, converting the intelligent driving data according to the target data conversion strategy matching the data category of the intelligent driving data, and obtaining a data frame in the target data format according to the converted data;

[0008] The data frame is pushed to the display device at a first push frequency; the data frame is used to instruct the display device to adopt a visualization processing strategy that matches the target data format, perform visualization processing on the data frame, obtain visualization data, and display the visualization data through a browser page.

[0009] In one embodiment, the method further comprises:

[0010] Determine a target functional unit among multiple functional units of the intelligent driving system according to data content that needs to be visualized in the intelligent driving control process;

[0011] According to the target functional unit, subscription information is constructed; the subscription information is used to instruct the target functional unit to push the intelligent driving data to the data conversion unit according to the second push frequency.

[0012] In one embodiment, the intelligent driving data is converted according to a target data conversion strategy that matches the data category of the intelligent driving data, and a data frame in a target data format is obtained according to the converted data, including:

[0013] A target data conversion strategy matching the data category of the intelligent driving data is obtained from a plurality of preset data conversion strategies, and the target data conversion strategy is used to perform format conversion on the intelligent driving data to obtain data to be combined in the target data format;

[0014] According to the message subject of the data to be combined, determining the writing position of the data to be combined in the data file of the target data format; the data file of the target data format is provided with a plurality of writing positions, and each writing position corresponds to a message subject;

[0015] The data to be combined are written into a data file in a target data format according to the writing position of the data to be combined, so as to obtain a data frame in the target data format.

[0016] In one embodiment, a target data conversion strategy matching the data category of the intelligent driving data is obtained, and the target data conversion strategy is used to convert the format of the intelligent driving data to obtain data to be combined in the target data format, including:

[0017] When the data category of the intelligent driving data is neither image data nor point cloud data, the intelligent driving data is format converted according to the first data conversion strategy to obtain data to be combined in JSON format.

[0018] In one embodiment, a target data conversion strategy matching the data category of the intelligent driving data is obtained, and the target data conversion strategy is used to convert the format of the intelligent driving data to obtain data to be combined in the target data format, including:

[0019] When the data category of the intelligent driving data is image data or point cloud data, the intelligent driving data is format converted according to the second data conversion strategy to obtain data to be combined in JPG format or PCD format.

[0020] In one embodiment, the acquisition methods in the plurality of preset data conversion strategies include:

[0021] Determining at least one data format that can be processed by the browser according to development information of the browser of the display device;

[0022] Determine the original data format of the intelligent driving data, obtain the data conversion strategy corresponding to each data format according to the format conversion relationship between the original data format and each data format, and obtain multiple preset data conversion strategies.

[0023] In a second aspect, the present application further provides an intelligent driving data processing device, which is applied to a data conversion unit in a vehicle-mounted intelligent driving system; the device comprises:

[0024] a processing module, configured to, upon receiving the intelligent driving data pushed by the target functional unit in the intelligent driving system, convert the intelligent driving data according to a target data conversion strategy matching the data category of the intelligent driving data, and obtain a data frame in a target data format according to the converted data;

[0025] The push module is used to push the data frame to the display device according to the first push frequency; the data frame is used to instruct the display device to adopt a visualization processing strategy that matches the target data format, perform visualization processing on the data frame, obtain visualization data, and display the visualization data through a browser page.

[0026] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0027] When receiving the intelligent driving data pushed by the target functional unit in the intelligent driving system, converting the intelligent driving data according to the target data conversion strategy matching the data category of the intelligent driving data, and obtaining a data frame in the target data format according to the converted data;

[0028] The data frame is pushed to the display device at a first push frequency; the data frame is used to instruct the display device to adopt a visualization processing strategy that matches the target data format, perform visualization processing on the data frame, obtain visualization data, and display the visualization data through a browser page.

[0029] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0030] When receiving the intelligent driving data pushed by the target functional unit in the intelligent driving system, converting the intelligent driving data according to the target data conversion strategy matching the data category of the intelligent driving data, and obtaining a data frame in the target data format according to the converted data;

[0031] The data frame is pushed to the display device at a first push frequency; the data frame is used to instruct the display device to adopt a visualization processing strategy that matches the target data format, perform visualization processing on the data frame, obtain visualization data, and display the visualization data through a browser page.

[0032] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0033] When receiving the intelligent driving data pushed by the target functional unit in the intelligent driving system, converting the intelligent driving data according to the target data conversion strategy matching the data category of the intelligent driving data, and obtaining a data frame in the target data format according to the converted data;

[0034] The data frame is pushed to the display device at a first push frequency; the data frame is used to instruct the display device to adopt a visualization processing strategy that matches the target data format, perform visualization processing on the data frame, obtain visualization data, and display the visualization data through a browser page.

[0035] The above-mentioned intelligent driving data processing method, device, computer equipment, computer-readable storage medium and computer program product, when the data conversion unit in the vehicle-mounted intelligent driving system receives the intelligent driving data pushed by the target functional unit in the intelligent driving system, converts and processes the intelligent driving data according to the target data conversion strategy that matches the data category of the intelligent driving data, and obtains a data frame in the target data format according to the converted data; pushes the data frame to the display device according to the first push frequency; the data frame is used to instruct the display device to adopt a visualization processing strategy that matches the target data format, visualizes the data frame, obtains visualization data, and displays the visualization data through a browser page. Using a browser to replace the original RVIZ tool that is strongly dependent on the platform increases the portability, platform independence and statelessness of data visualization. The real-time visualization software can be opened quickly and easily on various devices and operating systems, without or only relying on a small amount of software, greatly reducing the cost of data visualization. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0037] FIG1 is a diagram of an application environment of an intelligent driving data processing method according to an embodiment;

[0038] FIG2 is a schematic diagram of a flow chart of an intelligent driving data processing method in one embodiment;

[0039] FIG3 is a schematic diagram of an implementation method of an intelligent driving data processing method in an embodiment;

[0040] FIG4 is a schematic diagram of an implementation method of converting to obtain a JSON data frame in one embodiment;

[0041] FIG5 is a schematic diagram of an implementation method for converting to obtain a JPG or PCD data frame in one embodiment;

[0042] FIG6 is a schematic diagram of an implementation method of a data conversion unit for a single visualization window in one embodiment;

[0043] FIG7 is a schematic diagram of an implementation method of a data conversion unit for multiple visualization windows in one embodiment;

[0044] FIG8 is a structural block diagram of an intelligent driving data processing device in one embodiment;

[0045] FIG. 9 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] The intelligent driving data processing method provided in the embodiment of the present application can be applied in the application environment shown in FIG1. ​​Among them, the data conversion unit 102 in the vehicle-mounted intelligent driving system communicates with the target function unit 104 and the display device 106 in the vehicle-mounted intelligent driving system through the network, and the data conversion unit 102 and the target function unit 104 can be configured in the same terminal or in different terminals. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the vehicle-mounted terminal can be, but is not limited to, various vehicle-mounted computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc.

[0048] In an exemplary embodiment, as shown in FIG2 , a method for processing intelligent driving data is provided, which is described by taking the method applied to the data conversion unit 102 in FIG1 as an example, and includes the following steps 202 to 204 . Among them:

[0049] Step 202, when receiving the intelligent driving data pushed by the target functional unit in the intelligent driving system, convert the intelligent driving data according to the target data conversion strategy matching the data category of the intelligent driving data, and obtain a data frame in the target data format according to the converted data.

[0050] Among them, the intelligent driving system can be but not limited to the vehicle-mounted ROS system. The intelligent driving data can be but not limited to ROSBAG data, which is a tool in ROS for recording and replaying data. It can save the data of the message topic in the ROS system to a file and then replay the data when needed. The target data format can be but not limited to JSON format and PCD format, JSON is a data storage format similar to text type, and PCD is a format of laser radar point cloud data. Data categories include but are not limited to text data, image data, and point cloud data.

[0051] Optionally, the data conversion unit receives the ROSBAG data pushed by the target functional unit, and processes the ROSBAG data using a target data conversion strategy that matches the data category from a pre-set data conversion strategy, and converts the ROSBAG raw data into a form similar to a video frame. Different data conversion strategies will convert ROSBAG data of different data categories into data frames of different data formats.

[0052] Step 204, push the data frame to the display device at the first push frequency; the data frame is used to instruct the display device to adopt a visualization processing strategy that matches the target data format, perform visualization processing on the data frame, obtain visualization data, and display the visualization data through a browser page.

[0053] Optionally, the data conversion unit transmits the file to the display device in real time through the WebSocket technology according to the pre-set first push frequency, and the display device calls the local browser to visualize the data frame and display the visualized data on the browser page. Among them, Websocket is a way for the browser and the server to communicate in real time. The format of the content in the data frame needs to be negotiated and determined with the browser front end, based on the principle that each data frame file is a complete frame of data, containing the data of all topics at the current time.

[0054] In the above-mentioned intelligent driving data processing method, when the data conversion unit in the vehicle-mounted intelligent driving system receives the intelligent driving data pushed by the target functional unit in the intelligent driving system, it converts and processes the intelligent driving data according to the target data conversion strategy that matches the data category of the intelligent driving data, and obtains a data frame in the target data format according to the converted data; pushes the data frame to the display device according to the first push frequency; the data frame is used to instruct the display device to adopt a visualization processing strategy that matches the target data format, performs visualization processing on the data frame, obtains visualization data, and displays the visualization data through a browser page. Using a browser to replace the original RVIZ tool that is strongly dependent on the platform increases the portability, platform independence and statelessness of data visualization. The real-time visualization software can be opened quickly and easily on various devices and operating systems, without or only relying on a small amount of software, greatly reducing the cost of data visualization.

[0055] In one embodiment, the method also includes: determining a target functional unit among multiple functional units of the intelligent driving system according to the data content that needs to be visualized during the intelligent driving control process; constructing subscription information according to the target functional unit; the subscription information is used to instruct the target functional unit to push the intelligent driving data to the data conversion unit according to the second push frequency.

[0056] Optionally, there are two roles in the ROS system: publisher and subscriber. The publisher can publish several topics, and the subscriber can subscribe to several topics. As shown in Figure 3, in the intelligent driving system, the target functional unit is equivalent to the publisher, and the data conversion unit is equivalent to the subscriber. Each target functional unit will act as a publisher to send multiple topic data in real time. On the one hand, the data conversion unit will act as a subscriber to subscribe to the topic data that needs to be visualized, convert the topic in real time and update the data frame. On the other hand, it will send the composed data frame to the browser of the display device through websocket in real time at a fixed frequency (such as 100ms) for visualization.

[0057] In this embodiment, according to the data content that needs to be visualized in the intelligent driving control process, a target functional unit is determined among the multiple functional units of the intelligent driving system, and subscription information is constructed based on the target functional unit, so that the data conversion unit can regularly obtain the intelligent driving data, and then convert the format of the intelligent driving data to facilitate subsequent data visualization processing by the browser, thereby improving the efficiency of data visualization.

[0058] In one embodiment, the intelligent driving data is converted and processed according to a target data conversion strategy that matches the data category of the intelligent driving data, and a data frame in a target data format is obtained according to the converted data, including: obtaining a target data conversion strategy that matches the data category of the intelligent driving data from a plurality of preset data conversion strategies, and when the data category of the intelligent driving data is not image data and is not point cloud data, the intelligent driving data is format converted according to a first data conversion strategy to obtain data to be combined in JSON format. According to the message subject of the data to be combined, the write position of the data to be combined is determined in the data file in the target data format; a plurality of write positions are provided in the data file in the target data format, and each write position corresponds to a message subject; according to the write position of the data to be combined, the data to be combined is written into the data file in the target data format to obtain a data frame in the target data format.

[0059] Among them, the message subject refers to the topic. In ROS, the topic is a mechanism for message delivery and communication. Topic is a publish / subscribe (pub / sub) model in ROS, which allows different nodes to communicate by sending and receiving messages. In ROS, a Topic can be seen as a pipeline for data flow, where one node (publisher) publishes messages to a Topic, and other nodes (subscribers) can receive these messages by subscribing to the same Topic.

[0060] Optionally, when the data category of the intelligent driving data is neither image data nor point cloud data, all topic data contained in each frame is recorded in the form of a JSON file, and the frame rate is controlled by the backend, which synthesizes the JSON frame data at a fixed frequency. The format of the content in the JSON file needs to be negotiated and determined with the browser frontend, based on the principle that each JSON file is a complete frame of data, containing data for all topics at the current time.

[0061] In order to ensure that the JSON data can be visualized on the browser side, the format of the JSON data needs to be communicated and determined with the browser web developer. Here we only explain several important data conversion forms. The overall coordinates are uniformly based on the vehicle as the origin, the vehicle's forward direction as the positive direction of the x-axis, and the y-axis perpendicular to the vehicle's forward direction (positive on the left and negative on the right). The rest of the visualization data is drawn with coordinates relative to the vehicle. Therefore, the first step is to determine the position of the vehicle on the web page. Here we decide to directly use the longitude and latitude of the vehicle as the position of the vehicle. In addition to longitude and latitude, Euler angles are also needed to determine the body posture information of the vehicle. Since only the quaternion data of the vehicle is given in the original data, the quaternion needs to be converted into Euler angles using the general formula shown in Figure 3.

[0062]

[0063] For targets other than the ego vehicle, coordinate conversion is also required. The original data uses the universal ENU coordinate system, that is, the global coordinate system with the power-on position as the origin. Therefore, the ENU coordinates of the target object in the current frame need to be used, combined with the aligned ego vehicle ENU coordinates and posture, to convert them into coordinates relative to the current ego vehicle. The conversion formula is shown in Figure 4.

[0064]

[0065] The coordinates of other targets are transformed in the same way. After all transformations are completed, the data are combined in the JSON file to obtain a data frame in JSON format.

[0066] For example, as shown in Figure 4, take the positioning result topic ( / mla / egopose) and the planning trajectory result topic ( / msd / planning / plan) as examples. The data conversion unit will subscribe to these two topics as a subscriber. As long as any topic has new data sent out, it will trigger the callback function in the conversion service. The callback function will convert the original ROS message data and update the conversion result to the frame.json data frame.

[0067] In this embodiment, data format conversion is performed before data visualization, and the converted data can be directly visualized using a browser. The browser is used to replace the original RVIZ tool that is strongly dependent on the platform, thereby increasing the portability, platform independence and statelessness of data visualization. The real-time visualization software can be opened quickly and easily on various devices and operating systems, without the need for or reliance on a small amount of software, greatly reducing the cost of data visualization.

[0068] In one embodiment, according to a target data conversion strategy that matches the data category of the intelligent driving data, the intelligent driving data is converted and processed, and a data frame in a target data format is obtained according to the converted data, including: obtaining a target data conversion strategy that matches the data category of the intelligent driving data from a plurality of preset data conversion strategies, and when the data category of the intelligent driving data is image data or point cloud data, according to a second data conversion strategy, the intelligent driving data is format converted to obtain data to be combined in a JPG format or a PCD format. According to the message subject of the data to be combined, the write position of the data to be combined is determined in a data file in a target data format; a plurality of write positions are provided in the data file in the target data format, and each write position corresponds to a message subject; according to the write position of the data to be combined, the data to be combined is written into the data file in the target data format to obtain a data frame in the target data format.

[0069] Optionally, when the data category of intelligent driving data is image data or point cloud data, all topic data contained in each frame are recorded in the form of JPG or PCD files. The frame rate is controlled by the backend, and the backend synthesizes JPG or PCD frame data at a fixed frequency. The format of the content in the JPG or PCD file needs to be negotiated and determined with the browser frontend. The principle is that each JPG or PCD file is a complete frame of data, containing data for all topics at the current time. Since JPG or PCD contains image or laser point cloud data, the file is large, which will cause visualization to freeze. Through multiple WebSocket channels, big data (such as images or point clouds) can be processed by a separate backend process and compressed and transmitted to the frontend in a specific way. This ensures that the data visualization of different channels does not affect each other and reduces freezes.

[0070] The most critical problem that needs to be solved for real-time data visualization is to ensure that the front-end visualization is not stuck. This patent also pioneered the concept of data channeling. The visualization data conversion mentioned above is to pack all the current data frames into a JSON, and store and transmit them in the form of a string, which even contains a lot of image data and laser point cloud data. Representing these two types of data with strings will increase the amount of data transmitted and the workload of visualization analysis. Therefore, the image data and laser point cloud data are separated from the JSON separately, and stored and transmitted through two other special types of files (.jpg and .pcd). On the one hand, it greatly reduces the amount of data transmission and the workload of front-end analysis, and improves the speed of analysis and visualization; on the other hand, in order to improve the efficiency of back-end conversion, a more efficient language such as C++ can be used to replace the functions that can only be implemented in python, as shown in Figure 5, where the lidar_converter node is equivalent to the data conversion module in this embodiment.

[0071] The specific implementation of the point cloud data conversion channel is shown in Figure 6. The lidar_converter node in Figure 5 is modularized into communication thread, data conversion thread, point cloud data subscription thread, and cache cleaning thread. The threads share data through memory cache, including the cache of point cloud data and the cache of PCD result data. Among them, the number of point cloud data subscription threads can be adjusted according to the actual number of laser radars that need to be visualized, and is controlled by startup parameters during actual use; the point cloud data subscription thread will store the received point cloud data in the corresponding point cloud data cache respectively; the data conversion thread is responsible for obtaining multi-channel point cloud data from the point cloud data cache, merging and converting them. In order to ensure the real-time visualization of data, the thread can control the final point cloud data compression ratio by receiving startup parameters according to the actual network bandwidth, so as to ensure that the network will not be congested and avoid a series of avalanche effects caused by network congestion; the data conversion thread will temporarily store the generated point cloud visualization data in the pcd cache; the thread responsible for communicating with the browser page will obtain the point cloud visualization data from the pcd cache at a fixed frequency and send it to the browser side in real time through websocket; in order to ensure the space utilization of the cache, the cache cleaning thread will regularly clean up the data of the two caches, and the cleaning method is controlled by the set cache expiration time.

[0072] The above is the implementation for a single front-end visualization web page. If there are multiple visualization windows at the same time, due to the independence of the communication thread, the communication thread can be dynamically expanded separately to meet the real-time visualization of multiple windows, as shown in Figure 7.

[0073] In this embodiment, data format conversion is performed before data visualization, and the converted data can be directly visualized using a browser. The browser is used to replace the original RVIZ tool that is strongly dependent on the platform, thereby increasing the portability, platform independence and statelessness of data visualization. The real-time visualization software can be opened quickly and easily on various devices and operating systems, without the need for or reliance on a small amount of software, greatly reducing the cost of data visualization.

[0074] In one embodiment, a method for obtaining multiple preset data conversion strategies includes: determining at least one data format that can be processed by the browser based on development information of the browser of the display device; determining the original data format of the intelligent driving data, and obtaining the data conversion strategy corresponding to each data format based on the format conversion relationship between the original data format and each data format, thereby obtaining multiple preset data conversion strategies.

[0075] Optionally, the JSON, JPG and PCD data formats that can be processed by the browser are determined based on the development information of the browser of the display device. The original data format of the intelligent driving data is determined to be ROSBAG data, and according to the format conversion relationship between ROSBAG data and JSON, JPG and PCD data formats, ROSBAG to JSON conversion strategy, ROSBAG to JPG conversion strategy and ROSBAG to PCD conversion strategy are established, and the ROSBAG to JPG conversion strategy is matched with image data, the ROSBAG to PCD conversion strategy is matched with point cloud data, and the ROSBAG to JSON conversion strategy is matched with other data such as text, so as to obtain multiple preset data conversion strategies.

[0076] In this embodiment, by presetting multiple data conversion strategies for different data types, data format conversion is performed before data visualization. The converted data can be directly visualized using a browser, and the browser is used to replace the original RVIZ tool that is strongly dependent on the platform. This increases the portability, platform independence and statelessness of data visualization. The real-time visualization software can be opened quickly and easily on various devices and operating systems, without the need for or reliance on a small amount of software, greatly reducing the cost of data visualization.

[0077] In one embodiment, a method for processing intelligent driving data is provided, which is applied to a data conversion unit in a vehicle-mounted intelligent driving system; the method comprises:

[0078] According to the development information of the browser of the display device, at least one data format that can be processed by the browser is determined; the original data format of the intelligent driving data is determined, and according to the format conversion relationship between the original data format and each data format, a data conversion strategy corresponding to each data format is obtained to obtain multiple preset data conversion strategies.

[0079] According to the data content that needs to be visualized in the intelligent driving control process, a target functional unit is determined among multiple functional units of the intelligent driving system; subscription information is constructed according to the target functional unit; the subscription information is used to instruct the target functional unit to push the intelligent driving data to the data conversion unit according to the second push frequency.

[0080] When receiving the intelligent driving data pushed by the target functional unit in the intelligent driving system, a target data conversion strategy matching the data category of the intelligent driving data is obtained from a plurality of preset data conversion strategies. When the data category of the intelligent driving data is neither image data nor point cloud data, the intelligent driving data is format-converted according to the first data conversion strategy to obtain the data to be combined in JSON format; when the data category of the intelligent driving data is image data or point cloud data, the intelligent driving data is format-converted according to the second data conversion strategy to obtain the data to be combined in JPG format or PCD format.

[0081] The target data conversion strategy is used to convert the format of the intelligent driving data to obtain the data to be combined in the target data format.

[0082] According to the message subject of the data to be combined, the writing position of the data to be combined is determined in the data file of the target data format; a plurality of writing positions are provided in the data file of the target data format, and each writing position corresponds to a message subject; according to the writing position of the data to be combined, the data to be combined is written into the data file of the target data format to obtain a data frame in the target data format.

[0083] The data frame is pushed to the display device at a first push frequency; the data frame is used to instruct the display device to adopt a visualization processing strategy that matches the target data format, perform visualization processing on the data frame, obtain visualization data, and display the visualization data through a browser page.

[0084] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0085] Based on the same inventive concept, the embodiment of the present application also provides an intelligent driving data processing device for implementing the intelligent driving data processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more intelligent driving data processing device embodiments provided below can refer to the limitations of the intelligent driving data processing method above, and will not be repeated here.

[0086] In an exemplary embodiment, as shown in FIG8 , an intelligent driving data processing device 800 is provided, including: a processing module 801 and a push module 802, wherein:

[0087] The processing module 801 is used to convert the intelligent driving data according to the target data conversion strategy matching the data category of the intelligent driving data when receiving the intelligent driving data pushed by the target functional unit in the intelligent driving system, and obtain a data frame in the target data format according to the converted data;

[0088] Push module 802 is used to push data frames to the display device at a first push frequency; the data frames are used to instruct the display device to adopt a visualization processing strategy that matches the target data format, perform visualization processing on the data frames, obtain visualization data, and display the visualization data through a browser page.

[0089] In one embodiment, the processing module 801 is also used to determine the target functional unit among multiple functional units of the intelligent driving system according to the data content that needs to be visualized during the intelligent driving control process; construct subscription information according to the target functional unit; the subscription information is used to instruct the target functional unit to push the intelligent driving data to the data conversion unit according to the second push frequency.

[0090] In one embodiment, the processing module 801 is further used to obtain a target data conversion strategy that matches the data category of the intelligent driving data from a plurality of preset data conversion strategies, and use the target data conversion strategy to perform format conversion on the intelligent driving data to obtain data to be combined in a target data format; according to a message subject of the data to be combined, determine a write position of the data to be combined in a data file in a target data format; a plurality of write positions are provided in the data file in the target data format, and each write position corresponds to a message subject; according to the write position of the data to be combined, write the data to be combined into the data file in the target data format to obtain a data frame in the target data format.

[0091] In one embodiment, the processing module 801 is also used to convert the format of the intelligent driving data according to the first data conversion strategy to obtain the data to be combined in JSON format when the data category of the intelligent driving data is not image data and is not point cloud data.

[0092] In one embodiment, the processing module 801 is also used to convert the format of the intelligent driving data according to the second data conversion strategy when the data category of the intelligent driving data is image data or point cloud data, so as to obtain the data to be combined in JPG format or PCD format.

[0093] In one embodiment, the processing module 801 is also used to determine at least one data format that can be processed by the browser based on the development information of the browser of the display device; determine the original data format of the intelligent driving data, and obtain the data conversion strategy corresponding to each data format based on the format conversion relationship between the original data format and each data format, so as to obtain multiple preset data conversion strategies.

[0094] Each module in the above-mentioned intelligent driving data processing device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0095] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG9. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store intelligent driving data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for processing intelligent driving data is implemented.

[0096] Those skilled in the art will understand that the structure shown in FIG. 9 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0097] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the following steps when executing the computer program: when receiving intelligent driving data pushed by a target functional unit in an intelligent driving system, converting and processing the intelligent driving data according to a target data conversion strategy that matches the data category of the intelligent driving data, and obtaining a data frame in a target data format according to the converted data; pushing the data frame to a display device at a first push frequency; the data frame is used to instruct the display device to adopt a visualization processing strategy that matches the target data format, visualize the data frame, obtain visualization data, and display the visualization data through a browser page.

[0098] In one embodiment, when the processor executes the computer program, the following steps are also implemented: according to the data content that needs to be visualized in the intelligent driving control process, a target functional unit is determined among multiple functional units of the intelligent driving system; according to the target functional unit, subscription information is constructed; the subscription information is used to instruct the target functional unit to push the intelligent driving data to the data conversion unit according to the second push frequency.

[0099] In one embodiment, when the processor executes the computer program, the following steps are also implemented: from multiple preset data conversion strategies, a target data conversion strategy that matches the data category of the intelligent driving data is obtained, and the target data conversion strategy is used to convert the format of the intelligent driving data to obtain data to be combined in a target data format; according to the message subject of the data to be combined, a write position of the data to be combined is determined in a data file in a target data format; multiple write positions are provided in the data file in the target data format, and each write position corresponds to a message subject; according to the write position of the data to be combined, the data to be combined is written into the data file in the target data format to obtain a data frame in the target data format.

[0100] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the data category of the intelligent driving data is not image data and is not point cloud data, the intelligent driving data is format converted according to the first data conversion strategy to obtain the data to be combined in JSON format.

[0101] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the data category of the intelligent driving data is image data or point cloud data, the intelligent driving data is format converted according to the second data conversion strategy to obtain the data to be combined in JPG format or PCD format.

[0102] In one embodiment, when the processor executes the computer program, the following steps are also implemented: determining at least one data format that can be processed by the browser based on the development information of the browser of the display device; determining the original data format of the intelligent driving data, and obtaining the data conversion strategy corresponding to each data format based on the format conversion relationship between the original data format and each data format, thereby obtaining multiple preset data conversion strategies.

[0103] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: when receiving intelligent driving data pushed by a target functional unit in an intelligent driving system, converting and processing the intelligent driving data according to a target data conversion strategy that matches the data category of the intelligent driving data, and obtaining a data frame in a target data format according to the converted data; pushing the data frame to a display device at a first push frequency; the data frame is used to instruct the display device to adopt a visualization processing strategy that matches the target data format, visualize the data frame, obtain visualization data, and display the visualization data through a browser page.

[0104] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: according to the data content that needs to be visualized in the intelligent driving control process, a target functional unit is determined among multiple functional units of the intelligent driving system; according to the target functional unit, subscription information is constructed; the subscription information is used to instruct the target functional unit to push the intelligent driving data to the data conversion unit according to the second push frequency.

[0105] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: from multiple preset data conversion strategies, a target data conversion strategy that matches the data category of the intelligent driving data is obtained, and the target data conversion strategy is used to convert the format of the intelligent driving data to obtain data to be combined in the target data format; according to the message subject of the data to be combined, the write position of the data to be combined is determined in the data file of the target data format; multiple write positions are provided in the data file of the target data format, and each write position corresponds to a message subject; according to the write position of the data to be combined, the data to be combined is written into the data file of the target data format to obtain a data frame in the target data format.

[0106] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the data category of the intelligent driving data is neither image data nor point cloud data, the intelligent driving data is format converted according to a first data conversion strategy to obtain data to be combined in JSON format.

[0107] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the data category of the intelligent driving data is image data or point cloud data, the intelligent driving data is format converted according to the second data conversion strategy to obtain the data to be combined in JPG format or PCD format.

[0108] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining at least one data format that can be processed by the browser based on the development information of the browser of the display device; determining the original data format of the intelligent driving data, and obtaining the data conversion strategy corresponding to each data format based on the format conversion relationship between the original data format and each data format, so as to obtain multiple preset data conversion strategies.

[0109] In one embodiment, a computer program product is provided, including a computer program, which implements the following steps when executed by a processor: when receiving intelligent driving data pushed by a target functional unit in an intelligent driving system, converting and processing the intelligent driving data according to a target data conversion strategy that matches the data category of the intelligent driving data, and obtaining a data frame in a target data format according to the converted data; pushing the data frame to a display device at a first push frequency; the data frame is used to instruct the display device to adopt a visualization processing strategy that matches the target data format, visualize the data frame, obtain visualization data, and display the visualization data through a browser page.

[0110] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: according to the data content that needs to be visualized in the intelligent driving control process, a target functional unit is determined among multiple functional units of the intelligent driving system; according to the target functional unit, subscription information is constructed; the subscription information is used to instruct the target functional unit to push the intelligent driving data to the data conversion unit according to the second push frequency.

[0111] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: from multiple preset data conversion strategies, a target data conversion strategy that matches the data category of the intelligent driving data is obtained, and the target data conversion strategy is used to convert the format of the intelligent driving data to obtain data to be combined in the target data format; according to the message subject of the data to be combined, the write position of the data to be combined is determined in the data file of the target data format; multiple write positions are provided in the data file of the target data format, and each write position corresponds to a message subject; according to the write position of the data to be combined, the data to be combined is written into the data file of the target data format to obtain a data frame in the target data format.

[0112] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the data category of the intelligent driving data is neither image data nor point cloud data, the intelligent driving data is format converted according to a first data conversion strategy to obtain data to be combined in JSON format.

[0113] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the data category of the intelligent driving data is image data or point cloud data, the intelligent driving data is format converted according to the second data conversion strategy to obtain the data to be combined in JPG format or PCD format.

[0114] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining at least one data format that can be processed by the browser based on the development information of the browser of the display device; determining the original data format of the intelligent driving data, and obtaining the data conversion strategy corresponding to each data format based on the format conversion relationship between the original data format and each data format, so as to obtain multiple preset data conversion strategies.

[0115] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0116] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0117] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0118] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for processing intelligent driving data, characterized in that: A data conversion unit applied to an on-vehicle intelligent driving system; the method comprises: When receiving the intelligent driving data pushed by the target functional unit in the intelligent driving system, converting the intelligent driving data according to a target data conversion strategy matching the data category of the intelligent driving data, and obtaining a data frame in a target data format according to the converted data; The data frame is pushed to the display device at a first push frequency; the data frame is used to instruct the display device to adopt a visualization processing strategy that matches the target data format, perform visualization processing on the data frame, obtain visualization data, and display the visualization data through a browser page.

2. The method according to claim 1, characterized in that The method further comprises: Determining the target functional unit among the plurality of functional units of the intelligent driving system according to the data content that needs to be visualized during the intelligent driving control process; According to the target functional unit, subscription information is constructed; the subscription information is used to instruct the target functional unit to push the intelligent driving data to the data conversion unit according to a second push frequency.

3. The method according to claim 1, characterized in that The converting process of the intelligent driving data according to the target data conversion strategy matching the data category of the intelligent driving data, and obtaining a data frame in a target data format according to the converted data, includes: From a plurality of preset data conversion strategies, a target data conversion strategy matching the data category of the intelligent driving data is obtained, and the target data conversion strategy is used to perform format conversion on the intelligent driving data to obtain data to be combined in the target data format; According to the message subject of the data to be combined, in the data file of the target data format, determining the writing position of the data to be combined; the data file of the target data format is provided with a plurality of writing positions, and each writing position corresponds to a message subject; The data to be combined are written into a data file in the target data format according to the writing position of the data to be combined, so as to obtain a data frame in the target data format.

4. The method according to claim 3, characterized in that The step of acquiring a target data conversion strategy that matches the data category of the intelligent driving data, and converting the format of the intelligent driving data using the target data conversion strategy to obtain the data to be combined in the target data format includes: When the data category of the intelligent driving data is neither image data nor point cloud data, the intelligent driving data is format converted according to a first data conversion strategy to obtain data to be combined in a JSON format.

5. The method according to claim 3, characterized in that: The step of acquiring a target data conversion strategy that matches the data category of the intelligent driving data, and converting the format of the intelligent driving data using the target data conversion strategy to obtain the data to be combined in the target data format includes: When the data category of the intelligent driving data is image data or point cloud data, the intelligent driving data is format converted according to the second data conversion strategy to obtain data to be combined in JPG format or PCD format.

6. The method according to claim 3, characterized in that The acquisition methods in the plurality of preset data conversion strategies include: Determining at least one data format that can be processed by the browser according to development information of the browser of the display device; Determine the original data format of the intelligent driving data, and obtain a data conversion strategy corresponding to each data format according to the format conversion relationship between the original data format and each data format, so as to obtain a plurality of preset data conversion strategies.

7. An intelligent driving data processing device, characterized in that: A data conversion unit applied to a vehicle-mounted intelligent driving system; the device comprises: a processing module, configured to, upon receiving the intelligent driving data pushed by the target functional unit in the intelligent driving system, convert the intelligent driving data according to a target data conversion strategy matching the data category of the intelligent driving data, and obtain a data frame in a target data format according to the converted data; A push module is used to push the data frame to a display device at a first push frequency; the data frame is used to instruct the display device to adopt a visualization processing strategy that matches the target data format, perform visualization processing on the data frame, obtain visualization data, and display the visualization data through a browser page.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images