Vehicle automatic driving identification data playback method, system, device and storage medium

Abstract

The application provides a vehicle automatic driving recognition data playback method, system, device and storage medium, historical data is classified and stored in a database, at least two kinds of historical data are obtained from the database, the at least two kinds of historical data are fused to obtain target recognition data, then the multiple kinds of historical data and the target recognition data are visually displayed to complete the recognition data playback of the vehicle automatic driving. The application restores the recognition target data of the vehicle during driving by obtaining the historical data and according to the fusion result of the historical data, then the recognition target data is visually displayed, so as to help the research and development personnel to obtain the recognition data of the vehicle during driving, and help the research and development personnel to improve the assisted driving algorithm.

Description

Technical Field

[0001] This application relates to the field of automotive driver assistance technology, specifically to methods, systems, devices, and storage media for playing back vehicle autonomous driving recognition data. Background Technology

[0002] With the development of automotive technology, more and more cars are equipped with driver assistance functions. Current driver assistance functions rely on the recognition of road targets, such as lane lines and vehicles ahead. Existing intelligent driving systems use sensors such as cameras and LiDAR to identify road targets, obtain recognition results, and then the driver assistance algorithm controls the vehicle based on these results to complete the assisted driving. However, driver assistance algorithms need continuous improvement, and improving these algorithms requires a large amount of road recognition data. However, there is currently no technology that can retrospectively analyze the recognition data generated during vehicle driver assistance. Summary of the Invention

[0003] In view of the shortcomings of the prior art described above, the present invention provides a method, system, device and storage medium for playing back vehicle autonomous driving recognition data to solve the above technical problems.

[0004] This invention provides a method for replaying vehicle autonomous driving recognition data, the method comprising:

[0005] Acquire various historical data of the vehicle, which are collected by various sensors preset inside the vehicle;

[0006] The historical data is categorized and stored in a database;

[0007] Obtain at least two types of historical data from the database, identify the at least two types of historical data, and obtain at least two types of identification data;

[0008] The at least two types of identification data are fused to obtain target identification data;

[0009] The various historical data and target recognition data are visualized to complete the playback of recognition data for autonomous driving of vehicles.

[0010] In one embodiment of the present invention, the sensor data is classified and stored in a database, including:

[0011] The identification information of multiple sensors is acquired, and the identification information of each sensor corresponds one-to-one with the historical data.

[0012] The various historical data are serialized to obtain various serialized data.

[0013] The serialized data is classified according to the sensor's identity information;

[0014] The categorized serialized data is stored in the database.

[0015] In one embodiment of the present invention, at least two types of historical data are obtained from the database, including:

[0016] Obtain at least two types of serialized data from the database;

[0017] The at least two types of serialized data are deserialized to obtain at least two types of historical data.

[0018] In one embodiment of the present invention, fusing the at least two types of identification data to obtain target identification data includes:

[0019] Obtain the timestamp of each of the at least two types of identification data;

[0020] The target identification data is obtained by fusing the identification data at the same time point from the at least two types of identification data according to the timestamp.

[0021] In one embodiment of the present invention, the at least two types of historical data include video data, and the at least two types of recognition data include image recognition data obtained by recognizing the video data;

[0022] All time-synchronized identification data are fused to obtain the target identification data, including:

[0023] The identification target of the image recognition data in the video data, and the identification target of other recognition data besides the image recognition data in the at least two types of recognition data;

[0024] The associated range is obtained by using the target in the image recognition data as a base point to define the range.

[0025] When the target in other recognition data is within the associated range, it is determined that the target in the image recognition data is associated with the target in other historical data.

[0026] The target recognition data of the image recognition data of multiple frames in the video data is associated with the target recognition data of other recognition data. When the target recognition data of multiple frames of image recognition data is associated with the target recognition data of other recognition data, the target recognition data is constructed using the parameters of the target recognition data of the image recognition data and the parameters of the target recognition data of other recognition data.

[0027] In one embodiment of the present invention, before constructing the target recognition data using the parameters of the target in the image recognition data and the parameters of the target in other recognition data, the method further includes:

[0028] The parameters of the target in the image recognition data and the parameters of the target in other historical data are filtered.

[0029] In one embodiment of the present invention, the various historical data and the target identification data are visualized, including:

[0030] Place the various historical data and the target identification data on a timeline;

[0031] In response to externally inputted time point information, various historical data and target recognition data corresponding to the time point information will be output and displayed.

[0032] The various historical data and target recognition data on the timeline are output and displayed frame by frame.

[0033] The present invention also provides a vehicle autonomous driving recognition data playback system, the system comprising:

[0034] The acquisition module is used to acquire various historical data of the vehicle, which are collected by various sensors preset inside the vehicle.

[0035] A storage module is used to classify and store the historical data in a database;

[0036] The identification module is used to obtain at least two types of historical data from the database, identify the at least two types of historical data, and obtain at least two types of identification data;

[0037] A fusion module is used to fuse the at least two types of target recognition data to obtain target recognition data;

[0038] The playback module is used to visualize the various historical data and the target recognition data, and to complete the playback of the recognition data for autonomous driving of the vehicle.

[0039] The present invention also provides an electronic device, comprising:

[0040] One or more processors;

[0041] A storage device for storing one or more programs, which, when executed by one or more processors, enable the electronic device to implement the vehicle autonomous driving recognition data playback method as described above.

[0042] The present invention also provides a computer-readable storage medium, characterized in that it stores computer-readable instructions thereon, which, when executed by a computer's processor, cause the computer to execute the vehicle autonomous driving identification data playback method as described above.

[0043] The beneficial effects of this invention are as follows: The vehicle autonomous driving recognition data playback method, system, device, and storage medium of this invention store historical data in a database, categorizes and stores it; retrieves at least two types of historical data from the database, fuses them to obtain target recognition data; and then visualizes the various historical data and target recognition data to complete the playback of vehicle autonomous driving recognition data. This invention, by acquiring historical data and reconstructing the vehicle's target recognition data during driving based on the fusion results of the historical data, and then visualizing the target recognition data, helps researchers obtain vehicle recognition data during driving, thereby assisting them in improving driver assistance algorithms.

[0044] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0045] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0046] Figure 1 This is an exemplary embodiment of the present application illustrating an application scenario of a vehicle autonomous driving identification data playback method;

[0047] Figure 2 This is a flowchart illustrating a method for replaying vehicle autonomous driving identification data, as shown in an exemplary embodiment of this application.

[0048] Figure 3 yes Figure 2 Step S220 in the illustrated embodiment is a flowchart of an exemplary embodiment;

[0049] Figure 4 yes Figure 2 The flowchart of step S230 in the illustrated embodiment is shown in an exemplary embodiment;

[0050] Figure 5 yes Figure 2 The flowchart of step S240 in the illustrated embodiment is shown in an exemplary embodiment;

[0051] Figure 6 yes Figure 5 The flowchart of step S530 in the illustrated embodiment is shown in an exemplary embodiment;

[0052] Figure 7 yes Figure 6 A flowchart of an exemplary embodiment preceding step S640 in the illustrated embodiment;

[0053] Figure 8 yes Figure 2 The flowchart of step S250 in the illustrated embodiment is shown in an exemplary embodiment;

[0054] Figure 9 This is a block diagram illustrating a vehicle autonomous driving recognition data playback system, as shown in an exemplary embodiment of this application.

[0055] Figure 10 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0056] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0057] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0058] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0059] Figure 1 This is an exemplary embodiment of the present application illustrating an application scenario of a vehicle autonomous driving recognition data playback method. Figure 1In the process, the vehicle's infotainment system uploads historical data of the vehicle's driving process to the server via the Internet. The server categorizes and stores the historical data. The data processing equipment retrieves the historical data from the server, merges the historical recognition data, generates target recognition data, and then outputs the historical data and target recognition data to the display device for visualization.

[0060] in, Figure 1 The data processing device 110 shown can be any terminal device that supports data processing, such as a smartphone, in-vehicle computer, tablet computer, laptop computer, or wearable device, but is not limited to these. Figure 1 The server 120 shown is a car navigation server. It can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. No restrictions are placed on this. The data processing device 110 can communicate with the server 120 via wireless networks such as 3G (third-generation mobile information technology), 4G (fourth-generation mobile information technology), and 5G (fifth-generation mobile information technology). No restrictions are placed on this as well.

[0061] like Figure 2 As shown, in an exemplary embodiment, the vehicle autonomous driving recognition data playback method includes at least steps S210 to S240, which are described in detail below:

[0062] S210. Acquire various historical data of the vehicle, which are collected by various sensors pre-installed inside the vehicle;

[0063] In this embodiment, during the driving process of the vehicle performing assisted driving functions, various sensors collect historical data, such as target vehicle, point cloud data, and lane lines; in this embodiment, data from various sensors is obtained through the socket communication protocol.

[0064] S220. Classify and store historical data in a database;

[0065] In step S220, historical data is stored in a database to facilitate subsequent processing of the historical data;

[0066] S230. Obtain at least two types of historical data from the database, identify the at least two types of historical data, and obtain at least two types of identified data;

[0067] In step S230, when multiple sensors are installed in the vehicle, the data obtained by the multiple sensors is used to assist driving. The data will be identified. Therefore, in this embodiment, at least two kinds of historical data need to be identified to simulate the identification data of the vehicle during assisted driving.

[0068] S240. Fuse at least two types of identification data to obtain target identification data;

[0069] In step S240, multiple types of historical data from the vehicle need to be identified, resulting in various identification data sets, which may differ from each other. Therefore, to simulate the vehicle's target identification results during operation, it is necessary to fuse these multiple identification data sets.

[0070] S250. Visualizes various historical data and target recognition data to complete the playback of recognition data for autonomous driving of vehicles.

[0071] In this embodiment, by visualizing various historical data and target recognition data, researchers can intuitively view the data and more intuitively detect whether the correlation fusion and target matching in the development of autonomous driving algorithms are reasonable.

[0072] like Figure 3 As shown, in one embodiment of the present invention, the process of classifying and storing sensor data in a database may include steps S310 to S340, which are described in detail below:

[0073] S310. Acquire the identification information of multiple sensors, and the identification information of the sensors corresponds one-to-one with historical data;

[0074] In this embodiment, common types of vehicle sensors include front cameras, front radar, and corner radar; the sensor to which historical data belongs can be determined through the sensor's identity information (such as physical ID);

[0075] S320. Perform serialization processing on various historical data to obtain various serialized data;

[0076] In step S320, various historical data are converted into protobuf format sequence data using a preset serialization function. Protobuf, a data storage and communication tool developed by Google, supports multiple programming languages ​​and is well-suited for development work involving large amounts of data and frequent changes in data structures during development. Using protobuf to define sensor data structures can greatly improve algorithm development efficiency. Furthermore, for multi-sensor data, due to its massive volume and high latency of ordinary transmission methods, we use ZMQ to handle data transmission. ZMQ's significant feature is its extremely high transmission efficiency, which can address the low data transmission efficiency issues in multi-sensor autonomous driving. This way, whether it's sensor data sending, receiving, or data storage, we can reduce the latency of the autonomous driving system while better realizing online real-time data visualization functions.

[0077] S330. Classify the serialized data according to the sensor's identification information;

[0078] In step S330, the sensor to which the historical data belongs is determined by the sensor's identity information (such as physical ID);

[0079] S340. Store the classified serialized data in the database.

[0080] In this embodiment, the classified serialized data is stored in a database to facilitate subsequent data retrieval, processing, and analysis.

[0081] like Figure 4 As shown, in one embodiment of the present invention, the process of obtaining at least two types of historical data from the database may include steps S410 to S420, which are described in detail below:

[0082] S410. Obtain at least two types of serialized data from the database;

[0083] In step S410, at least two types of serialized data refer to serialized data obtained by converting historical data from two different sensors;

[0084] S420. Deserialize at least two types of serialized data to obtain at least two types of historical data.

[0085] In this embodiment, the serialized data can be restored to historical data by deserializing the serialized data.

[0086] like Figure 5 As shown, in one embodiment of the present invention, the process of fusing at least two types of historical data to obtain target identification data may include steps S510 to S530, which are described in detail below:

[0087] S510. Obtain the timestamp of each of at least two types of historical data;

[0088] In this embodiment, the timestamp is data generated using digital signature technology. When the sensor acquires historical data, it adds a timestamp to the historical data. Therefore, the acquisition time of the historical data can be obtained by acquiring the timestamp of the historical data.

[0089] S520. Based on the timestamp, the identification data at the same time point in the at least two types of identification data are fused to obtain the target identification data.

[0090] In step S520, after synchronizing each of the at least two types of historical data in time, the data acquisition status of various sensors when the vehicle is driving at a certain moment and the road recognition status through sensor data can be obtained. By fusing multiple recognition data, the recognition result (target recognition data) obtained by the assisted driving algorithm in the vehicle can be restored.

[0091] In one embodiment of the present invention, at least two types of historical data include video data, and at least two types of recognition data include image recognition data obtained by recognizing the video data;

[0092] like Figure 6 As shown, the process of fusing each time-synchronized recognition data to obtain target recognition data may include steps S610 to S640, which are described in detail below:

[0093] S610. Obtain the recognition target of image recognition data in video data, and the recognition target of other recognition data in at least two types of recognition data besides image recognition data.

[0094] In this embodiment, the target in the image recognition data can be: the vehicle in front, lane lines, traffic signs, etc., which can be used to assist driving. Other recognition data (such as recognition data based on historical data generated by LiDAR) can be: the vehicle in front, the vehicle on the side, etc.

[0095] S620. Using the target in the image recognition data as a base point to delineate the range, the associated range is obtained;

[0096] In this embodiment, a coordinate system is first established using the recognition target in the image recognition data as the base point, and then a range (which can be a circle) is defined using the base point of the coordinate system. This range is then used to associate the recognition targets of other recognition data.

[0097] S630. When the target in other recognition data is within the associated range, determine that the target in the image recognition data is associated with the target in other historical data;

[0098] In this embodiment, specifically, when the identification results of multiple identification data are consistent and multiple identification targets are all within the associated range, it indicates that the positions of the multiple identification results are basically consistent, and the identification targets in the multiple identification data are the same target;

[0099] S640. Associate the recognition targets of the image recognition data of multiple frames in the video data with the recognition targets of other recognition data. When the recognition targets of the multiple frames of image recognition data are all associated with the recognition targets in other recognition data, construct the target recognition data with the parameters of the recognition targets in the image recognition data and the parameters of the recognition targets in other recognition data.

[0100] In this embodiment, if 20 consecutive frames can be correlated to determine the accuracy of the target identification, fusion can be performed to combine multiple identification data to obtain the final target identification data.

[0101] like Figure 7 As shown, in one embodiment of the present invention, the process before constructing target recognition data using the parameters of the target in the image recognition data and the parameters of the target in other historical data may further include step S710, which is described in detail below:

[0102] S710. Filter the parameters of the target in the image recognition data and the parameters of the target in other historical data.

[0103] In this embodiment, Kalman filtering is used to filter the parameters of the target to be identified, such as the horizontal and vertical distances, length and width, heading and other parameters in the image recognition data.

[0104] like Figure 8 As shown, in one embodiment of the present invention, the process of visualizing various historical data and target recognition data may include steps S810 to S830, which are described in detail below:

[0105] S810. Place various historical data and target recognition data on the timeline;

[0106] In this embodiment, various historical data and target recognition data are placed on a timeline to facilitate the acquisition of various historical data and target recognition data in chronological order.

[0107] S820. In response to externally inputted time point information, it outputs and displays various historical data and target recognition data corresponding to the time point information;

[0108] In step S820, information is viewed by selecting a time point, enabling the data playback and dragging function along the time axis.

[0109] S830 outputs and displays various historical data and target recognition data on the timeline frame by frame.

[0110] In step S830, by outputting and displaying various historical data and target recognition data on the timeline frame by frame, the data of each sensor in single frame and frame by frame can be displayed. This can greatly improve the development efficiency of autonomous driving fusion algorithms and provide a very useful data visualization function for correlation fusion.

[0111] The vehicle autonomous driving recognition data playback method of this invention involves classifying and storing historical data in a database; retrieving at least two types of historical data from the database; fusing these two types of historical data to obtain target recognition data; and then visualizing the various historical data and target recognition data to complete the playback of vehicle autonomous driving recognition data. This invention, by acquiring historical data and reconstructing the vehicle's target recognition data during driving based on the fusion results of the historical data, and then visualizing the target recognition data, helps developers obtain recognition data during vehicle operation, thereby assisting them in improving driver assistance algorithms.

[0112] like Figure 9 As shown, the present invention also provides a vehicle autonomous driving recognition data playback system, the system comprising:

[0113] The acquisition module is used to acquire various historical data of the vehicle, which are collected by various sensors preset inside the vehicle.

[0114] A storage module is used to classify and store the historical data in a database;

[0115] The identification module is used to obtain at least two types of historical data from the database, identify the at least two types of historical data, and obtain at least two types of identification data;

[0116] A fusion module is used to fuse the at least two types of target recognition data to obtain target recognition data;

[0117] The playback module is used to visualize the various historical data and the target recognition data, and to complete the playback of the recognition data for autonomous driving of the vehicle.

[0118] The vehicle autonomous driving recognition data playback system of this invention stores historical data in a database, categorizes and retrieves at least two types of historical data from the database, fuses these two types of historical data to obtain target recognition data, and then visualizes the various historical data and target recognition data to complete the playback of vehicle autonomous driving recognition data. This invention, by acquiring historical data and reconstructing the vehicle's target recognition data during driving based on the fusion results of the historical data, and then visualizing the target recognition data, helps developers obtain recognition data during vehicle operation, thereby assisting them in improving driver assistance algorithms.

[0119] It should be noted that the vehicle autonomous driving recognition data playback system and the vehicle autonomous driving recognition data playback method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the vehicle autonomous driving recognition data playback system provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0120] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the vehicle autonomous driving recognition data playback method provided in the above embodiments.

[0121] Figure 10 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 10 The computer system 1000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0122] like Figure 10As shown, the computer system 1000 includes a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1002 or programs loaded from storage portion 1008 into Random Access Memory (RAM) 1003, such as performing the methods described in the above embodiments. The RAM 1003 also stores various programs and data required for system operation. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An Input / Output (I / O) interface 1005 is also connected to the bus 1004.

[0123] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0124] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs various functions defined in the system of this application.

[0125] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0126] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0127] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0128] Another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the vehicle autonomous driving recognition data playback method described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not incorporated into the electronic device.

[0129] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the vehicle autonomous driving recognition data playback method provided in the various embodiments described above.

[0130] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for replaying vehicle autonomous driving recognition data, characterized in that, The method includes: Acquire various historical data of the vehicle, which are collected by various sensors preset inside the vehicle; The historical data is categorized and stored in a database; Obtain at least two types of historical data from the database, identify the at least two types of historical data, and obtain at least two types of identification data; The process of fusing the at least two types of recognition data to obtain target recognition data includes: acquiring the timestamp of each type of recognition data; fusing recognition data at the same time point in the at least two types of recognition data according to the timestamps to obtain the target recognition data, including: fusing all time-synchronized recognition data to obtain the target recognition data, including: acquiring the recognition target of image recognition data in video data, and the recognition targets of other recognition data besides image recognition data in the at least two types of recognition data; delineating a range using the recognition target in the image recognition data as a base point to obtain an association range, including: first establishing a coordinate system using the recognition target in the image recognition data as a base point, then delineating a range using the base point of the coordinate system, and using this range to associate the recognition targets of other recognition data; when the recognition targets in other recognition data are located within the association range, it is determined that the recognition target in the image recognition data is associated with the recognition targets in other historical data; The identification targets of image recognition data from multiple frames of the video data are associated with the identification targets of other identification data. When the identification targets of multiple frames of image recognition data are associated with the identification targets in other identification data, the target identification data is constructed using the parameters of the identification targets in the image recognition data and the parameters of the identification targets in other identification data. Before constructing the target identification data using the parameters of the identification targets in the image recognition data and the parameters of the identification targets in other identification data, the method further includes filtering the parameters of the identification targets in the image recognition data and the parameters of the identification targets in other historical data. Specifically, this includes using Kalman filtering to filter the parameters of the identification targets, including filtering the horizontal and vertical distances, length and width, and headings in the image recognition data. The various historical data and target recognition data are visualized to complete the playback of recognition data for autonomous driving of vehicles.

2. The method for replaying vehicle autonomous driving identification data according to claim 1, characterized in that, The historical data is categorized and stored in a database, including: The identification information of multiple sensors is acquired, and the identification information of each sensor corresponds one-to-one with the historical data. The various historical data are serialized to obtain various serialized data. The serialized data is classified according to the sensor's identity information; The categorized serialized data is stored in the database.

3. The method for replaying vehicle autonomous driving identification data according to claim 2, characterized in that, Retrieve at least two types of historical data from the database, including: Obtain at least two types of serialized data from the database; The at least two types of serialized data are deserialized to obtain at least two types of historical data.

4. The method for replaying vehicle autonomous driving identification data according to claim 1, characterized in that, The various historical data and the target recognition data are visualized, including: Place the various historical data and the target identification data on a timeline; In response to externally inputted time point information, various historical data and target recognition data corresponding to the time point information will be output and displayed. The various historical data and target recognition data on the timeline are output and displayed frame by frame.

5. A vehicle autonomous driving recognition data playback system, characterized in that, The system includes: The acquisition module is used to acquire various historical data of the vehicle, which are collected by various sensors preset inside the vehicle. A storage module is used to classify and store the historical data in a database; The identification module is used to obtain at least two types of historical data from the database, identify the at least two types of historical data, and obtain at least two types of identification data; A fusion module is used to fuse the at least two types of recognition data to obtain target recognition data, including: obtaining the timestamp of each type of recognition data in the at least two types of recognition data; fusing recognition data at the same time point in the at least two types of recognition data according to the timestamp to obtain the target recognition data, including: fusing all recognition data after time synchronization to obtain the target recognition data, including: obtaining the recognition target of image recognition data in video data, and the recognition targets of other recognition data besides image recognition data in the at least two types of recognition data; delineating a range using the recognition target in the image recognition data as a base point to obtain an association range, including: first establishing a coordinate system using the recognition target in the image recognition data as a base point, then delineating a range using the base point of the coordinate system, and using this range to associate the recognition targets of other recognition data; when the recognition targets in other recognition data are located within the association range, it is determined that the recognition target in the image recognition data is associated with the recognition targets in other historical data; The identification targets of image recognition data from multiple frames of the video data are associated with the identification targets of other identification data. When the identification targets of multiple frames of image recognition data are associated with the identification targets in other identification data, the target identification data is constructed using the parameters of the identification targets in the image recognition data and the parameters of the identification targets in other identification data. Before constructing the target identification data using the parameters of the identification targets in the image recognition data and the parameters of the identification targets in other identification data, the method further includes filtering the parameters of the identification targets in the image recognition data and the parameters of the identification targets in other historical data. Specifically, this includes using Kalman filtering to filter the parameters of the identification targets, including filtering the horizontal and vertical distances, length and width, and headings in the image recognition data. The playback module is used to visualize the various historical data and the target recognition data, and to complete the playback of the recognition data for autonomous driving of the vehicle.

6. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the vehicle autonomous driving recognition data playback method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, It stores computer-readable instructions, which, when executed by the computer's processor, cause the computer to perform the vehicle autonomous driving identification data playback method according to any one of claims 1 to 4.

Images