Data processing method, device, storage medium, processor and electronic device

Abstract

The present invention discloses a data processing method, device, storage medium, processor, and electronic device. The method comprises: constructing a scene database based on a collected target data set, wherein the scene database uses a data frame as the minimum storage management unit; obtaining a query request, and filtering the scene database according to the query request to obtain a query result, wherein the information carried in the query request includes: multi-dimensional feature conditions; performing data verification on the query result to obtain a verification result; and performing data processing on the verification result to obtain a target processing result. The present invention solves the technical problems of poor target detection accuracy and weak target tracking capability caused by inappropriate storage management units of point cloud scene databases in autonomous driving and lack of verification and fine processing of scene data.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving, and in particular to a data processing method, device, storage medium, processor, and electronic device. Background Art

[0002] The rapid development of autonomous driving technology in recent years has generated a vast amount of driving scenario data. Due to the complexity of driving scenarios, it is crucial to manage and store this collected data according to scenario conditions. From a technical perspective, scenario databases are fundamental data resources for the R&D and testing of intelligent connected vehicles (ICVs). They serve as a crucial case library for evaluating the functional safety of ICVs and are a key data foundation for redefining their functional safety standards. Therefore, building a suitable scenario database and rationally processing and utilizing this data are of vital importance to the field of autonomous driving.

[0003] Currently, a comprehensive theoretical framework has been established within the field of scene data, encompassing data collection specifications, data processing workflows, feature extraction methods, scenario database structure specifications, test case data formats, and virtual simulation testing methods for driving scenarios. The feature scene data mining method based on these theories generally includes a scene data collection module, a scene database establishment module, a feature scene query module, and a feature scene acquisition module. When establishing the scene database, scene data is classified for subsequent query, acquisition, and processing. The classification criteria include natural conditions (such as weather and lighting), road type (such as road surface conditions and lane markings), traffic participants (such as vehicle and pedestrian position and speed), and environment type (such as highways, residential areas, shopping malls, and rural areas). While the technology for establishing scene databases is relatively mature, due to the complexity of real-world driving scenarios and the diverse technical requirements, there is still room for improvement. Consequently, methods for managing, analyzing, verifying, and further processing scene data require continuous improvement.

[0004] Therefore, how to establish a suitable scene database and process the scene data to improve target detection accuracy and target tracking capabilities, thereby optimizing autonomous driving technology, has become a key issue. To address the above issues, no effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of the present invention provide a data processing method, device, storage medium, processor, and electronic device to at least solve the technical problems of poor target detection accuracy and weak target tracking capability caused by an inappropriate storage management unit of a point cloud scene database in autonomous driving and a lack of verification and fine processing of scene data.

[0006] According to one embodiment of the present invention, a data processing method is provided, comprising:

[0007] A scene database is constructed based on the collected target data set, wherein the scene database uses a data frame as the minimum storage management unit; a query request is obtained, and query results are obtained by filtering from the scene database according to the query request, wherein the information carried in the query request includes: multi-dimensional feature conditions; data verification is performed on the query results to obtain verification results; and data processing is performed on the verification results to obtain target processing results.

[0008] Optionally, constructing a scene database based on the target data set includes: controlling the data acquisition device to collect road data on the target driving section to generate a first data set, controlling the data acquisition device to collect environmental data on the target driving section to generate a second data set, and controlling the data acquisition device to collect obstacle data and scene data on the target driving section to generate a third data set, wherein the data acquisition device has an automatic driving function and a radar detection function; performing data analysis on the first data set and the second data set to obtain analysis results, and obtaining motion state information of the data acquisition device based on the analysis results, wherein the analysis results include: radar point cloud data, image data, positioning data, map data, and the motion state information includes: motion trajectory, driving speed, driving acceleration, and heading angle of the data acquisition device; performing data annotation on the third data set to obtain attribute information of obstacles on the target driving section, wherein the attribute information includes: location, appearance, category, and motion state of the obstacle; and associating and storing the first data set, the second data set, the third data set, the analysis results, the motion state information, and the attribute information in units of data frames to construct a scene database.

[0009] Optionally, filtering the query results from the scene database according to the query request includes: parsing the query request to obtain multi-dimensional feature conditions; filtering the query results that meet the multi-dimensional feature conditions from the scene database, wherein the query results include: point cloud data corresponding to multiple point cloud frames.

[0010] Optionally, data verification is performed on the query results to obtain verification results, including: performing multiple data verifications on the point cloud data corresponding to multiple point cloud frames to obtain verification results, wherein the multiple data verifications include: single-frame point cloud quality verification, data size verification, coordinate system verification, and time verification.

[0011] Optionally, data processing is performed on the verification result to obtain a target processing result, including: single-frame data cropping of the verification result to obtain a cropping result, wherein the single-frame data cropping is used to retain the point cloud area of ​​interest from the verification result; multi-frame data fusion is performed on the cropping result to obtain a fusion result, wherein the multi-frame data fusion is used to splice the point cloud area of ​​interest into the same coordinate system; data filtering is performed on the fusion result to obtain a target processing result, wherein the data filtering is used to filter out obstacle categories within the point cloud area of ​​interest.

[0012] Optionally, multi-frame data fusion is performed on the cropping result to obtain a fusion result including: when the data acquisition device used to acquire the target data set is configured with multiple sub-radars, the point cloud data respectively collected by the multiple sub-radars at the current moment are converted to the same vehicle body coordinate system to obtain a single-frame point cloud data; the point cloud data respectively collected by the multiple sub-radars at adjacent moments of the current moment are first converted from the radar coordinate system of the adjacent moments to the world coordinate system, and then converted from the world coordinate system to the radar coordinate system of the current moment to obtain the converted point cloud data; the single-frame point cloud data and the converted point cloud data are spliced ​​and fused to obtain a fusion result.

[0013] According to one embodiment of the present invention, there is further provided a data processing device, comprising:

[0014] The scene database construction module is used to build a scene database based on the collected target data set, wherein the scene database uses the data frame as the minimum storage management unit; the screening module is used to obtain the query request and filter the query results from the scene database according to the query request, wherein the information carried in the query request includes: multi-dimensional feature conditions; the data verification module is used to perform data verification on the query results to obtain the verification results; the data processing module is used to perform data processing on the verification results to obtain the target processing results.

[0015] Optionally, the scene database construction module includes: an acquisition unit, which is used to control the data acquisition device to collect road data on the target driving section to generate a first data set, control the data acquisition device to collect environmental data on the target driving section to generate a second data set, and control the data acquisition device to collect obstacle data and scene data on the target driving section to generate a third data set, wherein the data acquisition device has an automatic driving function and a radar detection function; a data analysis unit, which is used to perform data analysis on the first data set and the second data set to obtain analysis results, and obtain motion state information of the data acquisition device based on the analysis results, wherein the analysis results include: radar point cloud data, image data, positioning data, map data, and the motion state information includes: motion trajectory, driving speed, driving acceleration, and heading angle of the data acquisition device; a labeling unit, which is used to perform data labeling on the third data set to obtain attribute information of obstacles on the target driving section, wherein the attribute information includes: location, appearance, category, and motion state of the obstacle; a storage construction unit, which is used to associate and store the first data set, the second data set, the third data set, the analysis results, the motion state information, and the attribute information in units of data frames to construct a scene database.

[0016] Optionally, the screening module is further used to: parse the query request to obtain multi-dimensional feature conditions; and filter the query results that meet the multi-dimensional feature conditions from the scene database, wherein the query results include: point cloud data corresponding to multiple point cloud frames.

[0017] Optionally, the verification module is also used to perform multiple data verifications on the point cloud data corresponding to multiple point cloud frames to obtain verification results, wherein the multiple data verifications include: single-frame point cloud quality verification, data size verification, coordinate system verification, and time verification.

[0018] Optionally, the data processing module includes: a cropping unit, which is used to perform single-frame data cropping on the verification result to obtain a cropping result, wherein the single-frame data cropping is used to retain the point cloud area of ​​interest from the verification result; a fusion unit, which performs multi-frame data fusion on the cropping result to obtain a fusion result, wherein the multi-frame data fusion is used to splice the point cloud area of ​​interest into the same coordinate system; a filtering unit, which performs data filtering on the fusion result to obtain a target processing result, wherein the data filtering is used to filter out the obstacle categories in the point cloud area of ​​interest.

[0019] Optionally, the fusion unit is also used to: when the data acquisition device used to collect the target data set is configured with multiple sub-radars, convert the point cloud data collected by the multiple sub-radars at the current moment into the same vehicle body coordinate system to obtain single-frame point cloud data; convert the point cloud data collected by the multiple sub-radars at adjacent moments of the current moment first from the radar coordinate system of the adjacent moments to the world coordinate system, and then convert from the world coordinate system to the radar coordinate system of the current moment to obtain converted point cloud data; splice and fuse the single-frame point cloud data with the converted point cloud data to obtain a fusion result.

[0020] According to one embodiment of the present invention, a non-volatile storage medium is further provided, characterized in that a computer program is stored in the storage medium, wherein the computer program is configured to execute any of the aforementioned data processing methods when running.

[0021] According to one embodiment of the present invention, a processor is further provided, characterized in that the processor is used to run a program, wherein the program is configured to execute any of the aforementioned data processing methods when running.

[0022] According to one embodiment of the present invention, there is further provided an electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute any of the aforementioned data processing methods.

[0023] In an embodiment of the present invention, a scene database with a data frame as the minimum storage management unit is constructed based on a collected target data set. A query result containing multi-dimensional feature conditions is obtained by screening the scene database according to the obtained query request. The query result is subjected to data verification to obtain a verification result, and the verification result is subjected to data processing to obtain a target processing result. This achieves the goal of constructing a suitable scene database to facilitate the storage and management of automobile driving scene data according to scene conditions, and improves the data analysis capability to meet the needs of actual applications by querying, verifying and processing (including clipping, fusion and filtering) the data, thereby achieving the technical effect of improving the target detection accuracy and target tracking capability of autonomous driving, and further solves the technical problems of poor target detection accuracy and weak target tracking capability caused by inappropriate storage management units of point cloud scene databases and lack of verification and fine processing of scene data in autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0025] Figure 1is a flow chart of a data processing method according to one embodiment of the present invention;

[0026] Figure 2 is a structural block diagram of a data processing device according to one embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] According to an embodiment of the present invention, an embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0030] The embodiment of the method can be executed in an electronic device or similar computing device in a vehicle that includes a memory and a processor. Taking the operation on the electronic device of the vehicle as an example, the electronic device of the vehicle may include one or more processors (the processor may include but is not limited to a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP) chip, a microprocessor (MCU), a programmable logic device (FPGA), a neural network processor (NPU), a tensor processing unit (TPU), an artificial intelligence (AI) type processor, etc.) and a memory for storing data. And a memory for storing data. Optionally, the electronic device of the above-mentioned car may also include a transmission device, an input and output device, and a display device for communication functions. It can be understood by those skilled in the art that the above-mentioned structural description is only illustrative and does not limit the structure of the electronic device of the above-mentioned vehicle. For example, the electronic device of the vehicle may also include more or fewer components than the above-mentioned structural description, or have a configuration different from the above-mentioned structural description.

[0031] The memory can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the information processing method in the embodiment of the present invention. The processor executes various functional applications and data processing by running the computer program stored in the memory, that is, realizing the above-mentioned information processing method. The memory may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0032] The transmission device is used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by the mobile terminal's communications provider. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In one embodiment, the transmission device may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0033] The display device can be, for example, a touch-screen liquid crystal display (LCD) and a touch display (also referred to as a "touch screen" or "touch display screen"). The liquid crystal display can enable a user to interact with the user interface of the mobile terminal. In some embodiments, the mobile terminal has a graphical user interface (GUI), and the user can interact with the GUI by finger contacts and / or gestures on the touch-sensitive surface. The human-computer interaction functions here optionally include the following interactions: creating web pages, drawing, word processing, making electronic documents, games, video conferencing, instant messaging, sending and receiving emails, call interfaces, playing digital videos, playing digital music and / or web browsing, etc. The executable instructions for performing the above-mentioned human-computer interaction functions are configured / stored in a computer program product or readable storage medium executable by one or more processors.

[0034] In this embodiment, a data processing method of an electronic device running on the above vehicle is provided. Figure 1 is a flow chart of a data processing method according to one embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0035] Step S10, constructing a scene database based on the collected target data set, wherein the scene database uses a data frame as the minimum storage management unit;

[0036] Step S12: obtaining a query request, and filtering the query results from the scene database according to the query request, wherein the information carried in the query request includes: multi-dimensional feature conditions;

[0037] Step S14, performing data verification on the query result to obtain a verification result;

[0038] Step S16: perform data processing on the verification result to obtain the target processing result.

[0039] In an embodiment of the present invention, a scene database with a data frame as the minimum storage management unit is constructed based on a collected target data set. A query result containing multi-dimensional feature conditions is obtained by screening the scene database according to the obtained query request. The query result is subjected to data verification to obtain a verification result, and the verification result is subjected to data processing to obtain a target processing result. This achieves the goal of constructing a suitable scene database to facilitate the storage and management of automobile driving scene data according to scene conditions, and improves the data analysis capability to meet the needs of actual applications by querying, verifying and processing (including clipping, fusion and filtering) the data, thereby achieving the technical effect of improving the target detection accuracy and target tracking capability of autonomous driving, and further solves the technical problems of poor target detection accuracy and weak target tracking capability caused by inappropriate storage management units of point cloud scene databases and lack of verification and fine processing of scene data in autonomous driving.

[0040] Optionally, in step S10, constructing scene data based on the target data set may include the following execution steps:

[0041] Step S101: Controlling a data acquisition device to collect road data on a target driving section to generate a first data set, controlling a data acquisition device to collect environmental data on the target driving section to generate a second data set, and controlling a data acquisition device to collect obstacle data and scene data on the target driving section to generate a third data set, wherein the data acquisition device has both an autonomous driving function and a radar detection function;

[0042] Step S102: parsing the first and second datasets to obtain parsing results, and acquiring motion state information of the data acquisition device based on the parsing results, wherein the parsing results include radar point cloud data, image data, positioning data, and map data, and the motion state information includes motion trajectory, driving speed, driving acceleration, and heading angle of the data acquisition device.

[0043] Step S103: annotating the third data set to obtain attribute information of obstacles on the target driving section, wherein the attribute information includes: location, appearance, category, and motion state of the obstacles;

[0044] Step S104 , using data frames as units, the first data set, the second data set, the third data set, the parsing results, the motion state information, and the attribute information are associated and stored to construct a scene database.

[0045] The control data acquisition device can be a vehicle equipped with an autonomous driving system or a vehicle equipped with an autonomous driving system and equipped with relevant sensors such as radar, cameras, and high-precision positioning equipment. This vehicle, therefore, possesses both autonomous driving and radar detection capabilities. The vehicle is capable of collecting data from the target driving section and filtering it to generate corresponding datasets, including: collecting road data to generate a first dataset, collecting environmental data to generate a second dataset, and collecting obstacle data and scene data to generate a third dataset.

[0046] The data analysis process targets a first dataset generated from road data of the target travel section and a second dataset generated from environmental data of the target travel section. The analysis results contain key information about the target travel section, including radar point cloud data, image data, positioning data, and map data. The analysis results are further processed to obtain information about the vehicle's motion state, including the data acquisition device's trajectory, driving speed, driving acceleration, and heading angle.

[0047] The data annotation process can be manual or semi-automatic. Semi-automatic annotation involves predicting annotation boxes using a deep learning algorithm, followed by manual verification. The result of this data annotation process is obstacle attribute information, including its location, appearance, category, and motion status.

[0048] The association relationship of the above-mentioned associative storage exists between all the first data sets, second data sets, third data sets, parsing results, motion state information and attribute information obtained by the above-mentioned operations, and the storage unit is data frame. A scene database is constructed through this associative storage method.

[0049] Table 1

[0050]

[0051] Table 1 is a schematic diagram of the scene database attribute entries and descriptions according to one optional embodiment of the present invention. As shown in Table 1, the scene database is constructed using MySQL software (or other software). The scene database stores data in data frames as the smallest unit. The database attributes include basic information for each frame of data and all first, second, and third data sets obtained through the above operations, as well as parsing results, motion state information, and attribute information.

[0052] Specifically, the basic information of each frame of data in the database attributes includes: id (a unique number for each frame of data, serving as the primary key of the data, which is generally a timestamp or a string containing a timestamp), date (data collection date, in the format of YYYYMMDD) and vehicle (collection vehicle).

[0053] Specifically, the first data set can be a raw data packet of road data collection, including: raw data from each sensor, positioning information, controller area network data (for example, vehicle control signals and vehicle body posture information), high-precision map data, and sensor calibration information. For example, database attributes include BagPath (the path of the raw data packet, i.e., the path of the first data set), Localization (positioning information path), calib_dir (calibration file path), lidar_scheme (radar configuration), and HdmapMask (high-precision map path).

[0054] Specifically, the second dataset may be generated by filtering environmental data of the target driving section, including information such as the prevailing weather, ground wetness, and driving section. For example, database attributes such as map (map name), route (route name), weather (weather), and road_condition (road conditions, mainly referring to ground wetness) may be used.

[0055] Specifically, the above-mentioned third data set can be generated by screening obstacle data and scene data, wherein the obstacle data includes: the type, position, relative speed, motion state, etc. of the target obstacle; the scene data includes: straight driving, turning, intersection, U-turn, ramp, tunnel, etc.

[0056] Specifically, the above analysis results can be key information such as each lidar point cloud data, each camera image data, positioning data, map data, etc. For example, database attributes include LidarMain (main lidar point cloud path), LidarTailLeft (rear left lidar point cloud path), LidarTailRight (rear right lidar point cloud path), LidarHeadMid (head center lidar point cloud path), MergePointCloudEgo (fused point cloud path), and Camera (each camera data path).

[0057] Specifically, the motion status information may include the vehicle's motion trajectory, vehicle speed, vehicle acceleration, heading angle, etc. For example, Vehicle_status (vehicle's motion trajectory, vehicle speed, vehicle acceleration, heading angle, etc.) in the database attributes.

[0058] Specifically, the obstacle attribute information obtained by the above data annotation (not listed in the table) includes: obstacle information (position, direction, size, shape, etc.), obstacle category (large vehicles, small vehicles, pedestrians, non-motor vehicles and other unidentified obstacles, etc.), and obstacle motion status (motion direction, speed, acceleration, angular velocity, trajectory, etc.).

[0059] Since the above-mentioned scene database construction method uses the data frame as the minimum management unit, the scene information of each data frame is added with the type, motion state, number, etc. of the obstacles contained, which increases the dimension of the scene feature description and provides richer conditions for data screening, thereby facilitating subsequent data processing and application.

[0060] Optionally, in step S12, filtering the scene database according to the query request to obtain the query result may include the following execution steps:

[0061] Step S121: Parse the query request to obtain multi-dimensional feature conditions;

[0062] Step S122 , filtering and obtaining query results that meet the multi-dimensional feature conditions from the scene database, wherein the query results include: point cloud data corresponding to multiple point cloud frames.

[0063] The query request can be a request to obtain specific data during actual use. For example, the specific data can include the cause of the problem, data type, and acquisition date. This request is parsed to obtain multi-dimensional feature conditions. Based on these multi-dimensional feature conditions, all point cloud frames that meet the conditions are filtered from the scene database. The information corresponding to each point cloud frame in these points is exported as the query result.

[0064] In one optional embodiment of the present invention, the screening items of the multi-dimensional feature conditions include:

[0065] (1) Natural scene characteristics, including but not limited to: weather (sunny, rainy, foggy, snowy, dusty, etc.);

[0066] (2) Road scene characteristics, including but not limited to: ordinary urban roads, intersections, expressways, internal roads in industrial parks, ramps, underbridges, ramps, tunnels, roundabouts, national highways, provincial highways, and rural roads;

[0067] (3) Driving scene characteristics, including but not limited to: going straight, turning left, turning right, U-turn, etc.;

[0068] (4) Obstacle scene characteristics, including but not limited to: cars, trucks, pedestrians, bicycles / electric vehicles, buses, trailers, engineering vehicles, police cars, ambulances, fire trucks, sprinkler trucks, pushable objects, cones, animals, temporary traffic barriers, baby strollers, and others; motion states such as moving, stopped, and temporarily stopped;

[0069] (5) Driving state characteristics of the vehicle, including but not limited to: sudden stop, acceleration, turning, etc.;

[0070] (6) Characteristics of special problem scenarios, including but not limited to: missed detection of certain obstacles, speed at which stationary obstacles appear, etc.

[0071] For example, in actual application, if it is necessary to analyze the data of an intelligent connected car "turning left at an intersection on a rainy day and encountering a sprinkler truck", it is necessary to filter the entire point cloud frame dataset D1 of "rainy day" from the scene database through the above (1) natural scene features, and then filter the dataset D1 for "intersection" in the above (2) road scene features to obtain the point cloud frame dataset D2, and then filter the dataset D2 for "left turn" in the above (3) driving scene features to obtain the point cloud frame dataset D3, and then filter the dataset D3 for "sprinkler truck" in the above (4) obstacle scene features to obtain the point cloud frame dataset D4, and finally export the dataset D4 as the query result.

[0072] Through the above multi-dimensional query results, the data required for actual use can be more accurately screened out from a large amount of scene data, making it easier to complete data analysis and processing in a more targeted manner.

[0073] Optionally, in step S14, data verification is performed on the query result, and obtaining the verification result may include the following execution steps:

[0074] In step S141, multiple data verification is performed on the point cloud data corresponding to the multiple point cloud frames to obtain verification results, wherein the multiple data verification includes: single-frame point cloud quality verification, data size verification, coordinate system verification, and time verification.

[0075] To ensure the quality of the resulting point cloud data, multiple data verifications are performed. This process can be accomplished using a verification module comprised of multiple verification units, including single-frame point cloud quality verification, data size verification, coordinate system verification, and time verification. Point cloud data that passes these multiple verifications is exported as the verification result.

[0076] In one optional embodiment of the present invention, quadruple data verification is performed on the point cloud data obtained by the query, including:

[0077] (1) Single-frame point cloud quality verification: Verify the fused point cloud of multiple point clouds obtained by query. The specific method is as follows: Since the point cloud attributes include the three-dimensional coordinates of the point cloud (x, y, z), reflection intensity i, timestamp t, and radar number sensor_id, obtain the point cloud data of a certain sub-radar, and use the sensor_id of the sub-radar to verify whether its storage location is correct; obtain the point cloud data after fusion of each sub-radar, and use the sensor_id of each sub-radar to verify whether the point cloud data after fusion of the frame is missing one or more sub-radars.

[0078] (2) Data size verification: Verify the size of the data file of the query result. The specific method is as follows: Since the size of the 3D point cloud data file is positively correlated with the number of point clouds saved in the file, under normal circumstances, when the number of saved point clouds is constant, the size of the data file will fluctuate within a certain reasonable range. Therefore, verifying the size of the data file according to the determined reasonable range can determine whether there is any abnormal point cloud storage in the data file.

[0079] (3) Coordinate system verification: Verify the coordinate system of the query point cloud. The specific method is: In autonomous driving, the coordinate systems involved in the three-dimensional point cloud data are the radar coordinate system, the vehicle coordinate system, and the world coordinate system. Usually, the three-dimensional point cloud data sent by the radar is in the radar coordinate system by default, but in actual application, the system will convert the three-dimensional point cloud data among the radar coordinate system, the vehicle coordinate system, and the world coordinate system as needed. Therefore, verifying the coordinate information of the three-dimensional point cloud data can determine whether there are coordinate system anomalies in multiple three-dimensional point cloud data obtained at a certain time.

[0080] (4) Time verification: Verify the timestamps of the data corresponding to the multiple point cloud frames obtained by the query. The specific method is to obtain the data of each sub-radar, image, positioning, and annotation results corresponding to a certain point cloud frame, and verify whether the time stamps corresponding to multiple data in the point cloud frame are the same or the difference is within a certain threshold to verify whether there is a time anomaly in the point cloud frame.

[0081] Through the above-mentioned multiple data verification, the quality of the data in the query results can be improved, at least in terms of single-frame point cloud quality, data size, coordinate system and time, ensuring a certain degree of high quality of the data.

[0082] Optionally, in step S16, performing data processing on the verification result to obtain a target processing result may include the following execution steps:

[0083] Step S161, performing single-frame data cropping on the verification result to obtain a cropped result, wherein the single-frame data cropping is used to retain a point cloud area of ​​interest from the verification result;

[0084] Step S162: performing multi-frame data fusion on the cropping result to obtain a fusion result, wherein the multi-frame data fusion is used to stitch the point cloud area of ​​interest into the same coordinate system;

[0085] Step S163 , performing data filtering on the fusion result to obtain a target processing result, wherein the data filtering is used to screen out obstacle categories within the point cloud area of ​​interest.

[0086] Optionally, in step S162, performing multi-frame data fusion on the cropping result to obtain the fusion result may include the following execution steps:

[0087] Step S1621: When the data acquisition device for acquiring the target data set is configured with multiple sub-radars, the point cloud data acquired by the multiple sub-radars at the current moment are converted to the same vehicle body coordinate system to obtain a single frame of point cloud data.

[0088] Step S1622: converting the point cloud data collected by the multiple sub-radars at adjacent times to the current time from the radar coordinate system at the adjacent times to the world coordinate system, and then converting it from the world coordinate system to the radar coordinate system at the current time to obtain converted point cloud data;

[0089] In step S1623 , the single-frame point cloud data and the converted point cloud data are spliced ​​and fused to obtain a fusion result.

[0090] The above-mentioned single-frame data cropping can crop each frame of point cloud data in the verification result, and the result is that the point cloud area of ​​interest in the actual use process in the verification result is retained. The above-mentioned multi-frame data fusion can include two steps: splicing the cropped point cloud area to the same coordinate system to generate a fused complete point cloud; for the fused complete point cloud, another complete point cloud of an adjacent time stamp can be fused as needed. During this fusion process, the aforementioned other complete point cloud needs to be converted from the radar coordinate system of the corresponding moment to the world coordinate system, and then converted to the radar coordinate system of the complete point cloud at the current moment, and then the two are spliced ​​and fused. The above-mentioned data filtering is mainly aimed at filtering obstacle type scenes. The result is to filter out the obstacle categories in the point cloud area of ​​interest during actual use, thereby eliminating the interference of other types of obstacles.

[0091] In one optional embodiment of the present invention, data processing is performed on the verification results, including data clipping, data fusion and data filtering. Specifically, the data processing process is implemented as follows:

[0092] (1) For the cropping of single-frame 3D point cloud data in the verification results, the cropping methods include but are not limited to:

[0093] a. Horizontal cropping: With the ego vehicle as the center, all point clouds are cropped into three point cloud regions based on the distance from the point cloud to the ego vehicle, including: close-range point cloud (the distance from the point cloud to the ego vehicle is less than 30 meters), medium-range point cloud (the distance from the point cloud to the ego vehicle is between 30 meters and 50 meters), medium-long-range point cloud (the distance from the point cloud to the ego vehicle is between 50 meters and 70 meters), and long-range point cloud (the distance from the point cloud to the ego vehicle is greater than 70 meters).

[0094] b. Height clipping: With the vehicle as the center (Z-axis coordinate 0), clipping is performed based on the point cloud height coordinate (z).

[0095] In particular, for the fused point cloud, the required sub-radar is screened according to the sub-radar number sensor_id, and different conditions are cropped for each sub-radar according to actual needs.

[0096] For example, a data collection vehicle is equipped with seven sub-radars, numbered N1 to N7. In actual use, the point cloud data of sub-radars N3 and N4 need to be cropped. The specific method is as follows:

[0097] Obtain 10 point cloud data points corresponding to sub-radar N3, numbered N3001 to N3010, with coordinate information of N3001 (10, 2), N3002 (35, 3), N3003 (2, 6), N3004 (55, 1), N3005 (75, 1), N3006 (3, 4), N3007 (60, 5), N3008 (40, 5), N3009 (7, 2), and N3010 (11, 3). The format of the coordinate information is (horizontal distance from the center of the vehicle, Z height coordinate), and the unit is meters.

[0098] Obtain 10 point cloud data points corresponding to sub-radar N4, numbered N4001 to N4010, with coordinate information of N4001 (35, 1), N4002 (5, 1), N4003 (75, 2), N4004 (10, 2), N4005 (40, 3), N4006 (55, 6), N4007 (6, 7), N4008 (45, 5), N4009 (7, 4), and N4010 (60, 3). The format of the coordinate information is (horizontal distance from the center of the vehicle, Z-axis height), and the unit is meters.

[0099] The above point cloud data are horizontally cropped. The close-range point clouds are: N3001, N3003, N3006, N3009, N3010, N4002, N4004, N4007 and N4009; the medium-range point clouds are: N3002, N3008, N4001, N4005 and N4008; the medium- and long-range point clouds are: N3004, N3007, N4006 and N4010; the long-range point clouds are: N3005 and N4003.

[0100] The above point cloud data is highly clipped, leaving point clouds with Z coordinates not exceeding 3. The clipping results are: N3001, N3002, N3004, N3005, N3009, N3010, N4001, N4002, N4003, N4004, N4005 and N4010.

[0101] (2) Multi-frame data fusion is performed on the cropped results. For example, a data acquisition vehicle is equipped with 7 sub-radars, and the point cloud data collected at two adjacent times t1 and t2 are as follows:

[0102] In the vehicle coordinate system, sub-radar N1 collects point cloud data Dt11 at time t1 and point cloud data Dt21 at time t2;

[0103] In the radar coordinate system, sub-radar N2 collects point cloud data Dt12 at time t1 and point cloud data Dt22 at time t2;

[0104] In the world coordinate system, sub-radar N3 collects point cloud data Dt13 at time t1 and point cloud data Dt23 at time t2;

[0105] In the ego-vehicle coordinate system, sub-radar N4 collects point cloud data Dt14 at time t1 and point cloud data Dt24 at time t2;

[0106] In the ego-vehicle coordinate system, sub-radar N5 collects point cloud data Dt15 at time t1 and point cloud data Dt25 at time t2;

[0107] In the radar coordinate system, sub-radar N6 collects point cloud data Dt16 at time t1 and point cloud data Dt26 at time t2;

[0108] In the world coordinate system, sub-radar N7 collects point cloud data Dt17 at time t1 and point cloud data Dt27 at time t2;

[0109] First, the point cloud data Dt11~Dt17 collected by the seven sub-radars at time t1 are converted to the same vehicle body coordinate system and fused into a single-frame complete point cloud Dt1r at time t1; similarly, the point cloud data Dt21~Dt27 collected by the seven sub-radars at time t2 are converted to the same vehicle body coordinate system and fused into a single-frame complete point cloud Dt2r at time t2.

[0110] Secondly, the single-frame complete point cloud data Dt2r at time t2 is converted from the radar coordinate system at time t2 to the world coordinate system, and then converted from the world coordinate system to the radar coordinate system at time t1 to obtain the converted point cloud data Dt2r'.

[0111] Finally, the single-frame complete point cloud data Dt1r at time t1 and the converted point cloud data Dt2r' at time t2 are spliced ​​and fused in the radar coordinate system at time t1 to obtain the fusion result Dt12r.

[0112] (3) Filter the processed three-dimensional point cloud data.

[0113] Data filtering primarily targets obstacle-based scenarios. While a single frame of data typically corresponds to unique scene conditions such as weather, road conditions, and vehicle driving status, the corresponding obstacles often come in multiple categories and quantities. When practical applications require data on one or more specific obstacle categories, it's necessary to filter the obstacle-based scenarios to retain the obstacle categories of interest, thereby eliminating interference from other obstacle categories and enabling better target detection and assessment.

[0114] For example, in practical applications, it is necessary to verify the detection of small, long-range obstacles by the target detection model on ordinary urban roads during rainy days. Accordingly, the scenario screening conditions are set: Rainy Day & Ordinary Urban Road. The screening results are obtained by querying the scenario database. Data verification is performed to improve the quality of the screening results, resulting in a verification result. The data frames in the verification result are subjected to the aforementioned data cropping and data fusion operations, returning a long-range multi-frame fused point cloud. The data annotation results from step S103 are read, which contain obstacle attribute information. Based on this obstacle attribute information, the long-range multi-frame fused point cloud returned by the aforementioned operation is filtered for obstacles, and data frames containing small obstacles such as bicycles and motorcycles are saved to obtain the data filtering results. The final filtered result returned is "data frames containing small obstacles such as bicycles and motorcycles in the long-range point cloud on ordinary urban roads during rainy days."

[0115] In this embodiment, a data processing device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments. Details that have been described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0116] Figure 2 is a structural block diagram of a data processing device according to one embodiment of the present invention. Figure 2 As shown, the device includes: a scene database construction module 21, which is used to construct a scene database based on the collected target data set, wherein the scene database uses a data frame as the minimum storage management unit; a screening module 22, which is used to obtain a query request and filter the scene database according to the query request to obtain a query result, wherein the information carried in the query request includes: multi-dimensional feature conditions; a data verification module 23, which is used to perform data verification on the query result to obtain a verification result; and a data processing module 24, which is used to perform data processing on the verification result to obtain a target processing result.

[0117] Optionally, the scene database construction module 21 includes: an acquisition unit 211 (not shown in the figure), which is used to control the data acquisition device to collect road data on the target driving section to generate a first data set, control the data acquisition device to collect environmental data on the target driving section to generate a second data set, and control the data acquisition device to collect obstacle data and scene data on the target driving section to generate a third data set, wherein the data acquisition device has an automatic driving function and a radar detection function; a data analysis unit 212 (not shown in the figure), which is used to analyze the first data set and the second data set to obtain an analysis result, and obtain the motion of the data acquisition device based on the analysis result. Status information, wherein the analysis results include: radar point cloud data, image data, positioning data, map data, and the motion status information includes: motion trajectory, driving speed, driving acceleration, and heading angle of the data acquisition device; a labeling unit 213 (not shown in the figure) is used to label the third data set to obtain attribute information of obstacles on the target driving section, wherein the attribute information includes: the position, appearance, category, and motion status of the obstacles; a storage construction unit 214 (not shown in the figure) is used to associate and store the first data set, the second data set, the third data set, the analysis results, the motion status information, and the attribute information in units of data frames to construct a scene database.

[0118] Optionally, the screening module 22 is further used to: parse the query request to obtain multi-dimensional feature conditions; and filter the query results that meet the multi-dimensional feature conditions from the scene database, wherein the query results include: point cloud data corresponding to multiple point cloud frames.

[0119] Optionally, the verification module 23 is also used to perform multiple data verification on the point cloud data corresponding to multiple point cloud frames to obtain verification results, wherein the multiple data verification includes: single-frame point cloud quality verification, data size verification, coordinate system verification, and time verification.

[0120] Optionally, the data processing module 24 includes: a cropping unit 241 (not shown in the figure), which is used to perform single-frame data cropping on the verification result to obtain a cropped result, wherein the single-frame data cropping is used to retain the point cloud area of ​​interest from the verification result; a fusion unit 242 (not shown in the figure), which performs multi-frame data fusion on the cropping result to obtain a fusion result, wherein the multi-frame data fusion is used to splice the point cloud area of ​​interest into the same coordinate system; a filtering unit 243 (not shown in the figure), which performs data filtering on the fusion result to obtain a target processing result, wherein the data filtering is used to filter out the obstacle categories in the point cloud area of ​​interest.

[0121] Optionally, the fusion unit 242 (not shown in the figure) is also used to: when the data acquisition device used to acquire the target data set is configured with multiple sub-radars, convert the point cloud data collected by the multiple sub-radars at the current moment into the same vehicle body coordinate system to obtain single-frame point cloud data; convert the point cloud data collected by the multiple sub-radars at adjacent moments of the current moment first from the radar coordinate system of the adjacent moments to the world coordinate system, and then convert from the world coordinate system to the radar coordinate system of the current moment to obtain converted point cloud data; splice and fuse the single-frame point cloud data with the converted point cloud data to obtain a fusion result.

[0122] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0123] An embodiment of the present invention further provides a non-volatile storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when running.

[0124] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0125] Step S1, constructing a scene database based on the collected target data set, wherein the scene database uses a data frame as the minimum storage management unit;

[0126] Step S2: obtaining a query request and filtering the query results from the scene database according to the query request, wherein the information carried in the query request includes: multi-dimensional feature conditions;

[0127] Step S3, performing data verification on the query result to obtain a verification result;

[0128] Step S4: perform data processing on the verification result to obtain the target processing result.

[0129] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0130] An embodiment of the present invention further provides a processor, which is configured to run a computer program to execute the steps in any of the above method embodiments.

[0131] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0132] Step S1, constructing a scene database based on the collected target data set, wherein the scene database uses a data frame as the minimum storage management unit;

[0133] Step S2: obtaining a query request and filtering the query results from the scene database according to the query request, wherein the information carried in the query request includes: multi-dimensional feature conditions;

[0134] Step S3, performing data verification on the query result to obtain a verification result;

[0135] Step S4: perform data processing on the verification result to obtain the target processing result.

[0136] An embodiment of the present invention further provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0137] Optionally, specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0138] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0139] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0140] In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, and can be electrical or other forms.

[0141] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs.

[0142] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0143] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0144] The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A data processing method, characterized in that: include: Building a scene database based on the collected target data set, wherein the scene database uses a data frame as the minimum storage management unit; Obtaining a query request, and filtering the scene database according to the query request to obtain a query result, wherein the information carried in the query request includes: multi-dimensional feature conditions, and the query result includes: point cloud data corresponding to a plurality of point cloud frames, wherein the multi-dimensional feature conditions include: natural scene features, road scene features, driving scene features, obstacle scene features, vehicle driving state features, and special problem scene features; Performing multiple data verification on the point cloud data corresponding to the multiple point cloud frames to obtain the verification result, wherein the multiple data verification includes: single-frame point cloud quality verification, data size verification, coordinate system verification, and time verification; Performing single-frame data cropping on the verification result to obtain a cropping result, wherein the single-frame data cropping is used to retain the point cloud area of ​​interest from the verification result; performing multi-frame data fusion on the cropping result to obtain a fusion result, wherein the multi-frame data fusion is used to splice the point cloud area of ​​interest into the same coordinate system; performing data filtering on the fusion result to obtain the target processing result, wherein the data filtering is used to filter out the obstacle categories in the point cloud area of ​​interest.

2. The data processing method according to claim 1, wherein: Constructing the scene database based on the target data set includes: controlling a data acquisition device to collect road data on a target driving section to generate a first data set, controlling the data acquisition device to collect environmental data on the target driving section to generate a second data set, and controlling the data acquisition device to collect obstacle data and scene data on the target driving section to generate a third data set, wherein the data acquisition device has an automatic driving function and a radar detection function; performing data parsing on the first data set and the second data set to obtain parsing results, and acquiring motion state information of the data acquisition device based on the parsing results, wherein the parsing results include: radar point cloud data, image data, positioning data, and map data; and the motion state information includes: motion trajectory, driving speed, driving acceleration, and heading angle of the data acquisition device; Performing data labeling on the third data set to obtain attribute information of obstacles on the target driving section, wherein the attribute information includes: the location, appearance, category, and motion state of the obstacles; The first data set, the second data set, the third data set, the analysis result, the motion state information, and the attribute information are associated and stored in units of the data frame to construct the scene database.

3. The data processing method according to claim 1, wherein: Filtering the scene database according to the query request to obtain the query result includes: Parsing the query request to obtain the multi-dimensional feature conditions; The query results that meet the multi-dimensional feature conditions are obtained by screening from the scenario database.

4. The data processing method according to claim 1, wherein: Performing multi-frame data fusion on the cropping result to obtain the fusion result includes: When the data acquisition device for acquiring the target data set is configured with multiple sub-radars, the point cloud data respectively acquired by the multiple sub-radars at the current moment are converted into the same vehicle body coordinate system to obtain a single frame of point cloud data; Converting point cloud data collected by the multiple sub-radars at adjacent moments to a world coordinate system, first from the radar coordinate system at the adjacent moments to a world coordinate system, and then converting the world coordinate system to the radar coordinate system at the current moment, to obtain converted point cloud data; The single-frame point cloud data and the converted point cloud data are spliced ​​and fused to obtain the fusion result.

5. A data processing device, characterized in that: include: A scene database construction module is used to construct a scene database based on the collected target data set, wherein the scene database uses a data frame as the minimum storage management unit; a screening module, configured to obtain a query request and filter the scene database according to the query request to obtain a query result, wherein the information carried in the query request includes: multi-dimensional feature conditions, and the query result includes: point cloud data corresponding to multiple point cloud frames, wherein the multi-dimensional feature conditions include: natural scene features, road scene features, driving scene features, obstacle scene features, vehicle driving state features, and special problem scene features; a data verification module, configured to perform multiple data verifications on the point cloud data corresponding to the plurality of point cloud frames to obtain the verification results, wherein the multiple data verifications include: single-frame point cloud quality verification, data size verification, coordinate system verification, and time verification; A data processing module is used to perform single-frame data cropping on the verification result to obtain a cropping result, wherein the single-frame data cropping is used to retain the point cloud area of ​​interest from the verification result; perform multi-frame data fusion on the cropping result to obtain a fusion result, wherein the multi-frame data fusion is used to splice the point cloud area of ​​interest into the same coordinate system; and perform data filtering on the fusion result to obtain the target processing result, wherein the data filtering is used to filter out the obstacle categories within the point cloud area of ​​interest.

6. A non-volatile storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the data processing method according to any one of claims 1 to 4 when run.

7. A processor, characterized in that: The processor is configured to run a program, wherein the program is configured to execute the data processing method according to any one of claims 1 to 4 when run.

8. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the data processing method according to any one of claims 1 to 4.

Images