Data processing method, computer equipment and storage medium

Abstract

The invention discloses a data processing method, computer equipment and a storage medium, and belongs to the technical field of automatic driving. The method disclosed by the invention mainly comprises the following steps: acquiring perception data and visual calibration data required by business processing based on a business type, and then performing general analysis on the data to acquire coordinate conversion parameter information, and constructing a fusion frame based on the sensing data, the coordinate conversion parameters and the truth value data, and performing unified coordinate system conversion on the object coordinates to obtain to-be-processed data, wherein the obtained to-be-processed data can be subjected to corresponding service processing according to different service types. According to the method provided by the invention, resource waste caused by the fact that multiple services independently develop a data preprocessing module in an algorithm iteration process can be reduced, the algorithm iteration efficiency of data preprocessing of each service can be greatly improved, and meanwhile, the maintenance cost of a general data processing part of each service is reduced.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a data processing method, a computer device, and a storage medium. Background Art

[0002] With the rapid development of autonomous driving today, the iteration of algorithms cannot be separated from large-scale data feeding. Today, autonomous driving includes many businesses, such as lane line recognition, obstacle recognition, traffic light recognition, etc. The data form required by each business is completely different. 4DGT in autonomous driving refers to 4D Annotation Technology. 4D Annotation Technology adds time series information on the basis of 3D annotation to form a four-dimensional annotation system. Although the data sources of most autonomous driving businesses have been unified at the level of 4DGT production and manual annotation, the true value data produced by 4DGT itself contains time series information, and the model training data of each business is time series data composed of a single frame or multiple single frames, which requires complex processing logic to complete the conversion. With the iteration of the three-dimensional scene established by the data and the iteration of the vehicle model, each business needs to be compatible with multiple scenes, which will increase the difficulty of maintaining the data processing link that produces the training data of each business model, reduce the development efficiency, and cause a lot of development cost waste.

[0003] To this end, the present application constructs a set of data processing methods that are common to multiple businesses to solve the above problems. Summary of the invention

[0004] In order to overcome the above-mentioned defects, the data processing method, computer equipment and storage medium proposed in this application can solve the problem that the 4DGT data of different algorithm services in the autonomous driving model each need to be compatible with multiple scenarios, resulting in high difficulty in maintaining the data processing link.

[0005] In a first aspect, the present application proposes a data processing method, comprising:

[0006] Acquire perception data and visual calibration data based on the service type, wherein the perception data includes sensor frames collected by different sensors;

[0007] Acquire coordinate conversion parameters based on the visual calibration data, wherein the coordinate conversion parameters are used to convert the perception data into a unified coordinate system;

[0008] Based on the perception data and the coordinate transformation parameters, converting the sensor frame into a fused frame with annotation information;

[0009] Based on the coordinate transformation parameters, the fused frame is coordinate transformed to obtain data to be processed in a unified coordinate system;

[0010] Perform business processing on the data to be processed according to the business type.

[0011] In some implementation schemes, the acquiring of the perception data and the visual calibration data based on the business type includes: based on a preset downloader, downloading the perception data and the visual calibration data from a download address corresponding to the business type.

[0012] Furthermore, the visual calibration data includes calibration data and posture data, the calibration data is used for sensor calibration, and the posture data is used for posture estimation;

[0013] The coordinate conversion parameters include a first coordinate conversion parameter and a second coordinate conversion parameter, the first coordinate conversion parameter is used to perform coordinate conversion on the calibration data, and the second coordinate conversion parameter is used to perform coordinate conversion on the posture data;

[0014] In some embodiments, obtaining coordinate transformation parameters based on the visual calibration data includes:

[0015] Obtaining a first conversion matrix and a second conversion matrix based on the calibration data, and determining the first coordinate conversion parameter according to the first conversion matrix and the second conversion matrix;

[0016] Obtain a third transformation matrix based on the posture data, and determine the second coordinate transformation parameters according to the third transformation matrix;

[0017] The first conversion matrix is ​​used to realize data conversion from the vehicle coordinate system to the pixel coordinate system, and the second conversion matrix and the third conversion matrix are used to realize data conversion from the vehicle coordinate system to the world coordinate system.

[0018] Further, when the posture data includes posture data in the vehicle coordinate system, obtaining the third transformation matrix based on the posture data includes: parsing the posture data in the vehicle coordinate system to obtain the third transformation matrix.

[0019] When the posture data includes posture data in a radar coordinate system, obtaining the third transformation matrix based on the posture data includes: parsing the posture data in the radar coordinate system to obtain a fourth transformation matrix and a fifth transformation matrix, and performing matrix operations on the fourth transformation matrix and the fifth transformation matrix to obtain a third transformation matrix; wherein the fourth transformation matrix is ​​used to realize data conversion from the radar coordinate system to the world coordinate system, and the fifth transformation matrix is ​​used to realize data conversion from the radar coordinate system to the vehicle coordinate system.

[0020] In some embodiments, the converting the sensor frame into a fused frame with annotation information based on the perception data and the coordinate conversion parameters comprises:

[0021] Acquire true value data with annotation information corresponding to the sensor frame;

[0022] Based on the timestamp of the sensor frame and the coordinate conversion parameter, a plurality of the sensor frames at the same time are fused and aligned with the true value data with the labeled information to obtain the fused frame with the labeled information.

[0023] In some implementation schemes, the business processing of the data to be processed according to the business type includes: performing frame-by-frame labeling processing on the data to be processed according to the target labeling requirements corresponding to the business type, obtaining corresponding multiple single-frame target labeling results, and aggregating the multiple single-frame target labeling results.

[0024] Furthermore, the single-frame target labeling result includes target object attribute information;

[0025] In some embodiments, the business processing of the data to be processed according to the business type also includes: configuring the target object attribute information according to a predefined data structure, wherein the configuration content includes at least one of the following: object-level attributes corresponding to the target object and point-level attributes corresponding to key points constituting the target object.

[0026] In a second aspect, the present application provides a computer device comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the data processing method described in any one of the technical solutions of the above-mentioned data processing method is implemented.

[0027] In a third aspect, the present application provides a computer-readable storage medium, which stores a plurality of program codes, wherein the program codes are suitable for being loaded and run by a processor to execute the data processing method described in any one of the technical solutions of the above-mentioned data processing method.

[0028] The above one or more technical solutions of the present application have at least one or more of the following beneficial effects:

[0029] In the technical solution for implementing the present application, a data processing method is provided. For the perception data and visual calibration data corresponding to different business types, the visual calibration data is generally analyzed and processed by the method provided by the present application to obtain coordinate conversion parameters. A fused frame with annotation information is obtained based on the perception data and the coordinate conversion parameters, and the annotations in the fused frame are converted to a unified coordinate system to obtain the data to be processed. The data to be processed is universal and can be applied to different business processing. The multi-business universal data processing method designed in the present application is mainly used for data preprocessing of autonomous driving model algorithm business. It can reduce the waste of resources caused by the independent development of data preprocessing modules for multiple businesses during the algorithm iteration process, and can greatly improve the algorithm iteration efficiency of data preprocessing of each business, while reducing the maintenance cost of the common data processing part of each business. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The disclosure of the present application will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. In addition, similar numbers in the drawings are used to represent similar components, among which:

[0031] Figure 1 It is a schematic flow chart of the main steps of a data processing method according to an embodiment of the present application;

[0032] Figure 2 It is a schematic diagram of an application scenario of a data processing method according to an embodiment of the present application;

[0033] Figure 3 This is a schematic diagram of the main working principle of a file download module in an embodiment of the present application;

[0034] Figure 4 This is a schematic diagram of the main working principle of a file parsing module in an embodiment of the present application;

[0035] Figure 5 It is a schematic diagram of the main working principle of a single-frame building module in an embodiment of the present application;

[0036] Figure 6 It is a hardware environment diagram of a data processing method according to an embodiment of the present application.

[0037] 100: common module, 200: business customized module, 11: memory, 12: processor. DETAILED DESCRIPTION

[0038] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.

[0039] In the description of the present application, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, memory, and may also include software parts, such as program code, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware or a combination of the two. Non-temporary computer-readable storage media include any suitable medium that can store program code, such as a disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B" and may include only A, only B or A and B. The singular terms "one" and "the" may also include plural forms.

[0040] Here are some terms involved in this application:

[0041] The world coordinate system, also known as the Cartesian coordinate system, is a fixed absolute space coordinate system with its origin fixed at a specific location (usually the center of the earth). It is usually used to describe the position and direction of objects in three-dimensional space. Each point in the world coordinate system can be represented by a three-dimensional vector (x, y, z), where x represents the position of the point on the x-axis, y represents the position of the point on the y-axis, and z represents the position of the point on the z-axis.

[0042] The ego-vehicle coordinate system, also known as the vehicle coordinate system, is a coordinate system centered on the vehicle itself and used to describe the movement of the vehicle. Its origin usually coincides with the center of mass of the vehicle, where the positive direction of the x-axis is the forward direction of the vehicle, the positive direction of the y-axis is the left direction of the vehicle, and the positive direction of the z-axis is vertically upward.

[0043] The radar coordinate system is a coordinate system used to describe the relative position of an object and a lidar. The origin of the coordinate system is the geometric center of the lidar, where the x-axis is horizontally forward, usually pointing directly in front of the lidar, the y-axis is horizontally left, pointing to the left side of the lidar, and the z-axis is vertically upward, conforming to the right-hand coordinate system rule.

[0044] The pixel coordinate system is a coordinate system with the upper left corner of the image as the origin, the u axis to the right horizontally, and the v axis vertically downward. It is mainly used to describe the position of each pixel on the image. In this coordinate system, the position of the pixel can be expressed as [u, v].

[0045] The following describes the embodiments of the present application in conjunction with the drawings in the specification. It should be understood that the embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments in the present application and the features in the embodiments may be combined with each other if there is no conflict.

[0046] like Figure 1 As shown, it is the main step flow of a data processing method provided by an embodiment of the present application. Based on the data processing method provided by the embodiment of the present application, it can be applied to the processing of 4DGT data of different businesses and produce model training data for each business. The 4DGT data in this embodiment can be understood as mainly including true value data with annotation information produced by 4D annotation. As shown in the figure, the data processing method provided by this embodiment mainly includes the following steps: Step S11 to Step S15:

[0047] Step S11: acquiring perception data and visual calibration data based on business type;

[0048] In this embodiment, the perception data includes sensor frames collected by different sensors. For example, the perception data corresponding to one service type may specifically include image frames collected by multiple cameras. For example, the perception data corresponding to another service type may specifically include image frames collected by multiple cameras and point cloud frames collected by a lidar. The visual calibration data may include calibration data for sensor calibration and pose data for pose estimation. The service types may include lane line recognition, obstacle recognition, traffic light recognition, etc.

[0049] A specific implementation of this step S11 is as follows: based on a preset downloader, the perception data and the visual calibration data are downloaded from a download address corresponding to the business type. In this embodiment, the perception data and the visual calibration data are stored in the form of files in a data storage location corresponding to the download address. For example, the data storage location corresponding to the download address stores at least a perception data file, a calibration data file, and a posture data file corresponding to the business type. Furthermore, the perception data file can also be stored as an image data file and a point cloud data file according to different sensors.

[0050] Exemplarily, the perception data and visual calibration data in this application can be obtained by designing a file download module, and the file download module is configured to provide a parallel downloader and download logic for files required for 4DGT processing. If each business needs to download additional files, you only need to pay attention to the original download address and the target download address. The design of the file download module realizes the general file management of the entire 4DGT processing flow. The change iteration at the production line file level will not be directly transmitted to the business, reducing business coupling. The production line file described in this application can be understood as a file used to record the data required for the training data production link (such as perception data and visual calibration data). An example of a practical problem solved by the file download module is that the data required for 4DGT processing is migrated from data set 1.0 to data set 2.0. If all businesses need to adapt to the download of the new data set according to the logic maintained by each business, the file download module based on this application only needs to replace the downloader 1.0 with the downloader 2.0 to complete the upgrade, thereby reducing the adaptation cost of each business.

[0051] Step S12: acquiring coordinate conversion parameters based on the visual calibration data;

[0052] In this embodiment, the coordinate conversion parameters are used to convert the perception data into a unified coordinate system. The coordinate conversion parameters may include a first coordinate conversion parameter and a second coordinate conversion parameter. The first coordinate conversion parameter is used to perform coordinate conversion on the calibration data, and the second coordinate conversion parameter is used to perform coordinate conversion on the posture data.

[0053] A specific implementation of step S12 may include steps 121 and 122 as shown below:

[0054] Step 121: parsing the calibration data to obtain a first conversion matrix and a second conversion matrix, and determining the first coordinate conversion parameter according to the first conversion matrix and the second conversion matrix;

[0055] Step 122: parsing the posture data to obtain a third transformation matrix, and determining the second coordinate transformation parameters according to the third transformation matrix;

[0056] Among them, the first conversion matrix is ​​used to realize the data conversion from the vehicle coordinate system to the pixel coordinate system, and the second conversion matrix and the third conversion matrix are used to realize the data conversion from the vehicle coordinate system to the world coordinate system. In addition, the first coordinate conversion parameter in the embodiment of the present application can be understood as a parameter including calibration data, the first conversion matrix and / or the second conversion matrix; the second coordinate conversion parameter can be understood as a parameter including posture data and the third conversion matrix.

[0057] Further, the posture data may include posture data in a radar coordinate system and / or posture data in a vehicle coordinate system, and parsing the posture data to obtain a third transformation matrix in step 122 may specifically include:

[0058] Analyze the pose data in the ego-vehicle coordinate system to obtain a third transformation matrix;

[0059] And / or, parsing the posture data in the radar coordinate system to obtain a fourth transformation matrix and a fifth transformation matrix, and performing matrix operations on the fourth transformation matrix and the fifth transformation matrix to obtain a second transformation matrix; wherein the fourth transformation matrix is ​​used to realize data conversion from the radar coordinate system to the world coordinate system, and the fifth transformation matrix is ​​used to realize data conversion from the radar coordinate system to the vehicle coordinate system.

[0060] In the embodiment of the present application, the method of parsing the calibration data or posture data to obtain the transformation matrix can be implemented by adopting the existing technology of calculating the transformation matrix based on the calibration data (such as camera parameters) or posture data (such as the coordinate position of the object in the vehicle coordinate system), which will not be described in detail here.

[0061] Step S13: based on the perception data and the coordinate conversion parameters, converting the sensor frame into a fused frame with annotation information;

[0062] A specific implementation of step S13 may include the following steps 131 and 132:

[0063] Step 131: Acquire true value data with annotation information corresponding to the sensor frame;

[0064] In this embodiment, the true value data refers to high-precision data used to train models in the field of autonomous driving and machine learning. These data are usually manually labeled data so that the machine learning model can understand and use these data.

[0065] This step may be specifically as follows: obtaining the true value data based on the 4DGT data corresponding to the business type. The 4DGT data includes timing information, and the sensor frame has a timestamp. Based on the 4DGT data, the true value data corresponding to the sensor frame may be obtained, specifically: obtaining the true value data whose timing information corresponds to the timestamp of the sensor frame from the 4DGT data.

[0066] Step 132: Based on the timestamp of the sensor frame and the coordinate conversion parameter, a plurality of the sensor frames at the same time are fused and aligned with the true value data with the labeled information to obtain the fused frame with the labeled information.

[0067] In this embodiment, taking the vehicle equipped with a forward wide camera (FW) and other cameras with multiple viewing angles, a total of 11 cameras, as an example, the other multiple visual cameras include the vehicle's four side panoramic cameras (SVC), a forward narrow camera (FN), a front left camera (FL), a front right camera (FR), a rear left camera (RL), a rear right camera (RR), and a rear narrow camera (RN).

[0068] In one embodiment, when the multiple sensor frames at the same moment are specifically image frames captured by the above-mentioned 11 cameras, this step can be specifically as follows: first, the timestamps of the sensor frames captured by cameras of other multiple viewing angles are matched one-to-one with the timestamp of FW by nearest neighbor matching to obtain a time-domain aligned data frame containing multiple sensors, and then based on the second coordinate transformation parameter, the data frame is spatially aligned with the true value data with annotation information to obtain the fused frame with spatiotemporal fusion characteristics.

[0069] In another embodiment, when the multiple sensor frames at the same moment specifically include image frames captured by the above-mentioned 11 cameras and point cloud frames captured by the lidar, this step can be specifically as follows: the timestamps of the sensor frames captured by cameras of other multiple perspectives are matched one-to-one with the timestamp of FW by nearest neighbor matching, and the matching results are time-aligned with the point cloud frames to obtain a time-domain aligned data frame containing multiple sensors, and then the data frame is spatially aligned with the true value data with annotation information based on the second coordinate transformation parameter to obtain the fused frame with spatiotemporal fusion characteristics.

[0070] It can be understood that since the second coordinate transformation parameters include posture data and the third transformation matrix, the third transformation matrix can realize the data conversion from the vehicle coordinate system to the world coordinate system. Therefore, the method of obtaining the fused frame in this step can realize the unified conversion of the annotations of different services in the world coordinate system into the annotations in the vehicle coordinate system.

[0071] It should be noted that the fused frame in this embodiment can also be called a BEV (Bird's Eye View) frame. The method of obtaining the fused frame in this step can also refer to the prior art method of achieving environmental perception by converting data obtained by multiple sensors into a BEV frame.

[0072] Optionally in this embodiment, if the frame frequency of the sensor is greater than the required frame rate, the fused frame may be constructed by sampling in interval time periods.

[0073] Step S14: performing coordinate transformation on the fused frame based on the coordinate transformation parameters to obtain data to be processed in a unified coordinate system;

[0074] In this embodiment, this step specifically includes: transforming the fused frame with the annotation information from the vehicle coordinate system to the pixel coordinate system based on the first coordinate transformation parameter.

[0075] Step S15: performing business processing on the data to be processed according to the business type.

[0076] In this embodiment, this step is specifically as follows: according to the target labeling requirements corresponding to the business type, the data to be processed is labeled frame by frame to obtain corresponding multiple single-frame target labeling results, and the multiple single-frame target labeling results are aggregated.

[0077] Exemplarily, taking the business type of traffic light recognition as an example, the frame-by-frame annotation processing of the data to be processed can be understood as mainly including: coordinate conversion of the target lamp head and the lamp frame; binding the target lamp head and the lamp frame; matching the 2D algorithm pre-labeling result and the 3D labeling result, wherein the 3D labeling result can be understood as the data to be processed obtained in the above step S14, wherein the 2D algorithm pre-labeling result can be the existing manually labeled data.

[0078] The single-frame target labeling result in this step may include target object attribute information.

[0079] Optionally, the business processing of the data to be processed according to the business type in this step may also include: configuring the target object attribute information according to a predefined data structure, wherein the configuration content includes at least one of the following: object-level attributes corresponding to the target object and point-level attributes corresponding to key points constituting the target object; for example, the object-level attributes can be used to characterize the object category, and the point-level attributes can be used to characterize the color of the key points; further, the object-level attributes can also be used to characterize the occlusion ratio of the object in different sensors, such as the occlusion ratio of the object in the lateral direction camera; the point-level attributes can also be used to characterize whether the key points are occluded in different sensors, such as whether the key points are occluded in the lateral direction camera.

[0080] The data obtained based on the data processing method described in steps S11 to S15 can be used as labeled data for training an autonomous driving model, such as for training a vehicle perception model. The training of a vehicle perception model usually requires a large amount of labeled data of different business types, such as labeled data with obstacle attribute information and labeled data with lane line attribute information. In actual applications, the data output from the above step S15 can be provided as perception training data to a vehicle perception model for training, and the trained vehicle perception model can be used in the vehicle's perception system to enhance the vehicle's environmental perception capability.

[0081] like Figure 2As shown, it is a schematic diagram of an application scenario of a data processing method provided by an embodiment of the present application; in this application scenario, the embodiment of the present application specifically designs a data processing method that is common to multiple businesses, which can be implemented by constructing a public module 100 maintained by the data middle platform and a business custom module 200 maintained by the business party. The design of the public module 100 can realize multi-business sharing in one iteration, and the design of the business custom module 200 reduces the business coupling degree. The superposition of the two modules constructs a complete set of high-cohesion, low-coupling, multi-business common data processing framework.

[0082] Among them, the public module 100 can mainly include a file download module, a file parsing module, a single frame construction module and a coordinate conversion module, and the business custom module 200 is mainly used to realize business-related single frame processing, such as lane line (Lane) single frame processing, obstacle (OD) single frame processing, traffic light (TLD) single frame processing and other business single frame processing. For the 4DGT data of different businesses, after the multi-business general data processing provided by this application is performed based on the public module and the business custom module, the model training data that can be used for each business can be aggregated. First, multiple files are downloaded in the file download module, and then the downloaded files are generally parsed in the file parsing module to obtain relevant information for subsequent processing, and then the fusion frame is constructed and sampled according to the characteristics of the 4DGT data in the single frame construction module, and unified coordinate conversion is performed in the coordinate conversion module. After being processed by the public module, each business can convert the object coordinates in the marked coordinate system into the self-vehicle coordinate system according to a unified coordinate conversion method, and then the logic filling of single frame processing is performed in the business custom module according to the processing logic of each business, and finally the training data is obtained by aggregation.

[0083] The following is combined with Figure 3 To Attachment Figure 5 The specific implementation method of the file download module, file parsing module and single frame construction module provided in the embodiment of the present application is explained.

[0084] For example, Figure 3The figure is a schematic diagram of the working principle of a file download module provided in an embodiment of the present application. As shown in the figure, the file download module can download calibration files, posture files and perception data files related to the business based on the downloader, and provide the downloaded files to the corresponding calibration file parser, posture file parser and perception data file parser in the file parsing module, and perform file parsing and data parsing based on each parser to obtain calibration data, posture data and perception data. When the data related to the business is changed and iterated, it is only necessary to replace the downloader. For example, when the data set 1.0 is changed and iterated to the data set 2.0, it is only necessary to replace the downloader 1.0 with the downloader 2.0. For example, in the present embodiment, the perception data file can be specifically a PCD file, and the perception data file parser can be specifically a PCD parser, where PCD stands for Point Cloud Data, which is a file format for storing point cloud data.

[0085] For example, Figure 4 The figure shows a schematic diagram of the main working principle of a file parsing module provided by an embodiment of the present application. After the file parsing module in this embodiment parses and processes the file downloaded by the download module to obtain the perception data, calibration data and position data, it is mainly used to parse and process the calibration data and posture data to obtain the coordinate conversion parameters, and the coordinate conversion parameters can convert the perception data to a unified coordinate system. As shown in the figure, the file parsing module based on this embodiment can parse the calibration data and the posture data in the self-vehicle coordinate system to obtain the conversion matrix from the world coordinate system to the self-vehicle coordinate system, and parse the posture data in the radar coordinate system to obtain the conversion matrix from the radar coordinate system to the world coordinate system, and then obtain the conversion matrix from the world coordinate system to the self-vehicle coordinate system through matrix operation.

[0086] For example, Figure 5 The figure shows a schematic diagram of the main working principle of a single-frame construction module provided in an embodiment of the present application. The single-frame construction module of the present embodiment is mainly used to construct a fused frame (or BEV frame) according to the perception data and the transformation matrix obtained by the file parsing module. As shown in the figure, the perception data includes image frames collected by 11 cameras on the vehicle (SVC, FN, FL, FR, RR, RL, RN and FW as shown in the figure) at the same time. The single-frame construction module of the present embodiment is mainly used to construct a BEV frame according to the true value data and the perception data and the transformation matrix output by the file parsing module. The specific construction method can be found in the description of the method embodiment of the present application, which will not be repeated here.

[0087] Another aspect of the present application provides a computer device.

[0088] In an embodiment of a computer device according to the present application, the computer device may include at least one processor; and a memory connected to the at least one processor in communication; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the data processing method described in any of the above embodiments is implemented or the functions of the above common module and the business custom module are implemented. Figure 6 , Figure 6 exemplarily shows that the memory 11 and the processor 12 are connected via a bus communication. The computer device described in this application may be, but is not limited to, a mobile phone, a computer, a cloud server, a server cluster, etc., and this embodiment of the application does not limit this.

[0089] Furthermore, the embodiment of the present application also provides a data processing system, including a common module and a business custom module, wherein:

[0090] The common module can be configured to: acquire perception data collected by different sensors and visual calibration data corresponding to the perception data based on business type, parse and process the visual calibration data to obtain coordinate conversion parameters that can convert the perception data into a unified coordinate system, convert sensor frames collected by different sensors in the perception data into fused frames with annotation information based on the perception data and the coordinate conversion parameters, and coordinate-convert the fused frames with annotation information based on the coordinate conversion parameters to obtain data to be processed in a unified coordinate system.

[0091] The business customization module may be configured to perform business processing on the data to be processed obtained by the common module according to the business type.

[0092] In actual applications, if objects of multiple services are marked at the same time in the same marking task (such as intersection services), the advantage of the system provided based on the embodiment of the present application is that it can process multiple services at the same time. It only needs to flexibly combine the single frame processing related to each business in the common module and the business custom module according to the marking task.

[0093] In the implementation mode provided in the present application, the design of the common module can eliminate the differences caused by different car models / different three-dimensional scene construction methods / different coordinate systems / different data storage methods. In the subsequent business processing, only the customized processing logic within the business needs to be considered, which avoids unknown errors caused by repeated development and differences in processing methods, and can reduce development costs and improve development stability. The business custom module unifies the data structure, and standardizes the processing structure of each business on the premise that the business can flexibly fill in the logic, so that all businesses can coexist under the same framework, decoupling and improving reusability. This complete set of processing frameworks ensures that the data processing system is in an efficient iterative state, indirectly improving R&D efficiency.

[0094] For ease of explanation, the introduction to the above data processing system only shows the part related to the embodiment of the present application. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application.

[0095] It should be understood that since the setting of each module is only for illustrating the functional units of the present application, the physical devices corresponding to these modules may be the processor itself, or a part of the software in the processor, a part of the hardware, or a part of the combination of software and hardware. Therefore, the number of each module in the figure is only schematic.

[0096] It is understood by those skilled in the art that each module in the system can be adaptively split or merged. Such splitting or merging of specific modules will not cause the technical solution to deviate from the principle of this application, and therefore, the technical solutions after splitting or merging will fall within the protection scope of this application.

[0097] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

[0098] In the embodiment of the present application, the computer device may be a control device device including various electronic devices. In some possible implementations, the computer device may include multiple memories and multiple processors. The program for executing the data processing method of the above method embodiment can be divided into multiple subprograms, each of which can be loaded and run by a processor to execute different steps of the data processing method of the above method embodiment. Specifically, each subprogram can be stored in different memories, and each processor can be configured to execute programs in one or more memories to jointly implement the data processing method of the above method embodiment, that is, each processor executes different steps of the data processing method of the above method embodiment, to jointly implement the data processing method of the above method embodiment.

[0099] The above-mentioned multiple processors may be processors deployed on the same device. For example, the above-mentioned computer device may be a high-performance device composed of multiple processors, and the above-mentioned multiple processors may be processors configured on the high-performance device. In addition, the above-mentioned multiple processors may also be processors deployed on different devices. For example, the above-mentioned computer device may be a server cluster, and the above-mentioned multiple processors may be processors on different servers in the server cluster.

[0100] Furthermore, the present application also provides a computer-readable storage medium.

[0101] In a computer-readable storage medium embodiment according to the present application, the computer-readable storage medium may be configured to store a program for executing the data processing method of the above method embodiment, and the program may be loaded and run by a processor to implement the above data processing method. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.

[0102] The relevant user personal information that may be involved in the various embodiments of this application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.

[0103] The user personal information processed by this application will vary depending on the specific product / service scenario, and will be based on the specific scenario in which the user uses the product / service. It may involve the user's account information, device information, driving information, vehicle information or other related information. The applicant will treat the user's personal information and its processing with a high degree of diligence.

[0104] This application attaches great importance to the security of user personal information and has taken reasonable and feasible security protection measures that meet industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.

[0105] So far, the technical solutions of the present application have been described in conjunction with the embodiments shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.

Claims

1. A data processing method, characterized in that: include: Acquire perception data and visual calibration data based on the service type, wherein the perception data includes sensor frames collected by different sensors; Acquire coordinate conversion parameters based on the visual calibration data, wherein the coordinate conversion parameters are used to convert the perception data into a unified coordinate system; Based on the perception data and the coordinate transformation parameters, converting the sensor frame into a fused frame with annotation information; Based on the coordinate transformation parameters, the fused frame is coordinate transformed to obtain data to be processed in a unified coordinate system; Perform business processing on the data to be processed according to the business type.

2. The method according to claim 1, characterized in that The acquiring of perception data and visual calibration data based on the business type includes: Based on a preset downloader, the perception data and the visual calibration data are downloaded from a download address corresponding to the service type.

3. The method according to claim 1, characterized in that The visual calibration data includes calibration data and posture data, the calibration data is used for sensor calibration, and the posture data is used for posture estimation; The coordinate conversion parameters include a first coordinate conversion parameter and a second coordinate conversion parameter, the first coordinate conversion parameter is used to perform coordinate conversion on the calibration data, and the second coordinate conversion parameter is used to perform coordinate conversion on the posture data; The acquiring of coordinate conversion parameters based on the visual calibration data comprises: Obtaining a first conversion matrix and a second conversion matrix based on the calibration data, and determining the first coordinate conversion parameter according to the first conversion matrix and the second conversion matrix; Obtain a third transformation matrix based on the posture data, and determine the second coordinate transformation parameters according to the third transformation matrix; The first conversion matrix is ​​used to realize data conversion from the vehicle coordinate system to the pixel coordinate system, and the second conversion matrix and the third conversion matrix are used to realize data conversion from the vehicle coordinate system to the world coordinate system.

4. The method according to claim 3, characterized in that The posture data includes posture data in the vehicle coordinate system; The obtaining of a third transformation matrix based on the posture data comprises: The pose data in the ego-vehicle coordinate system is analyzed to obtain a third transformation matrix.

5. The method according to claim 3 or 4, characterized in that: The posture data includes posture data in a radar coordinate system; The obtaining of a third transformation matrix based on the posture data comprises: Parsing the pose data in the radar coordinate system to obtain a fourth conversion matrix and a fifth conversion matrix, and performing a matrix operation on the fourth conversion matrix and the fifth conversion matrix to obtain a third conversion matrix; The fourth conversion matrix is ​​used to realize data conversion from the radar coordinate system to the world coordinate system, and the fifth conversion matrix is ​​used to realize data conversion from the radar coordinate system to the vehicle coordinate system.

6. The method according to claim 1, characterized in that The converting the sensor frame into a fused frame with annotation information based on the perception data and the coordinate conversion parameter comprises: Acquire true value data with annotation information corresponding to the sensor frame; Based on the timestamp of the sensor frame and the coordinate conversion parameter, a plurality of the sensor frames at the same time are fused and aligned with the true value data with the labeled information to obtain the fused frame with the labeled information.

7. The method according to claim 1, characterized in that The performing business processing on the data to be processed according to the business type includes: According to the target labeling requirements corresponding to the business type, the data to be processed is labeled frame by frame to obtain corresponding multiple single-frame target labeling results, and the multiple single-frame target labeling results are aggregated.

8. The method according to claim 7, characterized in that The single-frame target labeling result includes target object attribute information; The business processing of the data to be processed according to the business type also includes: configuring the target object attribute information according to a predefined data structure, wherein the configuration content includes at least one of the following: object-level attributes corresponding to the target object and point-level attributes corresponding to key points constituting the target object.

9. A computer device, characterized in that: comprising at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores a computer program, and when the computer program is executed by the at least one processor, the data processing method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the data processing method according to any one of claims 1 to 8.

Images