Data processing method and device

Abstract

The embodiment of the present application discloses a data processing method and device, which belongs to the field of sensor technology and can be used for assisted driving and autonomous driving. The method of the embodiment of the present application includes: obtaining a first output result through a Bayesian algorithm based on the category of the first target in each frame in multiple frames and the tracking position of the first target in each frame in multiple frames, and inputting the first category, the first confidence level and the target feature into a multi-frame classifier to obtain a second output result. By Bayesian multi-frame decision fusion and inputting the target feature into the multi-frame classifier, the problem of low single-frame recognition accuracy can be compensated. This method can be used in the data processing process of sensor perception. This method improves the terminal's advanced driving assistance system ADAS capability in autonomous driving or assisted driving, and can be applied to vehicle networks, such as vehicle-to-vehicle (V2X), long-term evolution technology for vehicle-to-vehicle communication (LTE‑V), and vehicle-to-vehicle (V2V).

Description

Technical Field

[0001] The embodiments of the present application relate to the field of autonomous driving, and in particular to a data processing method and device. Background Art

[0002] Autonomous vehicles, also known as unmanned vehicles, are intelligent vehicles that operate without human intervention through computer systems. Object detection is a crucial component of autonomous vehicles. Its primary purpose is to provide visibility into the vehicle's surroundings in all weather conditions. It plays a crucial role in assisting the vehicle with obstacle avoidance and understanding its surroundings. Target detection and recognition using radar is crucial for autonomous driving.

[0003] Currently, methods for achieving target recognition through radar include: weighting the classification results of multiple single frames through a sliding window, that is, performing weighted voting on the classification results of multiple single frames within the sliding window, and determining the maximum value as the final recognized category.

[0004] However, the above weighting method is too dependent on the classification results of a single frame. If the performance of the single frame classification is not good, the determination of the category will be affected. Summary of the Invention

[0005] The embodiments of the present application provide a data processing method and related equipment, which can improve the accuracy of target recognition.

[0006] In a first aspect, an embodiment of the present application provides a data processing method, which can be executed by a processor, which can be in a radar or outside the radar (for example, a processor in a device other than the radar in a vehicle). The method includes: obtaining a target point cloud of multiple frames; obtaining a classification result of a first target for each frame in the multiple frames based on the target point cloud of the multiple frames, the classification result of the first target including the category of the first target and the confidence corresponding to the category, and the first target is associated with the target point cloud; obtaining a first output result by using a Bayesian algorithm based on the category of the first target in each frame in the multiple frames and the tracking position of the first target in each frame in the multiple frames, the first output result including the first category of the first target and the first confidence corresponding to the first category, the tracking position of the first target in each frame in the multiple frames is the position obtained by tracking the target point cloud of the multiple frames; obtaining target features based on the target point cloud of the multiple frames, the target features being the spatial distribution morphological features of the target point cloud of the multiple frames; inputting the first category, the first confidence and the target features into a multi-frame classifier to obtain a second output result, the second output result including the second category of the first target and the second confidence corresponding to the second category.

[0007] In an embodiment of the present application, a first output result is obtained by using a Bayesian algorithm based on the category of the first target in each frame of the multi-frame and the tracking position of the first target in each frame of the multi-frame. The first category, the first confidence level, and the target features are input into a multi-frame classifier to obtain a second output result. By fusion of Bayesian multi-frame decisions and inputting the target features into the multi-frame classifier, the problem of low single-frame recognition accuracy can be compensated.

[0008] Optionally, in a possible implementation of the first aspect, the step of obtaining a first output result by using a Bayesian algorithm based on the category of the first target in each frame in the multiple frames and the tracking position of the first target in each frame in the multiple frames includes obtaining the first output result by using the following method based on the category of the first target in each frame in the multiple frames and the tracking position of the first target in each frame in the multiple frames:

[0009]

[0010] Ω=arg max{P(Ω|list[c,t])};

[0011] Among them, c represents the category of the first target in each frame, t represents the tracking position of the first target in each frame, list[c,t] represents the correspondence between the category and the tracking position, Z i represents a sequence of categories of multiple first targets, Ω represents the actual category of the first target, r represents the distance to the first target, a represents the angle to the first target, and Z' represents the category of the first target.

[0012] In this possible implementation, Bayesian multi-frame decision fusion can be used to fit the performance of each frame by adjusting the prior information therein, and significantly improve the categories with poor single-frame classification results.

[0013] Optionally, in a possible implementation of the first aspect, the above steps also include: obtaining a target confidence based on the first confidence and the second confidence; if the target confidence is greater than a locking threshold, determining that the category of the first target in at least one frame after multiple frames is the second category.

[0014] In this possible implementation, for categories whose target confidence is greater than the locking threshold, the category of the first target in at least one frame after multiple frames can be determined to be the second category. That is, for categories with high confidence, the category of the first target does not need to be re-estimated in at least one frame after multiple frames, and can be directly determined as the second category. On the one hand, it can effectively reduce computing power and time overhead and improve detection efficiency. On the other hand, the locking result is used for the missed detection frame results to effectively solve the missed detection problem.

[0015] Optionally, in a possible implementation of the first aspect, the above steps also include: obtaining the number of point clouds corresponding to the first target in at least one frame after multiple frames; if the target confidence is less than the locking threshold and the number of point clouds is less than a preset value, determining that the category of the first target in at least one frame after multiple frames is the second category.

[0016] In this possible implementation, for frames in which the number of point clouds corresponding to the first target is small, the category of the first target can be directly determined to be the second category, that is, the problem of missed frames due to the small number of point clouds is reduced. By using the previous second category for the missed frame results, the missed detection problem caused by too few point clouds can be effectively solved.

[0017] Optionally, in a possible implementation of the first aspect, the above steps further include: if the target confidence is less than the locking threshold and the number of point clouds is greater than a preset value, the position of the first target, the speed of the first target, and the radar cross section RCS corresponding to the position in at least one frame after multiple frames are input into a single-frame classifier to obtain the category of at least one frame after multiple frames and the confidence of the category.

[0018] In this possible implementation, for point clouds with a number greater than a preset value, a single-frame classifier can be input, that is, the first target corresponding to the point cloud number greater than the preset value can be recognized by the classifier, so that when the multi-frame classifier is subsequently input, the result of the single-frame classifier is included.

[0019] Optionally, in a possible implementation manner of the first aspect, the target feature in the above step includes at least one of the area, perimeter or length and width of the target point cloud, and the ratio of the number of point clouds in the target point cloud to the area.

[0020] A second aspect of the present application provides a data processing device, which may be a radar or a device integrated with a radar (such as a vehicle, a drone, etc.), and includes:

[0021] A first acquisition unit is used to acquire target point clouds of multiple frames;

[0022] a first processing unit, configured to obtain, based on the target point clouds of the multiple frames, a classification result of a first target for each of the multiple frames, wherein the classification result of the first target includes a category of the first target and a confidence score corresponding to the category, and the first target is associated with the target point cloud;

[0023] a first classification unit, configured to obtain a first output result by using a Bayesian algorithm based on the category of the first target in each frame of the multiple frames and the tracking position of the first target in each frame of the multiple frames, wherein the first output result includes a first category of the first target and a first confidence level corresponding to the first category, and the tracking position of the first target in each frame of the multiple frames is a position obtained by tracking the target point cloud of the multiple frames;

[0024] A second acquisition unit is used to acquire target features based on the target point clouds of multiple frames, where the target features are spatial distribution morphological features of the target point clouds of multiple frames;

[0025] The second classification unit is used to input the first category, the first confidence and the target feature into the multi-frame classifier to obtain a second output result, where the second output result includes a second category of the first target and a second confidence corresponding to the second category.

[0026] Optionally, in a possible implementation manner of the second aspect, the first classification unit is specifically configured to obtain the first output result according to the category of the first target in each frame in the multiple frames and the tracking position of the first target in each frame in the multiple frames in the following manner:

[0027]

[0028] Among them, c represents the category of the first target in each frame, t represents the tracking position of the first target in each frame, list[c,t] represents the correspondence between the category and the tracking position, Z i represents a sequence of categories of multiple first targets, Ω represents the actual category of the first target, r represents the distance to the first target, a represents the angle to the first target, and Z' represents the category of the first target.

[0029] Optionally, in a possible implementation of the second aspect, the data processing apparatus further includes:

[0030] A second processing unit is configured to obtain a target confidence based on the first confidence and the second confidence;

[0031] The first determining unit is configured to determine that the category of the first target in at least one frame after multiple frames is the second category if the target confidence is greater than a locking threshold.

[0032] Optionally, in a possible implementation of the second aspect, the data processing apparatus further includes:

[0033] A third acquisition unit is used to acquire the number of point clouds corresponding to the first target in at least one frame after multiple frames;

[0034] The second determining unit is configured to determine that the category of the first target in at least one frame after multiple frames is the second category if the target confidence is less than the locking threshold and the number of point clouds is less than a preset value.

[0035] Optionally, in a possible implementation of the second aspect, the data processing apparatus further includes:

[0036] The third processing unit is configured to input the position of the first target, the speed of the first target, and the radar cross section (RCS) corresponding to the position in at least one frame after multiple frames into a single-frame classifier to obtain the category and confidence of the category of at least one frame after multiple frames if the target confidence is less than the locking threshold and the number of point clouds is greater than a preset value.

[0037] Optionally, in a possible implementation manner of the second aspect, the target feature includes at least one of the area, perimeter or length and width of the target point cloud, and the ratio of the number of point clouds in the target point cloud to the area.

[0038] A third aspect of the present application provides a data processing device, which may be a radar or a device integrated with a radar (e.g., a vehicle, a drone, etc.), and which executes the method of the first aspect or any possible implementation of the first aspect.

[0039] A fourth aspect of an embodiment of the present application provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a computer program or instruction so that the chip implements the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0040] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a computer, the computer executes the method in the aforementioned first aspect or any possible implementation of the first aspect.

[0041] A sixth aspect of the embodiments of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method in the aforementioned first aspect or any possible implementation of the first aspect.

[0042] A seventh aspect of an embodiment of the present application provides a data processing device, comprising: a processor, the processor being coupled to a memory, the memory being used to store programs or instructions, and when the programs or instructions are executed by the processor, the data processing device implements the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0043] Among them, the technical effects brought about by the second, third, fourth, fifth, sixth, and seventh aspects or any possible implementation methods thereof can be referred to the technical effects brought about by the first aspect or different possible implementation methods of the first aspect, and will not be repeated here.

[0044] As can be seen from the above technical solution, the embodiment of the present application has the following advantages: a first output result is obtained by using a Bayesian algorithm based on the category of the first target in each frame in the multiple frames and the tracking position of the first target in each frame in the multiple frames, and the first category, first confidence level, and target features are input into a multi-frame classifier to obtain a second output result. The problem of too few missed detections in a single frame is effectively solved through Bayesian multi-frame decision fusion. The combination of multiple frames reduces false detections and jumps in a single frame, thereby improving the accuracy of first target recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is an application scenario diagram of the data processing method in the embodiment of the present application;

[0046] Figure 2 A flowchart of a data processing method in an embodiment of the present application;

[0047] Figure 3 This is a structural diagram of a frame in an embodiment of the present application;

[0048] Figure 4 This is a schematic diagram of the classification results of four frames in an embodiment of the present application;

[0049] Figure 5 This is a structural diagram of a data processing device in an embodiment of the present application;

[0050] Figure 6 This is another structural diagram of the data processing device in an embodiment of the present application;

[0051] Figure 7 This is another structural diagram of the data processing device in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The embodiments of the present application provide a data processing method and related equipment, which can improve the accuracy of target recognition.

[0053] The following will describe in detail the implementation principle, specific implementation methods and corresponding beneficial effects of the technical solution of this application in conjunction with the accompanying drawings.

[0054] The data processing method in the embodiments of the present application can be applied to intelligent vehicles, intelligent aircraft, intelligent robots and other equipment that require target recognition, and the specific details are not limited here.

[0055] The following is a schematic description using an example where the data processing device is an intelligent vehicle.

[0056] See also Figure 1 In the embodiment of the present application, the intelligent vehicle system architecture includes:

[0057] Figure 1FIG2 is a schematic diagram of a system architecture of an example intelligent vehicle 100 (or vehicle 100) provided in an embodiment of the present application. Components in intelligent vehicle 100 may include at least one of a perception system 103, a planning system 105, a control system 107, peripheral devices 109, a power supply 110, a computing device 111, and a user interface 112.

[0058] The main function of the perception system 103 is to perceive and identify the external environment and / or the vehicle's own conditions through sensors and other devices.

[0059] The sensor may include at least one of a global positioning system (GPS), an inertial measurement unit (IMU), a radio, a radar, and a camera.

[0060] The GPS can be any sensor used to estimate the geographic location of the smart vehicle 100. To this end, the GPS may include a transceiver that estimates the position of the smart vehicle 100 relative to the Earth based on satellite positioning data. In an example, the computing device 111 can be used to use the GPS in conjunction with the map data 116 to estimate the location of lane boundaries on a road on which the smart vehicle 100 can travel. The GPS can also take other forms.

[0061] The IMU can be used to sense the position and orientation changes of the smart vehicle 100 based on inertial acceleration and any combination thereof. In some examples, the combination of IMUs may include, for example, an accelerometer and a gyroscope. Other combinations of IMUs are also possible.

[0062] The radar can be used to identify targets around the smart vehicle 100, such as pedestrians, cars, etc.

[0063] Radar may also include millimeter wave radar or laser radar (LIDAR). Millimeter wave radar operates in the millimeter wave band. Typically, millimeter waves refer to the 30 to 300 GHz frequency band (wavelength of 1 to 10 mm). The wavelength of millimeter waves is between centimeter waves and light waves, so millimeter waves have the advantages of both microwave guidance and photoelectric guidance. LIDAR is an optical remote sensing technology that can measure the distance to a target or other properties of a target by illuminating the target with light. As an example, LIDAR may include a laser source and / or laser scanner configured to emit laser pulses, and a detector for receiving reflections of the laser pulses. For example, LIDAR may include a laser rangefinder reflected by a rotating mirror, and scan the laser in one or two dimensions around a digitized scene, thereby collecting distance measurements at specified angular intervals. In an example, LIDAR may include components such as a light (e.g., laser) source, a scanner and optical system, a light detector and receiver electronics, as well as a position and navigation system.

[0064] In an example, LIDAR can be configured to image objects using ultraviolet (UV), visible, or infrared light and can be used for a wide range of targets, including non-metallic objects. In one example, a narrow laser beam can be used to map the physical features of an object with high resolution.

[0065] Three-dimensional (3D) imaging can be achieved using both scanning and non-scanning LIDAR systems. “3D gated viewing laser radar” is an example of a non-scanning laser ranging system that uses a pulsed laser and a rapidly gated camera. Imaging LIDAR can also be performed using high-speed detector arrays and modulation-sensitive detector arrays typically built on a single chip using complementary metal oxide semiconductor (CMOS) and hybrid complementary metal oxide semiconductor / charge coupled device (CCD) manufacturing technologies. In these devices, each pixel can be processed locally by demodulating or gating at high speed so that the array can be processed to represent an image from a camera. Using this technology, thousands of pixels can be acquired simultaneously to create a 3D point cloud representing an object or scene detected by the LIDAR.

[0066] A point cloud can include a set of vertices in a 3D coordinate system. These vertices can be defined, for example, by X, Y, and Z coordinates and can represent the outer surface of an object. LIDAR can be configured to create a point cloud by measuring a large number of points on the surface of an object and output the point cloud as a data file. As a result of the 3D scanning process of an object using LIDAR, the point cloud can be used to identify and visualize the object.

[0067] In one example, a point cloud can be rendered directly to visualize an object. In another example, a point cloud can be converted into a polygonal or triangular mesh model through a process that can be referred to as surface reconstruction. Example techniques for converting a point cloud into a 3D surface may include Delaunay triangulation, alpha shapes, and rotosphere. These techniques involve constructing a network of triangles on the existing vertices of the point cloud. Other example techniques may include converting the point cloud into a volumetric distance field and reconstructing the implicit surface thus defined using a marching cubes algorithm.

[0068] In one example, a point cloud may also be points in a plane coordinate system, which may be defined by, for example, X and Y coordinates.

[0069] The camera can be any camera (e.g., a still camera, a video camera, etc.) that can be used to acquire images of the environment in which the smart vehicle 100 is located. To this end, the camera can be configured to detect visible light, or can be configured to detect light from other parts of the spectrum (such as infrared light or ultraviolet light). Other types of cameras are also possible. The camera can be a two-dimensional detector, or can have a three-dimensional spatial range. In some examples, the camera can be, for example, a distance detector that is configured to generate a two-dimensional image indicating the distance from the camera to several points in the environment. To this end, the camera can use one or more distance detection technologies. For example, the camera can be configured to use structured light technology, in which the smart vehicle 100 illuminates an object in the environment using a predetermined light pattern, such as a grid or checkerboard pattern, and uses the camera to detect reflections of the predetermined light pattern from the object. Based on the distortion in the reflected light pattern, the smart vehicle 100 can be configured to detect the distance to a point on the object. The predetermined light pattern can include infrared light or light of other wavelengths.

[0070] The primary function of the planning system 105 is to plan a path for the intelligent vehicle 100 based on the information received from the perception module and to send control instructions to the control system 107. The planning system 105 may also include an obstacle avoidance system, the primary function of which is to identify, evaluate, and avoid or otherwise navigate obstacles in the environment in which the intelligent vehicle 100 is located.

[0071] The main functions of the control system 107 are to control the operation of the intelligent vehicle 100 and its components and to receive control instructions from the planning system 105 .

[0072] The primary function of the peripheral device 109 is to interact with external sensors, other vehicles, and / or users.

[0073] The computing device 111 may include a processor 113 and a memory 114. The computing device 111 may be a controller or a portion of a controller of the intelligent vehicle 100. The memory 114 may include instructions 115 executable by the processor 113 and may also store map data 116. The map data 116 may be a raster map, a point cloud map, or other maps. The map data 116 may also take other forms.

[0074] The components of the smart vehicle 100 can be configured to operate in a manner that interconnects with each other and / or with other components coupled to various systems. For example, the power supply 110 can provide power to all components of the smart vehicle 100. The computing device 111 can be configured to receive data from and control the perception system 103, the planning system 105, the control system 107, and the peripheral devices 109. The computing device 111 can be configured to generate images for display on the user interface 112 and to receive input from the user interface 112.

[0075] Alternatively, the intelligent vehicle 100 may include more, fewer, or different systems or modules, and each system / module may include more, fewer, or different components. Figure 1 The systems / modules and components shown can be combined or divided in any manner. For example, the perception system 103 and the planning system 105 can be independent or integrated into one device. The planning system 105 and the control system 107 can be independent or integrated into one device.

[0076] In the application scenario of smart vehicles, the data processing device in the embodiment of the present application can be the above-mentioned smart vehicle 100, or the above-mentioned perception system 103, or a sensor module such as radar in the perception system 103, and the specific details are not limited here.

[0077] Of course, except Figure 1 The application scenarios of the smart vehicles described, the data processing method provided in this application can also be applied to scenarios where target recognition is required, such as smart transportation equipment, smart home equipment, robots, etc., which are not specifically limited here.

[0078] The following combination Figure 1 The data processing method in the embodiment of the present application is schematically described with reference to the system framework.

[0079] See also Figure 2 , an embodiment of the data processing method provided by the present application includes:

[0080] 201. A data processing device obtains multiple frames of target point clouds.

[0081] The data processing device may obtain multiple frames of target point clouds by having other devices directly send multiple frames of target point clouds to the data processing device, or by having the data processing device directly measure and obtain multiple frames of target point clouds (or point cloud data).

[0082] The target point cloud in the embodiment of the present application may be point cloud data, and the target point cloud may also include multiple point cloud data, which is not specifically limited here.

[0083] Optionally, the target point cloud includes multiple point cloud data, and the point cloud data corresponding to the target point cloud of each frame may be the same (for example, the position of the tracked first target has not changed, that is, the first target has not moved) or different (for example, the position of the tracked first target has changed). The target point cloud corresponds to the first target, and each frame in the multiple frames includes the first target, that is, the first target is determined and appears in the multiple frames.

[0084] In the embodiment of the present application, the number of multiple frames can be 2 or more, and the number of multiple frames is not limited here.

[0085] Optionally, if the number of multiple frames is 4, after acquiring the target point cloud of a single frame, the data processing device may acquire 4 frames close to the acquisition time of the single frame.

[0086] Optionally, the data processing device may have a tracking function, that is, the data processing device may directly measure and obtain target point clouds for multiple frames in the following manner: the data processing device scans the surrounding environment and objects to obtain a sparse point cloud, and the data processing device uses a clustering algorithm (common ones include k-means and DBSCAN) and some tracking algorithms (such as multiple motion models combined with Kalman filtering and particle filters) to frame a group of detection points based on the speed of the sparse point cloud and the different positions in multiple frames. This group of detection points is the target point cloud. When tracking multiple frames, each frame includes a group of detection points, that is, multiple target point clouds, which can be used to describe the motion process of the first target.

[0087] Optionally, when the data processing device does not have a tracking function, after the radar tracks and obtains multiple frames of target point clouds, the target point clouds are sent to the data processing device.

[0088] Optionally, each frame includes the set of detection points, that is, each frame in the multiple frames includes the first target corresponding to the target point cloud.

[0089] For example, the following is a schematic description using the data processing device being a vehicle as an example. In this step, the target point cloud of the multiple frames acquired by the data processing device is a target point cloud of 4 frames. Figure 3 As shown, 300 represents the first frame of the four frames, 301 represents the first target, 302 represents a part of the target point cloud, and 303 represents the vehicle. It can be understood that Figure 3 It is just a schematic diagram for understanding the relationship between the first target and the target point cloud.

[0090] 202. The data processing device obtains a classification result about the first target in each frame of the multiple frames based on the target point cloud of the multiple frames.

[0091] In the embodiment of the present application, a category range can be pre-set. In the application scenario of smart vehicles, the category range can be two categories (for example, cars and people), or four categories (for example, cars, people, electric vehicles, and bicycles), etc., or it can also be other categories, which are not limited here.

[0092] In the embodiment of the present application, the information of the first target in each frame may be the position of the first target in each frame, the speed of the first target in each frame, and the radar cross section (RCS) corresponding to the position of the first target in each frame, etc., which is not limited here.

[0093] The data processing device can input the position of the first object in each frame, the speed of the first object, and the RCS corresponding to the position into a single-frame classifier to obtain a classification result for the first object in each frame. The classification result includes the category of the first object and the confidence level corresponding to the category, where the confidence level indicates the stability of the classification for the frame. The category is within the category range described above. If the category range is the two categories described above (car or person), the category in the classification result is either car or person.

[0094] The position of the first target may be the coordinate position (x, y) of the first target in the XY coordinate system, the speed of the first target may be collected by a radar, and RCS is a physical quantity of the echo intensity generated under radar wave irradiation, which corresponds one-to-one to (x, y).

[0095] The single-frame classifier in the embodiment of the present application is a machine learning classification model, which can be a support vector machine (SVM), a random forest, or a decision tree classifier, the specific details of which are not limited here. The single-frame classifier is trained using a training set, which includes a single-frame target point cloud.

[0096] Exemplarily, the data processing device obtains the classification result of the first target in each of the five frames, that is, obtains five classification results of the first target.

[0097] 203. The data processing device obtains a first output result by using a Bayesian algorithm according to the category of the first target in each frame in the multiple frames and the tracking position of the first target in each frame in the multiple frames.

[0098] After the data processing device obtains the classification result of the first target in each frame of the multiple frames, it can obtain the first output result through the Bayesian algorithm according to the category of the first target in each frame of the multiple frames and the tracking position of the first target in each frame of the multiple frames.

[0099] In the embodiment of the present application, the first target remains unchanged in multiple frames. For the same target to be tracked, the data processing device may have different recognition results for the first target in each frame. See the following example for details.

[0100] The tracking position in the embodiment of the present application can be parameters such as distance and angle, which are not limited here.

[0101] For ease of understanding, the following Figure 4 This example illustrates a situation where the category range is two (person or vehicle), the data processing device is a vehicle, and the number of frames is four. Frame 401 represents the first frame, frame 402 represents the second frame, frame 403 represents the third frame, and frame 404 represents the fourth frame. Frame 405 represents a vehicle (i.e., the data processing device), and frame 406 represents the first object. Assume that the true category of first object 406 is a vehicle, and that the vehicle appears in all four frames. However, due to inaccurate recognition results, the data processing device may recognize the first object differently in each frame. For example, the data processing device may recognize the first object as a vehicle in frame 1, a vehicle in frame 2, a person in frame 3, and a person in frame 4. Specifically, in frame 1, the distance between the data processing device and the first object is r1, and the angle is a1, so the data processing device recognizes the first object as a vehicle. In frame 2, the distance between the data processing device and the first object is r2, and the angle is a2, so the data processing device recognizes the first object as a vehicle. In the third frame, the distance between the data processing device and the first target is r3, and the angle is a3. The data processing device identifies the first target as a person. In the fourth frame, the distance between the data processing device and the first target is r4, and the angle is a4. The data processing device also identifies the first target as a person. Of course, the data processing device can also directly obtain the category and tracking position of each of the four frames from other devices. The category and tracking position of each of the four frames can be shown in Table 1.

[0102] Table 1

[0103] Category of the first target Distance to the first target Angle to the first target Frame 1 car r1 a1 Frame 2 car r2 a2 Frame 3 people r3 a3 Frame 4 people r4 a4

[0104] The categories (people or cars), distances, and angles in Table 1 are just examples. In actual applications, you can add more categories (for example, bicycles, trucks, etc.), change the categories (related to color, gender, etc.), or use other parameters (for example, the size of the first target) as needed. No specific limitations are given here.

[0105] The data processing device can obtain a first output result through a Bayesian algorithm based on the category of the first target in each frame in the multiple frames and the tracking position of the first target in each frame in the multiple frames. The first output result includes the first category of the first target and the first confidence corresponding to the first category.

[0106] Optionally, the data processing device may obtain a first output result according to the category of the first target in each frame in the multiple frames and the tracking position of the first target in each frame in the multiple frames by using Formula 1 and Formula 2:

[0107] Formula 1:

[0108]

[0109] Formula 2:

[0110] Ω=arg max{P(Ω|list[c,t])};

[0111] Among them, c represents the category of the first target in each frame, t represents the tracking position of the first target in each frame, list[c,t] represents the correspondence between the category and the tracking position, Z i represents a sequence of categories of multiple first targets, Ω represents the actual category of the first target, r represents the distance to the first target, a represents the angle to the first target, and Z' represents the category of the first target obtained based on the target point cloud.

[0112] The following is combined with formula 1, table 1, Figure 4 The process of obtaining the first output result is illustrated by taking an example. For example, the examples and angles in Table 1 are shown in Table 2.

[0113] Table 2

[0114] Category of the first target Distance to the first target Angle to the first target Frame 1 car 150 meters -45 degrees Frame 2 car 100 meters -30 degrees Frame 3 people 75 meters 0 degrees Frame 4 people 50 meters 30 degrees

[0115] The examples and angle values ​​in Table 2 are just examples and are not intended to be limiting.

[0116] The terms in formula 1 are shown in Table 2 and Figure 4 The meanings in the example scenario are as follows:

[0117] P(Ω): Prior probability, that is, the probability of the true category appearing. In practical applications, it can be obtained based on training set statistics or by using equal probability. Here, k is the category type. In Table 1, the category type is 2, so P(Ω=person)=0.5, P(Ω=car)=0.5.

[0118] P(Z i|Ω): can be used to simulate the performance of a single-frame classifier. In the four frames, the single-frame categories of the first target are: car, car, person, person, that is, Z i = car car person person. Ω represents the true category of the first target. Assume that the true category of the first target is car, the probability of identifying it as a person is 0.1, and the true category is car, the probability of identifying it as a car is 0.9, that is, P(Z'=person|Ω=car)=0.1, P(Z'=car|Ω=car)=0.9. Assume that the true category of the first target is person, the probability of identifying it as a car is 0.05, and the true category is person, the probability of identifying it as a person is 0.95, that is, P(Z'=car|Ω=person)=0.05, P(Z'=person|Ω=person)=0.95.

[0119] P(Z i |r,a): This can be used to simulate detection effects, regardless of location, to determine the likelihood of identifying the first target. For example, if the first target is detected to be 100 meters away from the data processing device (i.e., r = 100 meters) and directly in front of the data processing device (i.e., a = 0), the probability that the first target is a person is very low (which can be statistically determined).

[0120] P(r, a|Ω): This represents the probability of the true class appearing at different locations. This can be statistically determined. Under different road conditions, the probability of the first target appearing at different locations varies. For example, on an elevated expressway, the probability of a pedestrian appearing in front is low, while the probability of a pedestrian appearing on the left is high. The probability of a car cutting in at close range is low. In the absence of real-world data, we can assume that the probabilities are equal, and P(r, a|Ω) = 1.

[0121] For example, the scene is on a city expressway, and the probability of pedestrians appearing is low. The statistical probability is as follows.

[0122] P(r=150, a=-45|Ω=people)=0.1, P(r=100, a=-30|Ω=people)=0.1;

[0123] P(r=75, a=0|Ω=people)=0.1, P(r=50, a=30|Ω=people)=0.1;

[0124] P(r=150, a=-45|Ω=car)=0.8, P(r=100, a=-30|Ω=car)=0.8;

[0125] P(r=75, a=0|Ω=car)=0.8, P(r=50, a=30|Ω=car)=0.8;

[0126] list[c,t]: used to represent multiple correspondences between tracking positions and categories. Among them, t can be distance and angle, that is, list[c,t] can be list[c,(r,a)], continuing the above example, Figure 4 The corresponding relationships in the four frames shown are: [vehicle, (150, -45)], [vehicle, (100, -30)], [person, (75, 0)], [person, (50, 30)].

[0127] Then substitute into Formula 1 and calculate as follows:

[0128] P(Ω = person | r = (150, 100, 75, 50), a = (-45, -30, 0, 30), Z i = vehicle vehicle person person) = [P(Z i = vehicle vehicle person person | Ω = person) * P(Ω = person) * P(Z i = vehicle vehicle person person | r = (150, 100, 75, 50), a = (-45, -30, 0, 30)) *

[0129] P(r = (150, 100, 75, 50), a = (-45, -30, 0, 30) | Ω = person)] / P(Z i = vehicle vehicle person person) = P(Z' = vehicle | Ω = person) * P(Z' = vehicle | Ω = person) * P(Z' = person | Ω = person) * P(Z' = person | Ω = person) *

[0130] P(Ω = person) * P(Z' = vehicle | r = 150, a = -45) * P(Z' = vehicle | r = 100, a = -30) * P(Z' = person | r = 75, a = 0) *

[0131] P(Z' = person | r = 50, a = 30) * P(r = 150, a = -45 | Ω = person) * P(r = 100, a = -30 | Ω = person) *

[0132] P(r = 100, a = 0 | Ω = person) * P(r = 50, a = 30 | Ω = person) / P(Z i = vehicle vehicle person person) = (0.05 * 0.05 * 0.95 * 0.95 * 0.5 * 0.95 * 0.95 * 0.05 * 0.05 * 0.1 * 0.1 * 0.1 * 0.1) / P(Z i = vehicle vehicle person person) = 2.55 * 10 -10 / P(Z i = vehicle vehicle person person).

[0133] Similarly,

[0134] P(Ω = vehicle | r = (150, 100, 75, 50), a = (-45, -30, 0, 30), Z i = vehicle vehicle person person) = (0.1 * 0.1 * 0.9 * 0.9 * 0.5 * 0.95 * 0.95 * 0.05 * 0.05 * 0.8 * 0.8 * 0.8 * 0.8) / P(Zi = P(Ω = vehicle|r=(150, 100, 75, 50), a=(-45, -30, 0, 30), Z -6 / P(Z i = P(Ω = vehicle).

[0135] Among them, P(Z i = P(Ω = vehicle)=2.55*10 -10 +3.74*10 -6 .

[0136] As can be seen from the above, the probability of P(Ω = vehicle|r=(150, 100, 75, 50), a=(-45, -30, 0, 30), Z i = P(Ω = vehicle) is greater, that is, the first category of the first target is vehicle, and the first confidence corresponding to the first category is

[0137] If there are floating-point precision problems in the calculated results, logarithmic or exponential operations can be used for recovery. Exemplarily, since the probability that the first target is a person is 2.55*10 -10 , taking the logarithmic operation gives log 10 (2.55*10 -10 ) = -10 + log 10 2.55. Similarly, 3.74*10 -6 after logarithmic operation is -6 + log 10 3.74, and then perform operations such as storage. Of course, logarithmic operations can also be used when calculating probabilities, and specific details are not limited here.

[0138] In the embodiments of this application, Formula One has various transformation forms. For example, Formula Three can be a transformation form of Formula One, and specific details are not limited here.

[0139] Formula Three:

[0140]

[0141] 204. The data processing device obtains target features based on multi-frame target point clouds.

[0142] The data processing device can obtain target features based on multi-frame target point clouds,

[0143] The data processing device can concentrate multi-frame target point clouds into a certain frame (for example, put the target point clouds of the first 5 frames in the 5th frame), obtaining a target point cloud set. And obtain the target features of this target point cloud set. The target features can also be called spatial distribution morphological features. The spatial distribution morphological features can be at least one of the convex hull area, perimeter, point density, the area of the circumscribed rectangle of this target point cloud set, density, aspect ratio, etc., and specific details are not limited here.

[0144] Optionally, the convex hull points of the target point cloud of multiple frames may be obtained by using a Graham scanning method or the like. The bounding rectangle of the target point cloud of multiple frames may be obtained by using a rotating calipers algorithm or the like.

[0145] Optionally, the target features may further include RCS of multiple frames, parameters of Weibull distribution of velocity (shape parameter, scale parameter), mean of RCS, variance of RCS, etc., which are not specifically limited here.

[0146] 205. The data processing device inputs the first category, the first confidence level, and the target feature into a multi-frame classifier to obtain a second output result.

[0147] The data processing device may use a Bayesian algorithm to obtain a first category and a first confidence score based on the category of the first target in each frame of the multiple frames and the tracking position of the first target in each frame of the multiple frames, and after obtaining the target features, the data processing device may input the first category, the first confidence score, and the target features into a multi-frame classifier to obtain a second output result. The second output result includes a second category of the first target and a second confidence score corresponding to the second category.

[0148] Through this step, the target features of multiple frames and the Bayesian decision results can be effectively combined to avoid the problem of low target recognition accuracy caused by poor performance of single-frame classification. By providing the target features of multiple frames and the first output result obtained by the Bayesian algorithm, the second category of the first target identified by the multi-frame classifier is more accurate than the first category, thereby improving the accuracy of the first target recognition.

[0149] For example, assuming that the true category of the first target is human, and the first category of the first target is determined to be car through step 203, the target features include the area of ​​the convex hull corresponding to the target point cloud set or the area of ​​the circumscribed rectangle of the target point cloud set, and the area of ​​the convex hull or the area of ​​the circumscribed rectangle is much smaller than the reference area of ​​the car (the reference area of ​​the car can be obtained through sampling statistics), then the second category output by the multi-frame classifier is human, that is, the category of the first target is finally determined to be human.

[0150] The multi-frame classifier in the embodiment of the present application is a machine learning classification model, which can be a support vector machine (SVM) or a classifier such as a random forest, which is not limited here. Among them, the multi-frame classifier is obtained by performing Bayesian fusion on the output of the trained single-frame classifier and training it together with the target features of multiple frames as a training set. 206. The data processing device obtains the target confidence based on the first confidence and the second confidence. This step is optional.

[0151] After obtaining the first confidence level and the second confidence level, the data processing device may obtain the target confidence level based on the first confidence level and the second confidence level.

[0152] That is, the target confidence is obtained based on the confidence of single-frame classification and the confidence of multi-frame classification, which can represent both the stability of the current frame and the confidence of the multi-frame results.

[0153] Optionally, the target confidence is the product of the first confidence and the second confidence.

[0154] 207. If the target confidence is greater than the lock threshold, the data processing device determines that the category of the first target in at least one frame after multiple frames is the second category. This step is optional.

[0155] The locking threshold in the embodiment of the present application can be set according to actual needs and is not limited here. The locking threshold can be used to measure whether the second category is accurate or stable, as described below:

[0156] After the data processing device obtains the target confidence, it can compare the target confidence with the locking threshold. According to the result, it can be divided into the following two situations:

[0157] The first type: if the target confidence is greater than or equal to (or greater than) the locking threshold, the data processing device determines that the category of the first target in at least one frame after multiple frames (i.e., subsequent frames, hereinafter referred to as subsequent frames) is the second category (i.e., the result of the previous frame), that is, the accuracy or stability of the second category is high. By determining that the category of the first target in the subsequent frame is the second category, the computing power overhead of the triggering classifier can be reduced and the detection efficiency can be improved.

[0158] The second type: If the target confidence is less than (or less than or equal to) the lock threshold, the data processing device can first determine whether the subsequent frame can be classified. If the subsequent frame can be classified, the relevant information of the subsequent frame is input into the single-frame classifier to obtain the category of the subsequent frame and the confidence of the category. If the subsequent frame cannot be classified, the category and confidence of the previous frame are used (i.e., the category of the first target in the subsequent frame is determined to be the category of the first target in the previous frame).

[0159] In the embodiment of the present application, there are multiple methods for the data processing device to determine whether a subsequent frame can be classified, which are not specifically limited here. The following description only uses the example of the data processing device determining whether a subsequent frame can be classified based on the number of point clouds corresponding to the first target pair in the subsequent frame.

[0160] The data processing device can obtain the number of point clouds corresponding to the first target in the subsequent frame, and determine whether the subsequent frame can be classified based on the relationship between the number of point clouds and a preset value.

[0161] The preset values ​​in the embodiments of the present application are set according to actual needs and are not specifically limited here.

[0162] Optionally, if the number of point clouds corresponding to the first target in the subsequent frame is greater than or equal to (or greater than) a preset value, it means that the number of point clouds corresponding to the first target in the subsequent frame is sufficient, and the position of the first target in the subsequent frame, the speed of the first target and the RCS corresponding to the position can be input into the single-frame classifier to obtain the category of the subsequent frame and the confidence of the category.

[0163] Optionally, if the number of point clouds corresponding to the first target in a subsequent frame is less than (or less than or equal to) a preset value, it indicates that the number of point clouds corresponding to the first target in the subsequent frame is too small to be classified, and the category of the first target in the previous frame is retained. By retaining the category of the previous frame, unstable point cloud data can be filtered out, and the classification result can be directly given, which can improve classification accuracy.

[0164] Optionally, the data processing device may obtain track information of the first target upon starting to track the first target. The track information may include data such as the identifier of the first target or the motion state of the first target, which is not specifically limited herein.

[0165] Furthermore, in order to facilitate the category determination of subsequent frames, the track information may also include indication information. In the case of the initial frame, the indication information may indicate that the category of the first target is not locked, because the initial frame does not have a category yet. If the category of the first target is not locked, identification can continue through a single-frame classifier or a multi-frame classifier until the confidence corresponding to the identified category is greater than the locking threshold. As the number of frames increases, if the target confidence corresponding to the identified second category is greater than the locking threshold, that is, the second category tends to be accurate, the indication information may indicate that the category of the first target is locked, that is, the category of the first target in subsequent frames also adopts the category when locked. Specifically, the indication information is 1, indicating that the category of the first target is locked; the indication information is 0, indicating that the category of the first target is not locked.

[0166] In the embodiment of the present application, the problem of low single-frame recognition accuracy can be compensated by Bayesian multi-frame decision fusion and inputting target features into a multi-frame classifier. Using multi-frame data for classification improves the recognition accuracy of the first target. By locking the threshold and the preset value, on the one hand, the computing power and time overhead can be effectively reduced, and the detection efficiency can be improved. On the other hand, by using the multi-frame results or the locked results for the missed detection frame results, the problem of missed detection caused by the small number of point clouds corresponding to the first target can be effectively solved.

[0167] Corresponding to the method provided in the above method embodiment, the present application embodiment also provides a corresponding device, including a module for executing the corresponding module of the above embodiment. The module can be software, hardware, or a combination of software and hardware.

[0168] The data processing device may be a vehicle with target recognition capabilities, or other components with target recognition capabilities. The data processing device includes, but is not limited to, an onboard terminal, an onboard controller, an onboard module, an onboard component, an onboard chip, an onboard unit, an onboard radar, or an onboard camera, and other sensors. The vehicle may implement the method provided in this application through the onboard terminal, onboard controller, onboard module, onboard component, onboard chip, onboard unit, onboard radar, or camera.

[0169] The data processing device may also be a smart terminal with target recognition capabilities other than a vehicle, or may be installed in a smart terminal with target recognition capabilities other than a vehicle, or installed in a component of such a smart terminal. The smart terminal may be other terminal devices such as smart transportation equipment, smart home appliances, robots, etc. The data processing device includes but is not limited to the smart terminal or the controller, chip, radar, camera, and other sensors within the smart terminal, as well as other components.

[0170] The data processing device can be a general-purpose device or a dedicated device. In a specific implementation, the device can also be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, an embedded device, or other device with processing capabilities. The embodiments of the present application do not limit the type of the data processing device.

[0171] The data processing device may also be a chip or processor with processing capabilities, and the data processing device may include multiple processors. The processor may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. The chip or processor with processing capabilities may be disposed in the sensor, or may not be disposed in the sensor but be disposed at the receiving end of the sensor output signal.

[0172] See also Figure 5 In one embodiment of the data processing device in the embodiments of the present application, the data processing device may be a radar or a device other than a radar in a vehicle, and the data processing device includes:

[0173] A first acquisition unit 501 is used to acquire a target point cloud of multiple frames;

[0174] A first processing unit 502 is configured to obtain a classification result of a first target for each of the multiple frames based on the target point clouds of the multiple frames, wherein the classification result of the first target includes a category of the first target and a confidence score corresponding to the category, and the first target is associated with the target point cloud;

[0175] A first classification unit 503 is configured to obtain a first output result using a Bayesian algorithm based on the category of the first target in each frame of the multiple frames and the tracking position of the first target in each frame of the multiple frames, wherein the first output result includes a first category of the first target and a first confidence level corresponding to the first category, and the tracking position of the first target in each frame of the multiple frames is a position obtained by tracking the target point cloud of the multiple frames;

[0176] A second acquisition unit 504 is configured to acquire target features based on the target point clouds of multiple frames, where the target features are spatial distribution morphological features of the target point clouds of multiple frames;

[0177] The second classification unit 505 is used to input the first category, the first confidence and the target feature into the multi-frame classifier to obtain a second output result, where the second output result includes a second category of the first target and a second confidence corresponding to the second category.

[0178] In this embodiment, the operations performed by each unit in the data processing device are the same as those described above. Figures 2 to 4 The operations performed by the data processing device in the illustrated embodiment are similar and will not be described in detail here.

[0179] In this embodiment, the second classification unit 505 uses data of multiple frames (target features) for classification, so that the recognition accuracy of the first target is improved.

[0180] See also Figure 6 Another embodiment of the data processing device in the embodiments of the present application, the data processing device may be a radar, or a device other than a radar in a vehicle, and the data processing device includes:

[0181] A first acquisition unit 601 is used to acquire target point clouds of multiple frames;

[0182] A first processing unit 602 is configured to obtain a classification result of a first target for each of the multiple frames based on the target point clouds of the multiple frames, wherein the classification result of the first target includes a category of the first target and a confidence score corresponding to the category, and the first target is associated with the target point cloud;

[0183] A first classification unit 603 is configured to obtain a first output result using a Bayesian algorithm based on the category of the first target in each frame of the multiple frames and the tracking position of the first target in each frame of the multiple frames, wherein the first output result includes a first category of the first target and a first confidence level corresponding to the first category, and the tracking position of the first target in each frame of the multiple frames is a position obtained by tracking the target point cloud of the multiple frames;

[0184] A second acquisition unit 604 is configured to acquire target features based on the target point clouds of multiple frames, where the target features are spatial distribution morphological features of the target point clouds of multiple frames;

[0185] The second classification unit 605 is used to input the first category, the first confidence and the target feature into the multi-frame classifier to obtain a second output result, where the second output result includes the second category of the first target and the second confidence corresponding to the second category.

[0186] The data processing device in this embodiment further includes:

[0187] A second processing unit 606 is configured to obtain a target confidence level based on the first confidence level and the second confidence level;

[0188] The first determining unit 607 is configured to determine that the category of the first target in at least one frame after multiple frames is the second category if the target confidence is greater than the locking threshold.

[0189] The third acquisition unit 608 is configured to acquire the number of point clouds corresponding to the first target in at least one frame after the multiple frames.

[0190] The second determining unit 609 is configured to determine that the category of the first target in at least one frame after multiple frames is the second category if the target confidence is less than the locking threshold and the number of point clouds is less than a preset value.

[0191] The third processing unit 610 is configured to input the position of the first target, the speed of the first target, and the radar cross section (RCS) corresponding to the position in at least one frame after multiple frames into a single-frame classifier to obtain a category and category confidence of at least one frame after multiple frames if the target confidence is less than a locking threshold and the number of point clouds is greater than a preset value.

[0192] The first classification unit 603 is specifically configured to obtain a first output result according to the category of the first target in each frame in the multiple frames and the tracking position of the first target in each frame in the multiple frames in the following manner:

[0193]

[0194] Ω=arg max{P(Ω|list[c, t])};

[0195] Among them, c represents the category of the first target in each frame, t represents the tracking position of the first target in each frame, list[c,t] represents the correspondence between the category and the tracking position, Z i represents a sequence of categories of multiple first targets, Ω represents the actual category of the first target, r represents the distance to the first target, a represents the angle to the first target, and Z' represents the category of the first target.

[0196] Optionally, the target feature includes at least one of the area, perimeter or length and width of the target point cloud, and the ratio of the number of point clouds in the target point cloud to the area.

[0197] In this embodiment, the operations performed by each unit in the data processing device are the same as those described above. Figures 2 to 4 The operations performed by the data processing device in the illustrated embodiment are similar and will not be described in detail here.

[0198] In this embodiment of the present application, the second classification unit 605 uses multi-frame data for classification, thereby improving the recognition accuracy of the first target. By locking the threshold and presetting the value, on the one hand, it can effectively reduce computing power and time overhead, thereby improving detection efficiency. On the other hand, the first determination unit 607 and the second determination unit 609 can effectively solve the problem of missed detection caused by too few point clouds corresponding to the first target by using the multi-frame results or the locked results for the missed detection frame results.

[0199] See also Figure 7 , is a possible schematic diagram of the data processing device 700 involved in the above-mentioned embodiments provided in an embodiment of the present application. The data processing device 700 may specifically be the data processing device in the aforementioned embodiment. The data processing device 700 may include but is not limited to a processor 701, a communication port 702, a memory 703, and a bus 704. In the embodiment of the present application, the processor 701 is used to control and process the actions of the data processing device 700.

[0200] In addition, the processor 701 can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and so on. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0201] It should be noted that Figure 7 The data processing device shown can be used to implement Figures 2 to 4 The functions of the steps executed by the data processing device in the corresponding method embodiment and the corresponding technical effects of the data processing device are achieved. Figure 7 The specific implementation of the data processing device shown can refer to Figures 2 to 4 The descriptions in the corresponding method embodiments will not be repeated here one by one.

[0202] The present application also provides a computer-readable storage medium storing one or more computer-executable instructions. When the computer executes the instructions on the computer, the computer executes the method described in the possible implementation of the data processing device in the above embodiment, wherein the data processing device can be specifically the above Figures 2 to 4 The data processing device in the corresponding method embodiment.

[0203] The present application also provides a computer-readable storage medium storing one or more computer-executable instructions. When the computer-executable instructions are executed by a processor, the processor executes the method described in the possible implementation of the data processing device in the above embodiment, wherein the data processing device can be specifically the above-mentioned Figures 2 to 4 The data processing device in the corresponding method embodiment.

[0204] The embodiment of the present application further provides a computer program product storing one or more computers. When the computer program product is executed by the processor, the processor executes the method of the possible implementation of the above-mentioned data processing device, wherein the data processing device can be specifically the above-mentioned Figures 2 to 4 The data processing device in the corresponding method embodiment.

[0205] The embodiment of the present application also provides a chip system, which includes a processor for supporting a data processing device to implement the functions involved in the possible implementation of the above-mentioned data processing device. In one possible design, the chip system may also include a memory for storing the necessary program instructions and data for the data processing device. The chip system may be composed of a chip, or may include a chip and other discrete devices, wherein the functions implemented by the chip may specifically be the aforementioned Figures 2 to 4 The functions implemented by the data processing device in the corresponding method embodiment.

[0206] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0207] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0208] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0209] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0210] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

Claims

1. A data processing method, characterized in that: include: Get target point clouds of multiple frames; Obtaining a classification result of a first target for each frame in the multiple frames based on the target point clouds of the multiple frames, wherein the classification result of the first target includes a category of the first target and a confidence score corresponding to the category, and the first target is associated with the target point cloud; Obtaining a first output result by a Bayesian algorithm based on the category of the first target in each frame of the multiple frames and the tracking position of the first target in each frame of the multiple frames, wherein the first output result includes a first category of the first target and a first confidence level corresponding to the first category, and the tracking position of the first target in each frame of the multiple frames is a position obtained by tracking the target point cloud of the multiple frames; Acquire target features based on the target point clouds of the multiple frames, where the target features are spatial distribution morphological features of the target point clouds of the multiple frames; The first category, the first confidence level, and the target feature are input into a multi-frame classifier to obtain a second output result, where the second output result includes a second category of the first target and a second confidence level corresponding to the second category.

2. The method according to claim 1, characterized in that Obtaining a first output result by using a Bayesian algorithm according to the category of the first target in each frame of the multiple frames and the tracking position of the first target in each frame of the multiple frames, including: The first output result is obtained according to the category of the first target in each frame of the multiple frames and the tracking position of the first target in each frame of the multiple frames in the following manner: Ω=argmax{P(Ω|list[c,t])}; Where c represents the category of the first target in each frame, t represents the tracking position of the first target in each frame, list[c,t] represents the corresponding relationship between the category and the tracking position, Z i represents a sequence of categories of multiple first targets corresponding to the multiple frames, Ω represents the actual category of the first target, r represents the distance to the first target, a represents the angle to the first target, Z i ' represents the category of the first target obtained by any frame in the multiple frames based on the target point cloud.

3. The method according to claim 1 or 2, characterized in that The method further comprises: Obtaining a target confidence level based on the first confidence level and the second confidence level; If the target confidence is greater than a locking threshold, it is determined that the category of the first target in at least one frame after the multiple frames is the second category.

4. The method according to claim 3, characterized in that The method further includes: obtaining the number of point clouds corresponding to the first target in at least one frame after the multiple frames; If the target confidence is less than the locking threshold and the number of point clouds is less than a preset value, it is determined that the category of the first target in at least one frame after the multiple frames is the second category.

5. The method according to claim 4, characterized in that The method further comprises: If the target confidence is less than the locking threshold and the number of point clouds is greater than the preset value, the position of the first target, the speed of the first target, and the radar cross section (RCS) corresponding to the position in at least one frame after the multiple frames are input into a single-frame classifier to obtain a category of the at least one frame after the multiple frames and the confidence of the category.

6. The method according to claim 1 or 2, characterized in that The target feature includes at least one of the area, perimeter or length and width of the target point cloud, and the ratio of the number of point clouds in the target point cloud to the area.

7. A data processing device, characterized in that: include: A first acquisition unit is used to acquire target point clouds of multiple frames; a first processing unit, configured to obtain, based on the target point clouds of the multiple frames, a classification result of a first target for each frame of the multiple frames, wherein the classification result of the first target includes a category of the first target and a confidence score corresponding to the category, and the first target is associated with the target point cloud; a first classification unit, configured to obtain a first output result by using a Bayesian algorithm based on the category of the first target in each frame of the multiple frames and the tracking position of the first target in each frame of the multiple frames, wherein the first output result includes a first category of the first target and a first confidence level corresponding to the first category, and the tracking position of the first target in each frame of the multiple frames is a position obtained by tracking the target point cloud of the multiple frames; A second acquisition unit is configured to acquire target features based on the target point clouds of the multiple frames, where the target features are spatial distribution morphological features of the target point clouds of the multiple frames; The second classification unit is used to input the first category, the first confidence and the target feature into a multi-frame classifier to obtain a second output result, where the second output result includes a second category of the first target and a second confidence corresponding to the second category.

8. The data processing device according to claim 7, characterized in that The first classification unit is specifically configured to obtain the first output result according to the category of the first target in each frame in the multiple frames and the tracking position of the first target in each frame in the multiple frames in the following manner: Ω=argmax{P(Ω|list[c,t])}; Where c represents the category of the first target in each frame, t represents the tracking position of the first target in each frame, list[c,t] represents the corresponding relationship between the category and the tracking position, Z i represents a sequence of categories of the first targets corresponding to the multiple frames, Ω represents the actual category of the first target, r represents the distance to the first target, a represents the angle to the first target, Z i ' represents the category of the first target obtained by any frame in the multiple frames based on the target point cloud.

9. The data processing device according to claim 7 or 8, characterized in that The data processing device further includes: a second processing unit, configured to obtain a target confidence level based on the first confidence level and the second confidence level; The first determining unit is configured to determine, if the target confidence is greater than a locking threshold, that the category of the first target in at least one frame after the plurality of frames is the second category.

10. The data processing device according to claim 9, characterized in that The data processing device further includes: a third acquiring unit, configured to acquire the number of point clouds corresponding to the first target in at least one frame after the multiple frames; The second determining unit is configured to determine that the category of the first target in at least one frame after the multiple frames is the second category if the target confidence is less than the locking threshold and the number of point clouds is less than a preset value.

11. The data processing device according to claim 10, characterized in that The data processing device further includes: a third processing unit, configured to input the position of the first target, the speed of the first target, and the radar cross section (RCS) corresponding to the position in at least one frame after the multiple frames into a single-frame classifier to obtain a category of the at least one frame after the multiple frames and a confidence level of the category, if the target confidence level is less than the locking threshold and the number of point clouds is greater than the preset value.

12. The data processing device according to claim 7 or 8, characterized in that: The target feature includes at least one of the area, perimeter or length and width of the target point cloud, and the ratio of the number of point clouds in the target point cloud to the area.

13. A data processing device, characterized in that: include: A processor is coupled to a memory, wherein the memory is used to store programs or instructions, and when the programs or instructions are executed by the processor, the data processing device executes the method according to any one of claims 1 to 6.

14. A computer-readable medium, characterized in that A computer program or instruction is stored thereon, and when the computer program or instruction is run on a computer, the computer is caused to perform the method according to any one of claims 1 to 6.

15. A chip, characterized in that: The chip includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is configured to run a computer program or instruction so that the method according to any one of claims 1 to 6 is executed.

Images