Data processing method and related product

Abstract

A data processing method and related products are applied to the technical field of detection. A combined angle measurement architecture is adopted, a plurality of detection devices (1,..., N) detect an object space by using detection signals to obtain reported data, and the reported data comprises speed information, distance information and channel data. The data processing device (10) determines at least one association point group based on the pose information of the at least one detection device (1,... N), each association point group comprises a plurality of associated detection points in the plurality of items of reported data, the detection points reflect associated channel data, and the channel data of the detection points of each association point group meets a channel consistency condition. The data processing device (10) performs angle measurement using channel data of a plurality of detection points in the associated point group, i.e., performs common angle measurement in conjunction with associated channels in the plurality of detection devices (1,..., N). By adopting the method, the angle measurement capability of a detection system can be greatly improved, the detection precision can be remarkably improved, and the confidence of a detection result can be remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of detection, and in particular, to a data processing method and related products. BACKGROUND

[0002] Radar (Radio Detection and Ranging) can detect space, find targets in space and measure the distance, speed and angle of the targets by using radio waves. Radar transmits electromagnetic wave signals and receives echo signals, and measures the distance of the target according to the time difference (i.e. time of flight) between the echo signals and the detection signals. Since the measured target has relative movement with the radar, the frequency of the echo signal is different from the frequency of the transmitted electromagnetic wave signal, i.e. Doppler effect, and radar can measure the speed of the target by using the Doppler effect. In terms of angle measurement, radar uses channel data of multiple channels to measure the angle, and the accuracy of angle measurement is affected by the number of channels and the antenna density of the radar.

[0003] At present, radar can achieve high accuracy in measuring distance and speed. However, due to the limited number of antennas in the antenna array of the radar and the small number of channels, the angle measurement accuracy cannot meet the user's demand for radar performance, which makes it difficult for the radar to locate the angle of the target and the confidence of the detection result is not high. Some solutions increase the number of antennas in the radar and increase the antenna density to increase the number of channels, which has high cost and increases the amount of data to be processed, and has poor realizability. SUMMARY

[0004] The present application provides a data processing method and related products, which can jointly measure the angle of the associated channels in multiple detection devices, greatly improve the angle measurement capability of the multiple detection device system, significantly improve the detection accuracy, and significantly improve the confidence of the detection result. Further, the channel data of the associated channels satisfies the channel consistency condition, which can accurately select the associated channels, thereby further improving the accuracy of joint angle measurement.

[0005] In a first aspect, the present application provides a data processing method, which can be executed by a data processing device. The data processing device has computing capability and can be an independent device or a module (such as a software and / or hardware module) in an independent device. For ease of description, the following description takes the execution subject as the data processing device as an example.

[0006] The data processing method comprises: a data processing device acquires a plurality of pieces of reported data from a plurality of detection devices, and determines at least one associated point group based on pose information of at least one detection device. Each associated point group comprises a plurality of associated detection points, the plurality of detection points in each associated point group are derived from at least two groups of reported data, and the channel data corresponding to the plurality of detection points in each associated point group satisfies a channel consistency condition. The at least one detection device belongs to the plurality of detection devices.

[0007] Further, the data processing method further comprises: the data processing device determines at least one angle information based on the channel data corresponding to the plurality of detection points in the first associated point group, and the at least one angle information is used to indicate an angle of a target in an object space.

[0008] The reported data is data obtained by the detection device detecting the object space by using a detection signal, and comprises distance information of a detection point, speed information of the detection point, and channel data corresponding to the detection point. For example, the reported data comprises range-velocity data (or RV data). The channel is an independent path for signal transmission, and the channel data is data obtained by processing a signal received by the channel, and comprises one or more pieces of information such as amplitude, phase, or noise level. In some cases, the data processing device processes the channel data to obtain the angle of the target. It should be understood that the distance information of the detection point, the speed information of the detection point, and the channel data corresponding to the detection point are included in each piece of reported data in the plurality of pieces of reported data. Of course, the detection threshold of the detection point, the number of detection points, and the like in each group of reported data can be set to be the same or different.

[0009] Currently, multiple detection devices are usually arranged in a terminal to detect an object space. When multiple detection devices detect the object space, different detection devices obtain different reported data for the same object space due to different poses of the multiple detection devices. However, the reported data can have certain correlations. For example, for two detection devices arranged at the front left and front of a vehicle, the two detection devices can measure different distances, speeds, and different channel data for the same target, but the information has correlations. In some cases, a data processing device can use the poses of the detection devices to compensate for the deviation between the reported data caused by the different poses of the detection devices, and find the correlations between the different reported data. For example, because the channel data can reflect the results of angle measurement, the consistency between two sets of channel data obtained by angle measurement for the same target is high. For another example, the data processing device converts the reported data of one detection device to the coordinate system of another set of reported data, so that the distances and speeds measured by the two sets of reported data for the same target can be aligned relative to the same coordinate system. Therefore, from the perspective of data, the data processing device can process the reported data to detect some associated detection points in the reported data of different detection devices. The associated detection points can reflect the detection points obtained by different detection devices for the same target (or a suspected same target). The same target does not necessarily refer to the same object in the object space, but refers to a target perceived by the detection device that has correlation characteristics (for example, the same distance, the same speed, or angle correspondence).

[0010] In some cases, the reported data includes channel data corresponding to a detection point. The channel data corresponding to the detection point can be used to detect one or more angle information corresponding to the detection point, so as to distinguish multiple targets with the same distance and speed. On this basis, because the associated multiple detection points can reflect the detection points obtained by detecting the same target, the channel data of the associated detection points can be combined to perform joint angle measurement, so as to obtain angle information (such as angle value or angle value range) of the target, which can significantly improve the angle resolution capability of the multiple detection device system.

[0011] In summary, in the above scheme, the data processing apparatus can acquire the reported data of multiple detection apparatuses, and determine associated detection points in multiple reported data, which can reflect the results obtained by different detection apparatuses for detecting the same target (or suspected same target), and the channel data corresponding to multiple detection points in an associated point group satisfies the channel consistency condition. The data processing apparatus uses the channel data corresponding to the associated detection points to jointly perform angle measurement, and obtains the angle information of the target in the object space. The obtained angle information can be used as the angle information of the point or target in the subsequent detection result. Since the number of channels of a single detection apparatus is limited, the angle measurement capability of a single detection apparatus is limited. The present application can jointly perform angle measurement on the associated channels in multiple detection apparatuses, greatly improving the angle measurement capability of the multi-detection-apparatus system, and significantly improving the detection accuracy. Moreover, the joint angle measurement result obtained by using the reported data of multiple detection apparatuses is equivalent to perceiving the target in the object space from multiple positions and angles, which can significantly improve the confidence of the detection result.

[0012] Further, the channel data of the detection points in the associated point group satisfies the consistency condition, that is, the channel consistency of the channel data of multiple detection points is good. In this case, it is indicated that the multiple detection points have a high possibility of detecting the same target (or suspected same target). Therefore, using these detection points with high channel consistency to jointly perform angle measurement can improve the accuracy of the joint angle measurement result.

[0013] In a possible implementation of the first aspect, the multiple reported data includes first reported data and second reported data, and the first associated point group includes a first detection point derived from the first reported data and a second detection point derived from the second reported data. The channel data of the detection points in the first associated point group satisfies the consistency condition, including that the channel data of the first detection point and the channel data of the second detection point satisfy the channel consistency condition. Illustratively, the channel data of the first detection point and the channel data of the second detection point satisfy the channel consistency condition, including that a first correlation angle is greater than or equal to a first threshold value, and the first correlation angle is related to the channel data of the first detection point and the channel data of the second detection point. Illustratively, the first threshold value is 0.9. For example, the first threshold value is 0.8.

[0014] In the above scheme, when evaluating the channel consistency of two detection points, the correlation angle can be calculated. Illustratively, a calculation method of the correlation angle is as follows:

[0015]

[0016] wherein chanData0 is the channel data of the first detection point, chanData1 is the channel data of the second detection point, corrAngle is the correlation angle, n is used to represent the dimension, that is, the dimension of the channel, and H is the conjugate transpose.

[0017] Optionally, the number of detection devices has multiple possible designs, but at least two detection devices are included in the system. For ease of description, the first detection device and the second detection device are taken as examples for description below, and in the case of more detection devices, the other detection devices can be referred to the first detection device and the second detection device.

[0018] In some schemes, the data processing device determines the pose information of the detection device used in the association point group process, including the position and / or attitude information obtained by online calibration. Online calibration refers to calibrating the position and / or attitude of the detection device by using the data reported by the detection device, so as to accurately determine the position and / or attitude of the detection device.

[0019] In another possible implementation of the first aspect, the plurality of detection devices includes a first detection device and a second detection device, and the pose information of the first detection device includes a calibrated orientation angle of the first detection device. The method further includes: aligning the reference system of the first reported data and the second reported data to obtain first calibration data and second calibration data, and determining the calibrated orientation angle of the first detection device according to the channel data of the detection points in the first calibration data and the channel data of the detection points in the second calibration data.

[0020] Optionally, when the orientation angle of the first detection device is the calibrated orientation angle, the channel data of the detection points in the first calibration data and the channel data of the detection points in the second calibration data have the maximum correlation. Illustratively, the first reported data includes distance information, speed information and channel data of a plurality of detection points, and the second reported data also includes distance information, speed information and channel data of a plurality of detection points. The data processing device aligns the reference system of the first reported data and the second reported data, and converts the distance information and the speed information in the first reported data and the second reported data. The converted distance information and speed information in the first reported data and the second reported data are coordinate system aligned. The data processing device performs angle search on the channel data of the detection points after position alignment, and obtains the angle with the maximum channel data correlation to determine the calibrated orientation angle of the first detection device. Illustratively, the angle search process is as follows: the data processing device first tries the case that the calibrated orientation angle of the first detection device is angle 1, performs correlation angle calculation to obtain a global correlation angle value, and the global correlation angle value is used to evaluate the correlation. The data processing device tries the case that the calibrated orientation angle of the first detection device is angle 2, performs correlation angle calculation to obtain a global correlation angle value. The data processing device performs angle search by continuously trying multiple angles, obtains the orientation angle with the maximum correlation, and takes the angle with the maximum correlation as the calibrated orientation angle of the first detection device.

[0021] In the above scheme, the reporting data provided by the detection device is affected by the reference system of the detection device, and the channel data in the reporting data is also affected by the difference in the orientation angle of the detection device. Therefore, the data processing device aligns the positions of the first reporting data and the second reporting data, and aligns the reporting data under the same coordinate system. At this time, the difference between the channel data can determine the orientation angle of the detection device, and therefore, the data processing device determines the calibrated orientation angle of the first detection device by using the channel data of the detection points in the first calibration data and the channel data of the detection points in the second calibration data. By using the first reporting data and the second reporting data, the calibrated orientation angle of the first detection device can be accurately obtained, the result accuracy of the detection point association can be improved, and the angle measurement accuracy can be improved.

[0022] In some cases, due to the change of the installation pose of the detection device, using the pose information in the previous period of time or using the offline calibrated pose information for calculation may cause deviation in angle measurement and fusion. In the above embodiment, the online calibration is used to accurately determine the orientation angle of the detection device, which can improve the accuracy of angle measurement and improve the confidence of the detection result.

[0023] Alternatively, the calibrated orientation angle calculated by the above embodiment is a relative orientation angle relative to the second detection device. Or alternatively, the calibrated orientation angle can also be a relative orientation angle relative to other reference systems, for example, the relative orientation angle of the second detection device relative to other reference systems is also used in the calculation process, so that the calibrated orientation angle of the first detection device is a relative orientation angle relative to other reference systems. Or alternatively, the calibrated orientation angle calculated by the above embodiment is an absolute orientation angle, for example, the absolute orientation angle information of the second detection device is also used in the process of calculating the calibrated orientation angle. For example, the absolute orientation angle of the second detection device is 2°, and the orientation of the first detection device deviates from the direction away from the absolute reference system by 30° relative to the second detection device, and then the calibrated orientation angle of the first detection device is 32°.

[0024] In another possible implementation of the first aspect, the data processing device determines the calibrated orientation angle of the first detection device according to the channel data of the detection points in the first calibration data and the channel data of the detection points in the second calibration data, including the following operations: compensating the channel data of the detection points in the first calibration data by using the installation position of the first detection device and the installation orientation angle of the first detection device, and determining the calibrated orientation angle of the first detection device according to the compensated channel data of the detection points in the first calibration data and the channel data of the detection points in the second calibration data.

[0025] The first calibration data includes distance information of the detection points after calibration and speed information of the detection points after calibration, and the first calibration data is positionally aligned with the second calibration data. Optionally, the first calibration data further includes lane data of the detection points, and in this case, the lane data of the detection points in the first calibration data is the same as the lane data of the detection points in the first reported data.

[0026] In some embodiments, the calibration orientation angle of the first detection device is related to a calibration bias angle of the first detection device and an installation orientation angle of the first detection device, the calibration bias angle representing a deviation between an actual orientation angle of the first detection device and the installation orientation angle. Further, when the bias angle of the first detection device is the calibration bias angle, the lane data of the detection points in the compensated first calibration data has the greatest correlation with the lane data of the second calibration data.

[0027] In the above embodiments, the data processing device compensates the lane data from the first detection device for the difference in data caused by the installation position and the installation orientation angle of the first detection device, and calculates the calibration bias angle based on the compensated lane data. Since the difference in position and installation orientation angle of the first detection device is compensated, the calibration bias angle can be calculated more accurately, and the accuracy of the determined calibration bias angle and calibration orientation angle can be improved, thereby further improving the angle measurement accuracy.

[0028] In some possible embodiments, when calculating the calibration bias angle, the data processing device attempts to match the first calibration data with the second reported data at different calibration angles (i.e., an angle search process), and calculates a correlation angle, and the angle at which the correlation angle is the greatest is taken as the calibration bias angle.

[0029] In another possible implementation of the first aspect, the data processing device also matches the distance and speed of the detection points during the online calibration process to obtain detection points with matched distance and speed. Further, for these detection points with matched distance and speed, the data processing device compensates the lane data for the difference in lane data caused by the position (and optionally the attitude) of the detection device using the position (and optionally the attitude) of the detection device. The data processing device determines the calibration orientation angle (or the calibration bias angle) of the first detection device based on the lane data (compensated) of the detection points with matched distance and speed.

[0030] Exemplarily, the data processing apparatus compensates the channel data of the detection points in the first calibration data by using the installation position of the first detection device and the installation orientation angle of the first detection device, including the following operations: the data processing apparatus determines at least one calibration associated point group based on the first calibration data and the second calibration data, each calibration associated point group including a plurality of detection points matched with the distance information, and each calibration associated point group including the detection points in the first calibration data and the detection points in the second calibration data. The data processing apparatus compensates the channel data corresponding to the detection points from the first reported data in the at least one calibration associated point group by using the installation position of the first detection device and the installation orientation angle of the first detection device. The data processing apparatus obtains the calibration deflection angle of the first detection device according to the channel data of the detection points in the first calibration data after compensation and the channel data of the detection points in the second calibration data, including the following operations: the data processing apparatus obtains the calibration deflection angle of the first detection device according to the channel data in the at least one calibration associated point group after compensation.

[0031] In a further possible implementation form of the first aspect, the data processing apparatus obtains the calibration deflection angle of the first detection device according to the channel data of the detection points in the first calibration data and the channel data of the detection points in the second calibration data, including the following operations: the data processing apparatus aligns the reference systems of the first reported data and the second reported data to obtain the first calibration data and the second calibration data, compensates the channel data of the detection points in the first calibration data by using the installation position of the first detection device, and obtains the calibration orientation angle of the first detection device according to the channel data of the detection points in the first calibration data after compensation and the channel data of the detection points in the second calibration data.

[0032] Further, when the orientation angle of the first detection device is the calibration orientation angle, the channel data of the detection points in the first calibration data after compensation has the maximum correlation with the channel data of the second calibration data.

[0033] In the above implementation form, the data processing apparatus compensates the first reported data to compensate the data difference caused by the installation position of the first detection device, and determines the calibration orientation angle of the first detection device based on the channel data after compensation. Since the position difference of the first detection device is compensated, the calibration orientation angle calculation is more accurate, and the accuracy of the determined calibration deflection angle and calibration orientation angle is improved, thereby further improving the angle measurement accuracy.

[0034] In some schemes, the reported data can include two parts of data, the RV spectrum data and the channel data of the detection points, wherein the RV spectrum data includes the velocity information of the detection points and the distance information of the detection points. When the positions are aligned, the RV spectrum data of the multiple radars can be aligned, i.e., the reference systems relative to the velocity information and the distance information of the detection points are aligned. When the channel data is compensated, the channel data corresponding to the detection points is compensated.

[0035] In a further possible implementation form of the first aspect, the data processing device determines the at least one group of associated points based on the pose information of the at least one probe device, comprising the following steps: the data processing device aligns the reference systems of the plurality of reported data based on the pose information of the at least one probe device to obtain a plurality of aligned data, wherein each aligned data corresponds to one reported data, and each aligned data comprises the aligned distance information of the detection points and the aligned velocity information of the detection points, and the plurality of aligned data are aligned with reference to the first coordinate system. Further, the data processing device performs detection point association based on the plurality of aligned data to obtain the at least one group of associated points.

[0036] It should be understood that the number of the group of associated points can be one or more, which is not strictly limited here.

[0037] In the above implementation form, the plurality of reported data are all subjected to coordinate system conversion. For a certain reported data, the pose information of the probe device corresponding to the group of reported data is used to compensate for the measurement deviation caused by the position difference of the probe device, so as to align the reported data of the plurality of probe devices under the same reference coordinate system. In addition, the use of the first coordinate system for alignment can also facilitate the fusion of the detection results of the plurality of probe devices, and can improve the practicability of the detection results.

[0038] The following will be described by taking the position alignment of the first reported data from the first probe device as an example. The plurality of reported data comprise the first reported data, and the first reported data is from the first probe device of the plurality of probe devices. The data processing device aligns the reference systems of the plurality of reported data based on the pose information of the at least one probe device to obtain a plurality of aligned data, comprising the following operations: the data processing device performs position alignment on the first reported data by using the pose information of the first probe device to obtain first aligned data. The first aligned data comprises the aligned distance information of the detection points and the aligned velocity information of the detection points.

[0039] In the above implementation form, the reported data can comprise two parts of data, the RV spectrum data and the channel data of the detection points, wherein the RV spectrum data comprises the velocity information of the detection points and the distance information of the detection points. When performing position alignment, the RV spectrum data of the plurality of radars can be subjected to position alignment, i.e., the reference systems relative to which the velocity information and the distance information of the detection points are aligned.

[0040] In a further possible implementation form of the first aspect, the first coordinate system is a coordinate system predefined in advance, for example, when the plurality of probe devices are arranged on a vehicle, the first coordinate system can be a vehicle body coordinate system.

[0041] In a further possible implementation form of the first aspect, the first coordinate system can be a coordinate system of one of the plurality of detection devices. The coordinate system of the detection device is the coordinate system of the reported data of the detection device.

[0042] In a possible implementation form of the first aspect, the plurality of detection devices comprises a first detection device and a second detection device, and the at least one detection device comprises the first detection device, and the first coordinate system is a coordinate system of the second detection device.

[0043] Exemplarily, the data processing device determines the at least one associated point group based on the pose information of the at least one detection device, comprising the following operations: the data processing device utilizes the pose information of the first detection device to perform position alignment on the first reported data to obtain first aligned data, the first aligned data comprising aligned distance information of the detection points and aligned velocity information of the detection points. The data processing device performs detection point association based on at least the second aligned data and the first aligned data to obtain the at least one associated point group, the first aligned data and the second aligned data being aligned with reference to the coordinate system of the second reported data. Optionally, the second aligned data is the same as the second reported data, and the second aligned data comprises the aligned distance information of the detection points and the aligned velocity information of the detection points.

[0044] In the above implementation form, the first detection device and the second detection device are two detection devices at different positions. When performing detection point association, the pose information of the first detection device is utilized to convert the first reported data reported by the first detection device to the coordinate system of the second reported data, thereby obtaining the second aligned data. Since the second reported data has been aligned with the coordinate system, it is not necessary to perform conversion, and thus the second reported data can be directly used as the aligned data of the coordinate system, i.e., the second aligned data. The data processing device determines the associated detection points based on the first aligned data and the second aligned data (or the second reported data), thereby determining which detection points in the first reported data and the second reported data are associated.

[0045] In addition, taking the coordinate system of the reported data of one of the detection devices as the converted coordinate system makes the coordinate systems of the plurality of reported data aligned with reference to the coordinate system of the reported data, which can reduce the data amount of coordinate conversion, reduce the matching deviation caused by coordinate conversion, is conducive to improving the accuracy of the joint angle measurement result, and improves the efficiency of data processing.

[0046] Optionally, the plurality of detection devices comprises a plurality of detection devices arranged along a first direction, for example, three radars installed at the head of the vehicle. The second detection device is a detection device located at a middle portion of the plurality of detection devices, or the plurality of detection devices is a detection device closest to a midpoint of the terminal along the first direction, the first direction being a front-rear direction or a left-right direction of the terminal. That is, the data processing device can transfer the reported data of the edge detection device to the coordinate system of the central detection device, so as to facilitate subsequent point cloud fusion, improve the usability of the detection result, and improve the intelligent level of the terminal.

[0047] In some cases, the pose information of the detection device can indicate the position and attitude of the detection device. Further, in the above-mentioned embodiments, the pose information of the first detection device can indicate the position and attitude of the first detection device relative to the second detection device.

[0048] In another possible implementation of the first aspect, the pose information of the detection device comprises an installation position of the detection device and attitude information of the detection device. Taking the first detection device as an example, the installation position of the first detection device can comprise one or more of the following information: a coordinate of the installation position of the first detection device, a distance of the installation position of the first detection device relative to the origin (or coordinate axis) of the first coordinate system, etc. The distance here can include a translation distance along one or more directions, or a radial distance. The attitude information of the first detection device comprises one or more of the following: a calibration orientation angle of the first detection device, a calibration deflection angle of the first detection device, an installation orientation angle of the first detection device, a measurement target deflection angle of the first detection device, etc. In some cases, the calibration deflection angle of the first detection device needs to be used in combination with the installation orientation angle of the first detection device to indicate the calibration orientation angle of the first detection device. While the installation angle of the first detection device can be used alone.

[0049] The following continues to take the process of position alignment of the first reported data from the first detection device as an example. In another possible implementation of the first aspect, the pose information of the first detection device comprises an installation position of the first detection device and attitude information of the first detection device. The aligned distance information of the detection points in the first aligned data is related to the distance information of the detection points in the first reported data, the attitude information of the first detection device, and the installation position of the first detection device. The aligned velocity information of the detection points in the first aligned data is related to the velocity information of the detection points in the first reported data and the installation position of the first detection device.

[0050] Optionally, the attitude information of the first detection device includes the orientation angle of the first detection device. This orientation angle can be the calibration orientation angle, the installation orientation angle, etc. In some cases, the orientation angle of the first detection device can be calculated from the calibration deflection angle and the installation orientation angle. In this case, the installation orientation angle is the angle obtained from offline calibration.

[0051] In another possible implementation of the first aspect, the transformed distance information R of the detection points in the alignment data i Distance information R between the detection points and the reported data i It satisfies the following formula:

[0052] R i ′=M i ·M shift_i ·R i

[0053] Among them, R i To report the distance information of the detection points in the data, M i Let M represent the rotation matrix. shift_i This represents the translation matrix. The aforementioned rotation and translation matrices are related to the pose information of the detection device. Rotation matrix M i Let be the rotation matrix of the detection device i relative to the first coordinate system.

[0054] The conversion process will now be described using the first detection device as an example. For instance, the rotation matrix M1 of the first detection device (taking i=1 as an example) satisfies the following equation:

[0055]

[0056] in, Let be the rotation angle of the first detection device relative to the first coordinate system. It satisfies the following formula:

[0057]

[0058] Where γ1 is the installation orientation angle (or installation angle) of the first detection device, ε mis_1 Let θ1 be the calibration deflection angle of the first detection device and θ2 be the target deflection angle measured by the first detection device. Taking the first coordinate system as the coordinate system of the second detection device as an example, the target deflection angle θ1 of the first detection device can be calculated by the following formula:

[0059]

[0060] Where R1 is the distance measured by the first detection device to a point in space, R2 is the distance between the second detection device and that point, and d is the distance between the first detection device and the second detection device.

[0061] Exemplarily, the first detection device (taking i = 1 as an example) has a translation matrix M i satisfies the following equation:

[0062]

[0063] wherein d is a translation distance of the first detection device relative to the origin of the first coordinate system, and in the case that the first coordinate system is the coordinate system of the second detection device, d is the distance between the first detection device and the second detection device.

[0064] In yet another possible implementation of the first aspect, the translated distance information V i of the detection point in the alignment data matches the distance information V i of the detection point in the report data, satisfying the following equation:

[0065] V i ′= V i *cosθ1.

[0066] In yet another possible implementation of the first aspect, the distance information indicates a distance or a distance range of the detection point. Exemplarily, the distance information comprises a distance bin index, also referred to as a distance index or a distance bin number.

[0067] Similarly, the velocity information indicates a velocity or a velocity range of the detection point. Exemplarily, the velocity information comprises a velocity bin index, also referred to as a velocity index or a velocity bin number.

[0068] In yet another possible implementation of the first aspect, the distance information and the velocity information corresponding to the plurality of detection points in the associated point group match.

[0069] As a possible case, the plurality of report data comprises first report data and second report data, and the first associated point group comprises a first detection point derived from the first report data and a second detection point derived from the second report data. The distance information comprises a distance bin index, and the velocity information comprises a velocity bin index. The distance information and the velocity information of the first detection point and the second detection point match under the following two conditions: Condition 1, the aligned distance bin index of the first detection point is the same as the aligned distance bin index of the second detection point, or the difference between the aligned distance bin index of the first detection point and the aligned distance bin index of the second detection point is less than a first distance threshold. Condition 2, the aligned velocity bin index of the first detection point is the same as the aligned velocity bin index of the second detection point, or the difference between the aligned velocity bin index of the first detection point and the aligned velocity bin index of the second detection point is less than a first velocity threshold.

[0070] As another possible scenario, the associated point group includes a first detection point and a second detection point, which originate from different reported data. The aligned distance information of the detection points is used to indicate the aligned distance, and the aligned velocity information is used to indicate the aligned velocity. The distance and velocity information of the first and second detection points match if at least two of the following conditions are met: Condition 1: The difference between the aligned distance of the first and second detection points is less than or equal to a second distance threshold. Condition 2: The difference between the aligned velocity of the first and second detection points is less than or equal to a second distance threshold.

[0071] In another possible implementation of the first aspect, the data processing device obtains a group of associated points by associating detection points based on multiple aligned data sets, including the following operations: the data processing device matches the velocity information and distance information of the detection points in the multiple aligned data sets to obtain a candidate group of associated points where the distance and velocity match. The matching conditions can refer to the two cases described above. This candidate group of associated points can be directly used as the group of associated points, or the candidate group of associated points needs to be further filtered to obtain the group of associated points. For example, in the latter case, the data processing device performs a channel consistency check on the detection points in the candidate group of associated points, filters the detection points in the candidate group of associated points that meet the channel consistency condition, and obtains at least one group of associated points.

[0072] In another possible implementation of the first aspect, the data processing device determines at least one angle information based on channel data corresponding to multiple detection points in the first associated point group, including the following steps: the data processing device obtains multiple frequency domain data based on the channel data corresponding to multiple detection points in the first associated point group, obtains summary data based on the multiple frequency domain data, and obtains at least one angle information based on the summary data.

[0073] The above embodiments provide an exemplary joint angle measurement process. The data processing device converts the channel data to the frequency domain and summarizes the frequency domain data. Using the summarized data, one or more angle values ​​(or angle value ranges) are obtained, for example, through spectrum analysis. Using channel data from multiple detection devices for joint angle measurement can improve angle measurement accuracy and increase the confidence level of the detection results.

[0074] In another possible implementation of the first aspect, the data processing method further includes: a data processing device generating joint point cloud data, the joint point cloud data including information of multiple points, the information of each point including the point's velocity information, the point's distance information and the point's angle information, the point's angle information being related to at least one angle information.

[0075] In the above-mentioned embodiments, the data processing apparatus can generate point cloud data, the speed information of the points in the point cloud data is obtained from the speed information of the detected points in the multiple pieces of reported data, the distance information of the points is obtained from the distance information in the multiple pieces of reported data, and the angle information of the points is related to the at least one angle information mentioned above. The point cloud data reflects the coordinates of the points and the related information of the points, can facilitate target identification, fusion perception and other processing of the back end, and has high ranging, speed measurement and angle measurement accuracy, high value and good usability.

[0076] Further, the point cloud data can be output as a detection result, or after target identification on the point cloud data, the result of the target identification is output as the detection result.

[0077] In a further possible implementation form of the first aspect, the plurality of detection apparatuses are installed on a terminal. The data processing apparatus generates joint point cloud data, including the following steps: the data processing apparatus generates the joint point cloud data based on the absolute pose information of the at least one detection apparatus, and the reference system of the joint point cloud data is an absolute coordinate system.

[0078] In the above-mentioned embodiments, the coordinate system of the joint point cloud data is the absolute coordinate system, for example, the reference is the geodetic coordinate system. In this way, when the terminal uses the joint point cloud data, in the case that the pose of the terminal changes, there is no need to convert the joint point cloud data, and the usability of the joint point cloud data can be improved.

[0079] In a further possible implementation form of the first aspect, the absolute pose information of the at least one detection apparatus includes an absolute orientation angle of the at least one detection apparatus. Further, the data processing method further includes: the data processing apparatus determines the absolute orientation angle of the first detection apparatus based on the calibrated orientation angle of the at least one detection apparatus, the pose information of the terminal, the joint point cloud data and the independent point cloud data of the first detection apparatus, and the first detection apparatus belongs to the at least one detection apparatus. The independent point cloud data of the first detection apparatus is the point cloud data obtained based on the reported data of the first detection apparatus.

[0080] In the above-mentioned embodiments, in the process of determining the absolute orientation angle of the first detection apparatus, the point cloud data independently detected by the first detection apparatus and the joint point cloud data obtained by the data processing apparatus can be used, which can further improve the accuracy of the calibrated bias angle of the detection apparatus, is conducive to improving the accuracy of the subsequent detection point association, and thus improves the accuracy of the angle obtained by joint angle measurement. In some cases, the result of target identification can also be used in the process of determining the absolute orientation angle of the detection apparatus.

[0081] In a further possible implementation form of the first aspect, the detection signal is a radio electromagnetic wave.

[0082] In a further possible implementation form of the first aspect, the probe signal is a light beam, such as a FMCW laser beam.

[0083] In a second aspect, the application provides a probe system, comprising a data processing apparatus and a plurality of probe apparatuses. The plurality of probe apparatuses are configured to provide report data to the data processing apparatus, each item of the report data being data obtained by a corresponding probe apparatus by probing a physical space using a probe signal. The data processing apparatus is configured to implement the method described in the first aspect or any possible implementation form of the first aspect.

[0084] Further, the plurality of probe apparatuses are arranged at different positions of the terminal. Optionally, the data processing apparatus is communicatively connected with the plurality of probe apparatuses.

[0085] In a possible implementation form, the plurality of probe apparatuses comprises a first probe apparatus, the first probe apparatus being configured to use a first probe signal to probe the physical space, and the second probe apparatus being configured to use a second probe signal to probe the physical space.

[0086] In a third aspect, the application provides a data processing apparatus, comprising a data obtaining module and a processing module, the data obtaining module being configured to transmit data with a plurality of probe apparatuses, and the processing module being configured to process the data. The data processing apparatus is configured to implement the method described in the first aspect or any possible implementation form of the first aspect.

[0087] In a fourth aspect, the application provides a data processing apparatus, comprising a memory and at least one processor, the memory being configured to store computer instructions, and the at least one processor being configured to invoke the computer instructions stored in the memory to implement the method described in the first aspect or any possible implementation form of the first aspect.

[0088] In a fifth aspect, the application provides a chip, comprising a communication interface and at least one processor, the communication interface being configured to input data, and the at least one processor being configured to execute computer instructions to implement the method described in the first aspect or any possible implementation form of the first aspect.

[0089] In a sixth aspect, the application provides a terminal, comprising the probe system described above, or comprising the data processing apparatus, or comprising the chip.

[0090] In a seventh aspect, the application provides a computer readable storage medium, comprising computer program instructions, which when executed by at least one processor, implement the method described in the first aspect or any possible implementation form of the first aspect.

[0091] In an eighth aspect, the application provides a computer program product comprising instructions, which when executed by at least one processor, implement the method described in the first aspect or any possible implementation form of the first aspect.

[0092] The beneficial effects of aspects two through eight of this application can be found in the beneficial effects of aspect one. Attached Figure Description

[0093] The accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0094] Figure 1 This is a schematic diagram of distance dimension data, velocity dimension data, and RV data;

[0095] Figure 2 This is a schematic diagram of a point cloud-level fusion detection system architecture;

[0096] Figure 3 This is a schematic diagram of a satellite radar-based detection system architecture;

[0097] Figure 4 This is a schematic diagram of the architecture of a detection system provided in an embodiment of this application;

[0098] Figure 5 This is a schematic diagram of a vehicle provided in an embodiment of this application;

[0099] Figure 6 This is a flowchart illustrating a data processing method provided in an embodiment of this application;

[0100] Figure 7 This is a schematic diagram of the poses of multiple detection devices provided in the embodiments of this application;

[0101] Figure 8 This is a schematic diagram of two reported data items provided in an embodiment of this application;

[0102] Figure 9 This is a schematic diagram of the installation location of a radar provided in an embodiment of this application;

[0103] Figure 10 This is a schematic diagram illustrating the consistency of the two channel data provided in the embodiments of this application;

[0104] Figure 11 This is a schematic diagram of two alignment data provided in an embodiment of this application;

[0105] Figure 12 This is a schematic diagram of a joint point cloud data provided in an embodiment of this application;

[0106] Figure 13 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0107] Figure 14 This is a schematic diagram of the structure of another data processing device provided in the embodiments of this application. Detailed Implementation

[0108] First, some technical terms are introduced.

[0109] The detection device refers to a device that uses a detection signal to detect the object space. The detection signal is an electromagnetic wave, such as a radio wave or a light beam, etc. When the detection signal is a radio wave, the detection device can be called a radar (radio detection and ranging, Radar). When the detection device is a light beam, the detection device can be called a Lidar (light detection and ranging, Lidar). Further, the detection signal is a frequency modulated continuous wave (FMCW), and the frequency of the FMCW changes with time, and the change law includes one or more of sawtooth, triangle or sine.

[0110] Channel refers to an independent path for signal transmission and processing, such as a path for transmitting (T) signals by the detection device, or a path for receiving (R) signals, or a path containing both transmission and reception. The number of channels is related to the number of transmitting units and receiving units in the detection device. Taking a radar as an example, if 1 transmitting antenna and 1 receiving antenna are set, the number of channels is 1. In some schemes, by using multiple-input multiple-output (MIMO) technology, virtual channels can be formed, thereby increasing the number of channels of the detection device. For example, a 3T4R radar can form 12 channels.

[0111] Fast Fourier transform (FFT) is a process of converting time-domain signals into frequency-domain signals. FFT can decompose a signal that varies with time (i.e., a time-domain signal) into a combination of different frequency components (i.e., a frequency-domain signal). The receiving end of the detection device receives signals in the object space to form a sample signal. The signal processing module can obtain range data after performing FFT (i.e., one-dimensional FFT, or 1D FFT) on the sample signal (which can be optionally pre-processed by filtering, noise reduction, etc.). Velocity data, or Doppler data, can be obtained after performing FFT again (i.e., 2D FFT).

[0112] In some embodiments, the distance data and the velocity data are fused to form range-velocity (RV) data, which includes information of three dimensions, i.e., distance information, velocity information, and channel data corresponding to the detection points. In some embodiments, the information of the three dimensions of the RV data can also be split, for example, into RV spectrum data and channel data, where the RV spectrum data includes distance dimension data and velocity dimension data of the detection points, and the channel data is channel data corresponding to the detection points. For ease of understanding, the three dimensions of the RV data are introduced as follows:

[0113] In the distance dimension, different targets at different distances will return echoes at different times after the detection device transmits a detection signal. Based on the time of arrival of the echoes, the entire detection distance range is divided into multiple distance bins (or distance units or distance gates). The relevant information of the echoes in each distance bin is recorded, such as the echo amplitudes and phases of multiple channels, to form channel data. For example, Figure 1 Part (a) of FIG. 1 is a distance dimension data, and the shaded part is a distance range in which echoes are suspected to exist. The distance data also records channel data corresponding to the distance bins in which echoes exist. In the RV data, the distance dimension is usually a coordinate axis, and the data points thereof correspond to different distance bins. The data in these detection points can include the amplitudes, phases, and the like of the echoes received by multiple channels in the distance bins.

[0114] In the velocity dimension, when the targets in the object space have radial motion relative to the detection device, the frequency of the received echoes will change according to the Doppler effect. By analyzing the frequency change of the echo signals, the radial velocity information of the targets can be obtained. The detection device will perform spectral analysis on the echo signals to separate different frequency components, and these frequency components correspond to targets at different velocities. For example, Figure 1 Part (b) of FIG. 1 separates different frequency components to obtain multiple velocity bins, and the shaded part is a velocity range in which targets are suspected to exist. The velocity data also records channel data corresponding to the velocity bins in which echoes exist. In the RV data, the velocity (or Doppler) dimension is also a coordinate axis, and the data points thereof correspond to different velocity bins (or velocity units or velocity gates). The data in each velocity bin can include the amplitudes, phases, and the like of the echoes received by multiple channels in the velocity bin.

[0115] In the RV data, if a feature signal (detected after CFAR processing) that is obviously different from background noise appears at a certain position (i.e., the intersection point of a distance bin and a velocity bin), it indicates that a target is likely to exist, and the position of the feature signal can be used to determine the distance and velocity of the target, thereby realizing the positioning of the target. For example, Figure 1In the (c) part of the RV data, a plurality of detection points satisfying a detection condition are included, and the detection points have corresponding distance bin indexes and velocity bin indexes, and corresponding channel data. Alternatively, Figure 1 The channel dimension can also be replaced by an antenna dimension, and the data of the channel dimension can be separated from the data received by the antenna.

[0116] Constant false-alarm rate (CFAR) processing is a signal processing algorithm that aims to maintain a relatively stable false alarm probability in different background noise and interference environments, so as to effectively detect the real target signal. False alarm refers to the case where the detection system incorrectly determines noise or other interference signals as target signals. The signals received by the detection device include target reflections in the space and background noise (and possibly interference signals). Exemplarily, the CFAR algorithm includes one or more of cell average CFAR (CA-CFAR), ordered statistics CFAR (OS-CFAR), and minimum selection CFAR (SO-CFAR).

[0117] As a possible example, CFAR processing is based on analysis of the noise and interference statistical characteristics in the reference cells (also referred to as background cells) around the received signal. Based on the measurement data in these reference cells, such as signal power or amplitude statistics, the detection threshold is adaptively adjusted. Exemplarily, assume that a signal is detected in a certain distance bin (i.e., the detected cell) during the detection process. Taking CA-CFAR as an example, the CFAR processing selects some cells (such as cells in a ring or rectangular region) around the detected cell as reference cells, calculates the average power of these reference cells (assuming that the average power CFAR algorithm is used), and then determines the appropriate detection threshold according to the preset false alarm rate and some statistical parameters. If the signal power of the detected cell exceeds this threshold, it is determined that there is a target signal, and the detected cell can be used as a detection point. Otherwise, it is determined that there is no target signal.

[0118] Direction of arrival (DOA) processing is mainly used to determine the direction of arrival of a signal received by a detection device. Through DOA processing, the device can determine the azimuth and elevation of the target relative to the transceiver. In a detection system, DOA processing relies on an array antenna to achieve, which is arranged by a plurality of antenna units in a certain geometric shape (such as linear array, planar array, etc.). When the target signal arrives at the array antenna, due to the difference in spatial position of each antenna unit, the received signal will produce a difference, including phase difference and / or amplitude difference. These differences can be recorded by channel data, so based on the channel data, the direction of arrival of the echo of the target can be calculated, thereby locating the angle of the target.

[0119] The foregoing explanations of technical terms can be optionally used in the embodiments hereinafter.

[0120] Currently, more and more terminals are deployed with multiple detection devices in order to comprehensively detect the object space from multiple angles. Currently, for a detection system including multiple detection devices, manufacturers mainly adopt a point cloud level fusion architecture or a satellite radar architecture, both of which have their own limitations. The following takes a radar as an example to introduce the two architectures and the problems existing in each of them.

[0121] Please refer to Figure 2 , Figure 2This is a schematic diagram of a point cloud-level fusion detection system architecture. The system comprises N radars, where N is a positive integer. Each radar includes a monolithic microwave integrated circuit (MMIC), which integrates a radio frequency (RF) module, a 2DFFT module, a CFAR module, and a DOA module. The RF module transmits and receives signals. The MMIC processes the received signals to obtain sampled signals. The 2DFFT module processes the sampled signals to obtain range and velocity data. The CFAR module processes the range and velocity data to obtain RV data, which includes the range, velocity, and channel data corresponding to the detection point. The DOA uses the RV data for angle measurement to obtain the angle information of the detection point. Finally, the radar outputs independent point cloud data to the backend (taking a data processing device as an example). The points in the point cloud include range, velocity, and angle. For a detection system comprising N (N is an integer and N≥2) radars, each radar performs range, velocity, and angle measurements before outputting point cloud data to the data processing device. The data processing device performs point cloud-level fusion based on point clouds from N radars to obtain detection results, which can be either fused point clouds or target identification results. Currently, due to the limited angle measurement capability of a single radar, fusing data from multiple detection devices in the form of point clouds provides only a slight improvement in angle measurement accuracy, making it difficult to meet the required confidence level of the fused detection results. Moreover, this architecture requires high-performance processing chips in the radar to complete the angle measurement process, resulting in high costs.

[0122] compared to Figure 2 The architecture shown retains the MMIC (Micro-MIC) within the satellite radar architecture. The MMIC processes the received signals to obtain sampled signals and provides these samples to the backend. The following explanation uses the backend data processing unit as an example. The data processing module completes other signal and data processing procedures. Please refer to... Figure 3 The sampled signal can be a signal after analog-to-digital converter (ADC), therefore in Figure 3 The data is represented as ADC data. In this architecture, the data processing device obtains an independent point cloud for each radar based on the ADC data output by a single radar, and then fuses the independent point clouds of multiple radars to obtain the detection result. Since the angle measurement capability of a single radar is limited, this architecture does not solve the problems of low angle measurement accuracy and poor confidence of the detection result. Moreover, in this architecture, the radar needs to transmit ADC data to the backend, resulting in a large amount of data transmission and high transmission costs.

[0123] In summary, the current detection system generally faces the problems of weak angle measurement capability and poor confidence of detection result, and is difficult to meet the performance requirements of the detection device. Especially in the field of vehicles, due to the complex environment around the vehicle, the distance and speed of the target in the environment have a wide range of values. The detection system needs to use angles to distinguish targets with the same distance and speed. On this basis, if the angle measurement capability of the detection device is weak, it may be difficult to distinguish two targets with the same distance and speed but located at different angles, resulting in poor confidence of the detection result of the detection system. For example, for the static objects on the guardrail (including living objects), static targets hanging on the suspended objects, etc., the detection system is difficult to identify them, which may cause the driving decision of the vehicle to be wrong, for example, to appear sudden braking or even not to brake in time, affecting the comfort and safety of vehicle driving.

[0124] Therefore, the embodiments of the present application provide a data processing method and related products, which can jointly measure angles by combining the associated channels in multiple detection devices, greatly improve the angle measurement capability of the multi-detection device system, significantly improve the accuracy of the detection result, and significantly improve the confidence of the detection result. Further, the channel data of the associated channels meet the channel consistency condition, which can accurately select the associated channels, thereby further improving the accuracy of joint angle measurement.

[0125] The architecture and business scenarios of the detection system to which the embodiments of the present application can be applied are introduced below. It should be noted that the system architecture and business scenarios described in the present application are for more clearly illustrating the technical solutions of the present application, and do not constitute a limitation on the technical solutions provided by the present application. It should be understood that as the system architecture evolves and new business scenarios appear, the technical solutions provided by the present application are also applicable to similar technical problems.

[0126] Please refer to Figure 4 , Figure 4 is a schematic diagram of the architecture of a detection system provided by the embodiments of the present application, and the detection system 100 includes N radars and a data processing device 10, where N is an integer and N≥2. Among them:

[0127] The radar can utilize the probe signal to probe the space, obtain the reporting data and provide it to the data processing device. The reporting data includes the distance information of the detection point, the speed information of the detection point and the channel data corresponding to the detection point. The channel data includes the amplitude, phase and other data of the channel corresponding to the detection point. The angle information of the target can be obtained by processing the channel data. The radar can include MMIC, 2DFFT module and CFAR module (optional). The MMIC module is used to transmit and receive signals and process the received signals to obtain sampling signals. After 2DFFT and CFAR processing (CRAF processing is optional), the reporting data can be obtained. The reporting data includes the distance information (e.g., represented as R) of the detection point, the speed information (e.g., represented as V) of the detection point and the channel data.

[0128] The data processing device 10 is a device with computing capability. The data processing device can perform channel association and joint angle measurement based on the reporting data provided by the N radars. Channel association is to determine the associated detection points in the reporting data of multiple radars, i.e., to establish the association relationship of the detection points. There can be an association relationship between the channels corresponding to these detection points. Joint angle measurement refers to calculating the angle using the channel data corresponding to the associated detection points. In some cases, the data processing device 10 is also used for joint calibration. Joint calibration refers to updating the attitude (or pose) information of the radar using the reporting data provided by at least two radars and combining the existing attitude (or pose) information of the radar. The foregoing existing attitude (or pose) information of the radar can include the attitude (or pose) information of the radar obtained by offline calibration and / or the attitude (or pose) information of the radar obtained by joint calibration of the data processing device.

[0129] In some possible implementation, the radar and the data processing device 10 are communicatively connected, for example, through a wired link or a wireless link. For example, the radar 1 and the data processing device 10 can be connected through wired connection technologies such as Ethernet (e.g., vehicle-mounted Ethernet), media oriented systems transport (MOST), controller area network (CAN), universal serial bus (USB), etc. For another example, the radar 1 and the data processing device 10 can also be connected through wireless connection technologies such as star flash technology, Bluetooth technology, Wireless Fidelity (Wi-Fi) technology, etc. Of course, the connection manners between the N radars and the data processing device can be the same or different. In addition, the present application is also applicable to the case where the radar and the data processing device 10 are not directly connected, for example, the N radars can provide the reported data to an intermediate device, and the intermediate device provides the reported data of the multiple radars to the data processing device 10.

[0130] In some possible implementation, the N radars can be arranged in a terminal, for example, a vehicle, a robot, or a drone, etc. Please refer to Figure 5 , the vehicle can include 6 radars (the number is only an example), of which 3 radars are installed on the front part of the vehicle, and the other 3 radars are installed on the rear part of the vehicle, of course, Figure 5 The installation positions and the number of the radars shown are only examples.

[0131] In some cases, the fields of view of some or all of the multiple detection devices overlap. For example, the radars 1, 2 and 3 are used to detect the field of view in front of the vehicle, and the fields of view of the three radars overlap, that is, the radars 1, 2 and 3 can detect a common detection area.

[0132] Optionally, in the case where the N radars are arranged in the terminal, the data processing device 10 can be arranged in the terminal or outside the terminal, for example, in a server. The possible implementation of the data processing device 10 is described below.

[0133] As mentioned above, the data processing device 10 has computing capability, which can include a hardware module with computing capability and / or a software module with computing capability. The following examples are introduced based on the hardware implementation and the software implementation, respectively.

[0134] As an example of a hardware implementation, data processing apparatus 10 can include at least one processor that is a module having processing capability. In one implementation, the processor can include circuitry having an instruction read and execution capability, such as an arithmetic logic unit, a processor core, a central processing unit (CPU), a microprocessor, a microcontroller unit (MCU), a graphics processing unit (GPU), or a digital signal processor (DSP), etc. In another implementation, the processor can implement certain functionality through a fixed or reconfigurable logic relationship of hardware circuitry, such as an application-specific integrated circuit (ASIC) or a programmable logic device (PLD) such as a field programmable gate array (FPGA). In a reconfigurable hardware circuit, the processor loads a configuration document to implement a configuration of the hardware circuit, which can be understood as the processor loading instructions to implement a corresponding functionality. In addition, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc. In some implementations, data processing apparatus 10 includes at least one processor integrated together in the form of a system-on-chip (SOC), which is commonly referred to by those skilled in the art as an SOC. The SOC can include at least one processor, and when the SOC includes multiple processors, the types of the multiple processors can be different, such as including a CPU and an NPU, etc.

[0135] Exemplarily, the detection system 100 can be applied to a vehicle perception scenario, and the data processing apparatus 10 can be a computing device in the vehicle. For example, the data processing apparatus 10 is an ECU in the vehicle. Exemplarily, the data processing apparatus 10 includes but is not limited to a domain controller (DC), a mobile data center (MDC), an electronic control unit (ECU), a vehicle integrated / integration unit (VIU), etc. Among them, the DC can include a cockpit domain controller (CDC), an intelligent driving domain controller, etc.

[0136] As an example of a software implementation, the data processing apparatus 10 can include a software functional unit. As an example of a software functional unit, the data processing apparatus 10 can include one or more of a computer program, computer code, or computer instructions, which are executable, i.e., run on a processor or computing instance. As another example of a software functional unit, the data processing apparatus 10 can include a computing instance, which includes a virtual machine, a container, etc. Among them, the virtual machine is a computer system that is simulated by software, has complete hardware system functions, and runs in an isolated environment. The container is an isolated environment obtained by packaging an application and application dependency package.

[0137] In some possible implementations, the data processing apparatus 10 is independent of the N radar apparatuses, i.e., as an independent apparatus. In other cases, the data processing apparatus 10 can also be arranged in one of the radars, or be arranged in part of the radars in a separate form of a plurality of modules, in other words, the data processing apparatus 10 can be designed in an integrated manner with the radars.

[0138] Optionally, Figure 4 The radar in the above can be replaced by other detection apparatuses, such as Lidar.

[0139] The method provided by the embodiments of the present application will be described below.

[0140] Please refer to Figure 6 , Figure 6 is a flow diagram of a data processing method provided by the embodiments of the present application. Optionally, the data processing method can be applied to a detection system, such as the detection system shown in the foregoing Figure 4 or Figure 5 . As Figure 6The data processing method shown may include one or more steps S601 to S603. It should be understood that, for ease of description, the method is described in the order of S601 to S603, and is not intended to limit the execution to this specific order. Furthermore, Figure 6 The execution entity in the illustrated embodiment is for ease of description of the method; in actual implementation, the names of the device, information, etc., can be replaced. This application embodiment does not limit the order of execution, execution time, or number of executions of one or more of the above steps. S601 to S603 are as follows:

[0141] S601, the data processing device acquires multiple reported data from multiple detection devices.

[0142] The detection device is a device that uses detection signals to probe the object space, such as radar and lidar. Radar uses radio waves to detect the object space; these radio waves can be transmitted into the object space via an RF module and received by the RF module. Lidar uses a light beam to detect the object space; the transmitting end includes a laser, and the receiving end is a photodetector. Here, the light beam is FMCW light, which enables the detection of target velocity information.

[0143] The reported data is obtained by the detection device using detection signals to probe the object space, including distance information, velocity information, and channel data corresponding to the detection point. Distance information indicates distance or a distance range, such as a distance value or a distance index. Velocity information indicates velocity or a velocity range, such as a velocity value or a velocity index. A channel is an independent path for signal transmission; channel data is the characteristic data of the signal received by the channel (including virtual channels), which may include one or more information such as amplitude, phase, or noise level. Processing the channel data allows for the detection of the target's angle. Figure 1 Part (c) is a schematic diagram of the information expressed by the reported data, which includes distance information, speed information and channel data corresponding to multiple detection points.

[0144] For example, the reported data can be RV data, such as Figure 1 The data shown in section (c) describes the detection point from the perspectives of velocity, distance, and channel. Further, the reported data is the result of 2DFFT and CFAR processing. Of course, in some cases, the reported data can be 2DFFT processed data, while the CFAR processing can be performed by the data processing device. That is, the data processing device can perform CFAR processing on the reported data to obtain the processed reported data. The processed reported data includes the distance information of the detection point, the velocity information of the detection point, and the channel data corresponding to the detection point.

[0145] It should be understood that the reported data obtained by different detection devices for detecting the same object space is usually different due to different poses of the detection devices.

[0146] Exemplarily, in combination with Figure 7 , taking a radar as the detection device and a coordinate system of the radar 2 as the reference system as an example, the orientation angle of the radar 1 relative to the radar 2 is β1, and the orientation angle of the radar 3 relative to the radar 2 is β2. The orientation angle is the angle between the normal line of the radar or the direction of 0° direction angle and / or 0° pitch angle and the reference coordinate system. In some schemes, the orientation angle of the detection device during installation is recorded in the offline calibration process, that is, the installation orientation angle. For example, the installation orientation angle of the radar 1 is γ1. Optionally, the installation orientation angle of the detection device can be used as the orientation angle of the radar. Optionally, the installation orientation angle can be calculated during offline calibration (such as written by the user or automatically written by the program). In some other schemes, the pose of the detection device can change. At this time, the actual orientation angle of the radar can deviate from the installation orientation angle, and this deviation is the deflection angle of the detection device. For example, Figure 7 , the deflection angle of the radar 1 can be represented as ε mis_1 Therefore, in some cases, the orientation angle of the detection device can be calculated by the installation orientation angle and the calibration deflection angle. For example, the orientation angle and / or the calibration deflection angle can be given in the joint calibration process (described below).

[0147] In addition to the difference in the orientation angle, the position of the target in the object space also affects the measurement deviation of the radar. For example, for the target 1, the radar 1 and the radar 2 have a measurement deflection angle θ1, and the radar 3 and the radar 2 have a measurement deflection angle θ2. The measurement deflection angle can also be calculated, for example, by determining the measurement distance of the target 1 by the radar 1 and the radar 2, respectively, and the distance between the radar 1 and the radar 2.

[0148] Please refer to Figure 8 , Figure 8 is a schematic diagram of two reported data, wherein Figure 8 (a) is the reported data from the radar 1 shown in Figure 7 , and Figure 8 (b) is the reported data from the radar 2 shown in Figure 7 . The radar 1 and the radar 2 have an overlapping area that can be detected, that is, the field of view overlaps, so they can detect the same target in the object space. Since the installation positions of the radar 1 and the radar 2 on the terminal are different, the information obtained by detecting the same target is different, but the detection data of the object space by the two is related. In combination with Figure 8As can be seen from (a) and (b), the reporting data provided by the radar 1 includes the detection point L1 and the detection point L2, and the reporting data provided by the radar 2 includes the detection point M1 and the detection point M2, and the detection points in the two reporting data may include the detection results of the same target.

[0149] S602, the data processing apparatus determines at least one group of associated points based on the pose information of the at least one detection apparatus.

[0150] The pose information of the detection apparatus includes the installation position of the detection apparatus and the attitude information of the detection apparatus. Taking the first detection apparatus as an example, the installation position of the first detection apparatus can include one or more of the following information: the coordinates of the installation position of the first detection apparatus, the distance of the installation position of the first detection apparatus relative to the origin (or coordinate axis) of the first coordinate system, etc. The distance here can include a translation distance in one or more directions, or a radial distance. The distance here can include a translation distance in one or more directions, or a radial distance. Please refer to Figure 9 , Figure 9 is a schematic diagram of the installation position of a radar. Taking the coordinate system of the radar 2 as the pre-defined reference coordinate system as an example, in the first direction (such as the X direction), the distance between the radar 1 and the radar 2 is d.

[0151] Continuing to take the first detection apparatus as an example, the attitude information of the first detection apparatus includes one or more of the following: the calibrated orientation angle of the first detection apparatus, the installation orientation angle of the first detection apparatus, the measurement target angle of the first detection apparatus, etc. As a possible example, the orientation angle of the detection apparatus refers to the angle between the normal line (or center line, or 0° azimuth angle and / or 0° pitch angle direction) of the detection apparatus and the reference (or reference coordinate system). In combination with Figure 9 , the orientation angle of the radar 2 is 0°, that is, the coordinate system of the radar 2 is taken as the reference, and the orientation angle of the radar 1 is β1. The target angle is the angle between the target and the origin of the reference coordinate system, and the angle between the target and the detection apparatus. Taking the radar 2 as the reference, the target angle of the radar 2 is 0, and the target angle of the radar 1 is θ1.

[0152] In some possible implementations, the orientation angle of the detection apparatus is the installation orientation angle. The installation orientation angle refers to the deviation angle of the detection apparatus relative to the reference when installed, and the installation orientation angle can be the value obtained during offline calibration.

[0153] In yet some possible implementation, the orientation angle of the detection device is obtained through online calibration. First, the online calibration process is introduced. Online calibration refers to calibrating the position and / or pose of the detection device by using the data reported by the detection device. The online calibration can accurately determine the position and / or pose of the detection device. The online calibration needs to use the reported data of at least two detection devices, and thus is also referred to as joint calibration. Taking the calibration of the orientation angle of the first detection device as an example, the obtained orientation angle is referred to as the calibrated orientation angle of the first detection device, and can be included in the pose information (or the attitude information) of the detection device.

[0154] In the online calibration process, taking the calibration between the first detection device and the second detection device as an example, the data processing device obtains the first calibration data and the second calibration data according to the reference system of the aligned first reported data and the second reported data, and determines the calibrated orientation angle of the first detection device according to the channel data of the detection points in the first calibration data and the channel data of the detection points in the second calibration data. Further, when the orientation angle of the first detection device is the calibrated orientation angle, the channel data of the detection points in the first reported data and the channel data of the detection points in the second reported data have the maximum correlation.

[0155] Exemplarily, the first reported data includes the distance information, the speed information and the channel data of multiple detection points, and the second reported data also includes the distance information, the speed information and the channel data of multiple detection points. The data processing device aligns the reference system of the first reported data and the second reported data, and converts the distance information and the speed information in the first reported data and the second reported data. The converted distance information and speed information in the first reported data and the second reported data are aligned in the coordinate system. The data processing device performs angle search on the channel data of the detection points after the position alignment, and obtains the angle with the maximum channel data correlation to determine the calibrated orientation angle of the first detection device. Exemplarily, the angle search process is as follows: the data processing device first tries the case that the calibrated orientation angle of the first detection device is angle 1, performs correlation angle calculation to obtain the global correlation angle value, and the global correlation angle value is used to evaluate the correlation. The data processing device then tries the case that the calibrated orientation angle of the first detection device is angle 2, performs correlation angle calculation to obtain the global correlation angle value. The data processing device performs angle search by continuously trying multiple angles, and obtains the orientation angle with the maximum correlation. The angle with the maximum correlation is taken as the calibrated orientation angle of the first detection device.

[0156] Optionally, the calibrated orientation angle obtained through the online calibration is the relative orientation angle relative to the second detection device, i.e., the second detection device is the reference coordinate system. For example, the relative orientation angle of the second detection device is 0°, and the calibrated orientation angle of the first detection device is 30°.

[0157] Or alternatively, the calibrated orientation angle can also be a relative orientation angle relative to other reference system, for example, the relative orientation angle of the second detection device relative to other reference system is also used in the calculation process, so that the calibrated orientation angle of the first detection device is relative to the orientation angle of the reference system. For example, the relative orientation angle of the second detection device is 5°, the orientation of the reference system is 0°, and the orientation of the first detection device deviates from the direction away from the reference system by 30° relative to the second detection device, and the calibrated orientation angle of the first detection device is 35°.

[0158] Or alternatively, the calibrated orientation angle calculated by the above embodiment is an absolute orientation angle, for example, the absolute orientation angle of the second detection device can also be used in the process of calculating the calibrated orientation angle. For example, the absolute orientation angle of the second detection device is 2°, the orientation of the first detection device deviates from the direction away from the absolute reference system by 30° relative to the second detection device, and the calibrated orientation angle of the first detection device is 32°.

[0159] In the above online calibration process, the data processing device needs to convert the reported data to the same coordinate system, the purpose is to match the speed dimension and the distance dimension, so that the position of the distance information and the speed information of the detection point in the multiple groups of reported data is aligned (i.e. relative to the same coordinate system). After matching the speed dimension and the distance dimension, the data processing device compensates for the difference between the channel data of the detection point by using the installation position (optionally also including the orientation angle) of the detection device, and determines the calibrated orientation angle of the detection device based on the compensated channel data.

[0160] In order to facilitate understanding, two possible compensation methods are introduced as follows:

[0161] Method 1, the data processing device compensates the channel data from the first detection device, compensates the data difference caused by the installation position and installation orientation angle of the first detection device, and calculates the calibrated orientation angle based on the compensated channel data. Since the position and installation orientation angle difference of the first detection device is compensated, the calibrated orientation angle calculation is more accurate, which can improve the accuracy of the determined calibrated orientation angle and calibrated orientation angle, thereby further improving the angle measurement accuracy.

[0162] As a possible implementation, the data processing device compensates the channel data of the detection points in the first calibration data by using the installation position of the first detection device and the installation orientation angle of the first detection device, and obtains the calibration bias angle of the first detection device according to the channel data of the detection points in the compensated first calibration data and the channel data of the detection points in the second calibration data. The first calibration data includes the distance information of the detection points after calibration and the speed information of the detection points after calibration, and the first calibration data is aligned with the second calibration data in position. Optionally, the first calibration data further includes the channel data of the detection points, and in this case, the channel data of the detection points in the first calibration data is the same as the channel data of the detection points in the first reported data.

[0163] In some embodiments, the calibration orientation angle of the first detection device is related to the calibration bias angle of the first detection device and the installation orientation angle of the first detection device, and the calibration bias angle represents the deviation between the actual orientation angle of the first detection device and the installation orientation angle. Further, when the bias angle of the first detection device is the calibration bias angle, the channel data of the detection points in the compensated first calibration data has the maximum correlation with the channel data of the second calibration data.

[0164] In some cases, the angle searching and compensation process are combined, and the compensation is performed in the process of converting the channel data to the sin domain. In combination Figure 7 Taking radar 1 and radar 2 as an example, the data processing device attempts to take a1 as the calibration bias angle of radar 1, and at this time the orientation angle of the detection device is b1, b1=a1+g1. The data processing device compensates the channel data of radar 1 in the process of converting the channel data of radar 1 to the sin domain. Specifically, the data processing device compensates the channel data obtained by the first detection device by using the distance d between radar 1 and radar 2, the calibration bias angle a1 of radar 1, and the installation orientation angle g1 of radar 1. The compensation process can be represented as follows:

[0165] FFT(exp(-i·2p·d·sin(a1+g1)))

[0166] Wherein, g1 is the installation orientation angle of radar 1, and d is the distance between radar 1 and radar 2. Wherein, the installation orientation angle of radar 2 is 0°, and exp is the exponential of natural logarithm e.

[0167] In some cases, the channel data of radar 2 is also converted to the sin domain. An exemplary conversion process is as follows:

[0168] FFT(exp(-i·2p·d·sin(a1)))

[0169] For the converted sin domain data, the data processing device performs correlation angle calculation to obtain a global correlation angle value, which is used to evaluate the correlation. Similarly, the data processing device continues to attempt to calibrate the bias angle to be a1, a3, and so on, and then the angle value with the largest correlation, such as a1, is taken as the calibration bias angle of the radar 1. j , as the calibration bias angle of the radar 1.

[0170] The calibration orientation angle of the first detection device is related to the calibration bias angle of the first detection device and the installation orientation angle of the first detection device. For example, in combination with Figure 7 , the installation orientation angle of the first detection device is 30°, and the calibration bias angle is 0.3°, so the calibration orientation angle is 30.3°. For another example, the installation orientation angle of the first detection device is 30°, and the calibration bias angle is -0.2°, so the calibration orientation angle is 29.8°. As can be seen, the installation orientation angle and the calibration bias angle of the first detection device can be used to accurately obtain the orientation angle value of the detection device.

[0171] In some possible implementations, when calculating the calibration bias angle, the data processing device attempts to match the first calibration data with the second reported data at different calibration angles (i.e., an angle search process), and calculates the correlation angle. The angle at which the correlation angle is the largest is taken as the calibration bias angle. The angle search process can be referred to in the foregoing description.

[0172] In mode 2, the data processing device compensates the first reported data to compensate for the data difference caused by the installation position of the first detection device, and determines the calibration orientation angle of the first detection device based on the compensated channel data. Compared with case 1, case 2 only compensates for the difference in position, and does not use the installation orientation angle, so the determined angle value is the calibration orientation angle.

[0173] As a possible implementation, the data processing device aligns the reference system of the first reported data and the second reported data to obtain the first calibration data and the second calibration data, compensates the channel data of the detection points in the first calibration data using the installation position of the first detection device, and obtains the calibration orientation angle of the first detection device according to the channel data of the detection points in the compensated first calibration data and the channel data of the detection points in the second calibration data. Further, when the orientation angle of the first detection device is the calibration orientation angle, the channel data of the detection points in the compensated first calibration data and the channel data of the second calibration data have the largest correlation.

[0174] The above two cases are only examples, and in the specific implementation process, the calibration orientation angle of the detection device can also be calculated using the reported data of the detection device through other calculation methods.

[0175] In some embodiments, the data processing apparatus also matches the distance and the speed of the detection points in the online calibration process to obtain detection points with matched distance and speed. Further, for the detection points with matched distance and speed, the data processing apparatus compensates the channel data of the detection points for the difference in the channel data caused by the position (and optionally the attitude) of the detection device using the position (and optionally the attitude) of the detection device. The data processing apparatus determines the calibration orientation angle (or the calibration deflection angle) of the first detection device based on the channel data (compensated) of the detection points with matched distance and speed. In this way, the associated detection point group is determined first, and then the calibration deflection angle of the detection device is calculated based on the associated detection point group, which can improve the accuracy of the calculation of the deflection angle.

[0176] For example, in the case of the first scenario, the data processing apparatus compensates the channel data of the detection points in the first calibration data using the installation position of the first detection device and the installation orientation angle of the first detection device, including the following operations: the data processing apparatus determines at least one calibration associated point group based on the first calibration data and the second calibration data, each calibration associated point group including distance information and a plurality of detection points with matched distance information, and each calibration associated point group including detection points in the first calibration data and detection points in the second calibration data. The data processing apparatus compensates the channel data of the detection points from the first reported data in the at least one calibration associated point group using the installation position of the first detection device and the installation orientation angle of the first detection device. The data processing apparatus obtains the calibration deflection angle of the first detection device based on the channel data of the detection points in the compensated first calibration data and the channel data of the detection points in the second calibration data, including the following operations: the data processing apparatus obtains the calibration deflection angle of the first detection device based on the channel data in the at least one compensated calibration associated point group.

[0177] In some embodiments, the data processing apparatus can also calculate the absolute orientation angle of the detection device based on the calibration orientation angle of the detection device. The absolute orientation angle can be used in the position alignment stage in the subsequent channel association process, or in the stage of generating joint point cloud data, etc. For example, the calibration orientation angle is determined based on a certain reference coordinate system, and the absolute orientation angle of the detection device can be calculated by converting the position of the reference coordinate system. For example, the calibration orientation angle of the first detection device is 30°, and the reference coordinate system is the coordinate system of the second detection device. The absolute orientation angle of the first detection device is 50° by using the absolute pose information of the second detection device, which includes the absolute orientation deflection angle, for example, the absolute orientation deflection angle deviates from the relative absolute reference system (such as the ground reference system) by 20°. In some cases, the absolute pose information of the second detection device can be calculated based on the attitude information (such as the yaw angle, the pitch angle, and the roll angle) of the terminal on which the detection device is installed.

[0178] Optionally, the data processing device can be designed in various ways for the timing of online calibration, such as conditional triggering and / or periodic triggering. As a possible example, the data processing device periodically updates the calibration offset angle of the at least two detection devices based on one or more frames of reported data of the at least two detection devices, and the updated calibration offset angle can be used in subsequent calculation processes, such as the position alignment process (i.e. the coordinate system conversion process). As another example, the data processing device periodically updates the calibration offset angle of the at least two detection devices based on one or more frames of reported data of the at least two detection devices each time the vehicle is powered on. As another example, the data processing device determines the calibration offset angle used by the reported data of the current frame in the position alignment based on the last frame of reported data. Here, the one frame of reported data refers to the data obtained by the detection device in one detection period. Alternatively, the one or more frames of reported data can be replaced by one or more frames of independent point cloud data, or by one or more frames of joint point cloud data.

[0179] The above describes the process of online calibration of the detection device, which can calibrate the actual pose of the detection device and make the results of the channel association stage more accurate. The process of obtaining the associated point group by the data detection device, or the channel association process, is described below.

[0180] As mentioned above, the difference in pose information of the detection device makes the data reported by different detection devices different. Therefore, by combining the positional differences of the detection devices, the association between multiple sets of reported data can be found, and thus the associated detection points in the multiple sets of reported data can be obtained, and at least one associated point group can be determined. Each associated point group includes multiple associated detection points, and the multiple detection points come from at least two sets of reported data. For example, referring to Figure 8 , by combining the pose information of radar 1, it can be determined that detection point L1 and detection point M1 are two associated detection points belonging to one associated point group. However, detection point L2 and detection point M2 are two associated detection points belonging to one associated point group. It should be noted that not all detection points can be matched to associated detection points, i.e. there can be detection points in one set of reported data that are not associated with other reported data.

[0181] Optionally, the multiple detection points in the associated point group can have the following situations: Situation 1, any two detection points among the multiple detection points are associated. Situation 2, a certain detection point is associated with other detection points other than itself, but the other detection points are not necessarily associated. Situation 3, each detection point is associated with at least one other detection point. Of course, other situations can also exist in specific implementations, which are not listed here.

[0182] In a possible implementation, the data processing apparatus can compensate for the deviation between the reported data due to the pose difference of the detection apparatuses, convert the reported data of the plurality of detection apparatuses to the same coordinate system, and match the converted and coordinate-aligned data to determine the associated detection points, which can reflect the detection points obtained by different detection apparatuses for the same target (or the suspected same target). Here, the same target does not necessarily refer to the same object in the object space, but refers to the target with the same characteristics perceived by the detection apparatuses. For example, since the channel data can reflect the results of angle measurement, the consistency between the two sets of channel data obtained by angle measurement for the same target is high. For another example, the data processing apparatus converts the reported data of one detection apparatus to the coordinate system of another set of reported data, so that the distance and speed measured by the two sets of reported data for the same target can be aligned with respect to the same coordinate system. Therefore, from the perspective of data, the data processing apparatus can process the reported data to detect some associated detection points in the reported data of different detection apparatuses, which can reflect the detection points obtained by different detection apparatuses for the same target (or the suspected same target). Here, the same target does not necessarily refer to the same object in the object space, but refers to the target with the same characteristics perceived by the detection apparatuses.

[0183] In some cases, the associated points in at least one associated point group satisfy a certain association relationship. As a possible association relationship, the channel data of the plurality of detection points belonging to the same associated point group satisfies the channel consistency condition. The better the channel consistency of the channel data of the plurality of detection points, the higher the possibility that the plurality of detection points detect the same target (or the suspected same target). Therefore, detecting the channel consistency of the detection points by using the channel data corresponding to the detection points can better screen the associated detection points, and can improve the accuracy of the joint angle measurement result.

[0184] In some cases, the degree of channel consistency reflects the difference in signal strength of the channel data corresponding to the detection points at different azimuth angles. For example, in combination with Figure 10 part (a), the difference in signal strength of the channel data of detection point 1 and detection point 2 within a certain azimuth angle range is small, that is, the channel consistency of the two is good, and the two may belong to the associated detection points. On the contrary, in combination with Figure 10 part (b), the channel consistency of detection point 1 and detection point 3 is poor, and the two may not be associated detection points.

[0185] Exemplarily, the multiple pieces of reported data include first reported data and second reported data, and the first group of associated points includes a first detection point derived from the first reported data and a second detection point derived from the second reported data. The channel data of the first detection point and the channel data of the second detection point satisfy a channel consistency condition, including: a first correlation angle is greater than or equal to a first threshold value, the first correlation angle being related to the channel data of the first detection point and the channel data of the second detection point. Wherein, the first threshold value can represent TH1, TH1≥0.5. For example, TH1 is 0.9. For another example, TH1 is 0.8.

[0186] For example, a calculation method of a correlation angle is as follows:

[0187]

[0188] Wherein, chanData0 is the channel data of the first detection point, chanData1 is the channel data of the second detection point, corrAngle is the correlation angle, H is a conjugate transpose, and n is a dimension of a matrix, for example, a number of channels.

[0189] In yet some schemes, distance information and speed information corresponding to multiple detection points belonging to the same group of associated points are matched. That is, the distance information and the speed information of the multiple detection points in the group of associated points have an associated relationship.

[0190] It should be understood that the matching here is a matching considering the pose difference, for example, in combination with Figure 7 If radar 1 detects target T1, the distance information of the detection point L1 obtained is RL1, and the speed information is VL1. Radar 2 detects target T1, and the distance information of the detection point M1 obtained is RM1, and the speed information is VM1. Wherein, the distance and speed of the detection point L1 are obtained with reference to the coordinate system of the radar 1 detection device, and the detection point M1 is obtained with reference to the coordinate system of the radar 2, and the two detection points are equivalent to detecting the target T1 from different positions and angles. When observing whether the distance and speed of the detection points are matched, the coordinate systems of the reported data of the two can be aligned. For example, the data processing device converts the reference coordinate system of the distance information and the speed information of the detection point L1 to the coordinate system of the radar 2 by using the pose information of the radar 1, and the converted detection point L1 can be represented as L1', the distance information of which is RL1', and the speed information is VL1'. After conversion, the coordinate systems of the two radars are aligned, which is equivalent to detecting the object space at the same position. On this basis, if RL1' and RM1 are the same or similar, and VL1' and VM1 are the same or similar, it means that the two radars may have detected the same target, and the detection point L1 and the detection point M1 may be associated detection points.

[0191] As a possible implementation, in the process of determining the associated point group, the data processing apparatus first aligns the reported data in position. The purpose of position alignment is to convert the reported data of multiple detection devices into the same coordinate system, so as to facilitate the matching of the distance information and the speed information of the detection points. Specifically, the data processing apparatus aligns the reference system of the multiple reported data based on the pose information of at least one detection device, to obtain multiple alignment data, wherein each alignment data corresponds to a reported data, each alignment data includes the aligned distance information of the detection points and the aligned speed information of the detection points, and the multiple alignment data are aligned with reference to the first coordinate system. Further, the data processing apparatus performs detection point association based on the multiple alignment data, to obtain at least one associated point group.

[0192] The following takes the position alignment of the first reported data from the first detection device as an example. The multiple reported data obtained by the data processing apparatus includes the first reported data from the first detection device in the multiple detection devices. The data processing apparatus aligns the first reported data in position by using the pose information of the first detection device, to obtain the first alignment data. The first alignment data includes the aligned distance information of the detection points and the aligned speed information of the detection points. Similarly, the second reported data from the second detection device is aligned in position to obtain the second alignment data.

[0193] The aforementioned first coordinate system has multiple possible cases. Exemplarily, the first coordinate system can be the coordinate system of one of the multiple detection devices, i.e., the coordinate system of the reported data of the detection device. Further exemplarily, the first coordinate system is a coordinate system defined in advance. For example, when the multiple detection devices are arranged on a vehicle, the first coordinate system can be the vehicle body coordinate system. The following will be introduced in detail respectively for the two cases:

[0194] Case 1: The first coordinate system can be the coordinate system of one of the detection devices, i.e., the coordinate system of the reported data of the detection device. For example, in combination with Figure 7 and Figure 8 Since the reported data of the radar 1, the radar 2 and the radar 3 are all obtained based on the respective coordinate systems. The data processing apparatus can align the reported data of the radar 1 and the reported data of the radar 3 to the coordinate system of the radar 2, and determine the associated detection points based on the converted and aligned reported data.

[0195] A specific position alignment process is introduced below. Taking the position alignment of the reported data of the first detection device as an example, the multiple reported data obtained by the data processing device includes first reported data and second reported data, the first reported data is from the first detection device, and the second reported data is from the second detection device, wherein the coordinate system of the first reported data is the coordinate system of the first detection device, and the coordinate system of the second reported data is the coordinate system of the second detection device.

[0196] As a possible example, the data processing device aligns the first reported data to obtain first aligned data using the pose information of the first detection device, and the first aligned data includes the aligned distance information of the detection points and the aligned speed information of the detection points. Please refer to Figure 11 (a), the first aligned data includes detection point L1' and detection point L2', wherein detection point L1' is converted based on detection point L1, and detection point L2' is obtained based on detection point L2, so that the detection points before and after the position conversion are one-to-one. The second reported data has been aligned in the coordinate system, so it does not need to be converted, and thus the second reported data can be directly used as the data aligned in the coordinate system, i.e., the second aligned data, as shown in Figure 11 (b), the second aligned data is consistent with Figure 8 (b). After conversion, the first aligned data and the second aligned data are aligned with reference to the coordinate system of the second reported data, and the data processing device performs detection point association based on at least the second aligned data and the first aligned data to obtain an associated point group, which includes detection points from the first reported data and detection points from the second reported data (the conditions for detection point association are described below).

[0197] Optionally, when converting the first reported data to the coordinate system of the second reported data, the pose information of the first detection device can indicate the position and attitude of the first detection device relative to the second detection device, such as the distance relative to the first detection device, the fixed angle and the measurement angle relative to the second detection device.

[0198] Case 2, the first coordinate system is a coordinate system outside the multiple detection devices. For example, when the multiple detection devices are arranged on a vehicle, the first coordinate system can be a vehicle body coordinate system. At this time, the multiple reported data are all aligned in position, and the aligned reported data all refer to the first coordinate system.

[0199] The above introduces two possible cases of position alignment. In either case, the reported data of one or more detection devices is converted to obtain the converted alignment data. In some schemes, when performing position alignment, the data processing device only converts the distance information and speed information of the detection points to obtain the aligned distance information and aligned speed information of the detection points. For example, the data processing device can calculate the aligned distance information of the detection points using the distance information of the detection points combined with the pose information, and calculate the aligned speed information of the detection points using the distance information of the detection points combined with the pose information. That is, the position conversion process only converts the speed information and speed information of the detection points, which is equivalent to coordinate alignment in the distance dimension and the speed dimension, without changing the channel data of the detection points.

[0200] The above introduces the specific data processing process of position alignment, and the following introduces a possible position alignment calculation process.

[0201] As an example of distance conversion, the aligned distance information of the detection points in the alignment data is related to the distance information of the detection points in the reported data, the pose information of the detection device corresponding to the reported data, and the installation position of the detection device corresponding to the reported data. Exemplarily, the converted distance information R i ′ of the detection points in the alignment data satisfies the following formula: i

[0202] R i ′=M i ·M shift_i ·R i

[0203] wherein R i is the distance information of the detection points in the reported data, M i represents a rotation matrix, and M shift_i represents a translation matrix. The rotation matrix and the translation matrix are related to the pose information of the detection device. The rotation matrix M i is the rotation matrix of the detection device i relative to the first coordinate system.

[0204] In combination with (a) of Figure 8 and (a) of Figure 11 , if the reported data of radar 1 is the first reported data, the converted alignment coordinate system is the coordinate system of radar 2, and the converted reported data is the first reported data. At this time, the rotation matrix M1 of radar 1 (taking i = 1 as an example, which can be regarded as the first detection device) satisfies the following formula:

[0205]

[0206] wherein, ​is a rotation angle of radar 1 relative to radar 2, satisfies the following equation:

[0207]

[0208] wherein γ1 is an installation orientation angle of radar 1, ε mis_1 is a calibrated bias angle of radar 1, and θ1 is a measured target bias angle of radar 1. Taking the first coordinate system as the coordinate system of radar 1 as an example, the measured target bias angle θ1 of radar 1 can be calculated by the following equation:

[0209]

[0210] wherein R1 is a measured distance of radar 1 to a point in space, R2 is a distance between radar 2 (which can be regarded as a second detection device) and the point, and d is a distance between radar 1 and radar 2.

[0211] Exemplarily, the translation matrix M i satisfies the following equation:

[0212]

[0213] wherein d is a translation distance of the first detection device relative to the origin of the first coordinate system, and in the case that the first coordinate system is the coordinate system of the second detection device, d is a distance between the first detection device and the second detection device.

[0214] As an example of speed conversion, the aligned speed information of the detection point in the aligned data is related to the speed information of the detection point in the reported data, and the installation position of the detection device corresponding to the aligned data. Exemplarily, the converted distance information V i ′ of the detection point in the aligned data satisfies the following equation: i

[0215] V i ′=V i *cosθ1.

[0216] As mentioned above, the distance information and speed information corresponding to multiple detection points belonging to the same associated point group are matched. After the data processing device aligns multiple reported data to the same coordinate system, the distance information and speed information can be matched to determine multiple detection points with matched distance and speed.

[0217] ​Exemplarily, the associated point group comprises a first detection point and a second detection point, and the first detection point and the second detection point are derived from different report data. The aligned distance information of the detection point comprises an aligned distance bin index, and the aligned speed information of the detection point comprises an aligned speed bin index. The distance information and the speed information of the first detection point and the second detection point match in at least the following two conditions: Condition 1, the aligned distance bin index of the first detection point is the same as the aligned distance bin index of the second detection point, or the difference between the aligned distance bin index of the first detection point and the aligned distance bin index of the second detection point is less than a first distance threshold. Condition 2, the aligned speed bin index of the first detection point is the same as the aligned speed bin index of the second detection point, or the difference between the aligned speed bin index of the first detection point and the aligned speed bin index of the second detection point is less than a first speed threshold. For example, in combination with Figure 8 and Figure 11 The distance bin index (or distance) of the detection point L1 is RL1, and the speed bin index is VL1. The distance bin index of the detection point M1 obtained by the radar 2 detecting the target T1 is RM1, and the speed bin index is VM1. After position alignment, the detection point L1 is converted into L1', and the distance bin index of L1' is RL1', and the speed bin index is VL1'. If RL1' and RM1 are the same or the difference is less than the first distance threshold, and VL1' and VM1 are the same or the difference is less than the first speed threshold, the distance information and the speed information of the detection point L1 and the detection point M1 match.

[0218] The above is an example of representing distance information and speed information in the form of distance bin index and speed bin index based on the report data. In some cases, distance information and speed information can be represented as specific data values. In this case, the data processing apparatus can also match the detection points based on the specific data values.

[0219] Exemplarily, the associated point group comprises a first detection point and a second detection point, and the first detection point and the second detection point are derived from different report data. The aligned distance information of the detection point is used to indicate the aligned distance of the detection point, and the aligned speed information of the detection point is used to indicate the aligned speed of the detection point. The distance information and the speed information of the first detection point and the second detection point match in at least the following two conditions: Condition 1, the difference between the aligned distance of the first detection point and the aligned distance of the second detection point is less than or equal to a second distance threshold. Condition 2, the difference between the aligned speed of the first detection point and the aligned speed of the second detection point is less than or equal to a second speed threshold.

[0220] In a possible implementation, the data processing apparatus matches the speed information and the distance information of the detection points in the multiple pieces of alignment data to obtain a candidate associated point group in which the distance and the speed are matched. The matching condition can refer to the two examples described above. Alternatively, the candidate associated point group can be directly used as the associated point group, or the candidate associated point group needs to be further screened to obtain the associated point group. For example, the data processing apparatus matches the speed information and the distance information of the detection points in the multiple pieces of alignment data to obtain a candidate associated point group in which the distance and the speed are matched. The data processing apparatus performs channel consistency detection on the channel data of the detection points in the candidate associated point group, removes the detection points that do not satisfy the channel consistency condition, and obtains the associated point group. For example, the candidate associated point group 1 includes the detection point 1, the detection point 2, and the detection point 3, and the three detection points come from different reporting data. The channel data of the detection point 1 and the detection point 2 satisfy the consistency condition, and the detection point 3 does not satisfy the consistency condition with the detection point 1 and the detection point 2. Therefore, the detection point 3 is removed. In some cases, there can be a case where the detection point 3 does not satisfy the consistency condition with the detection point 1, but satisfies the consistency condition with the detection point 2. In this case, it can be pre-designed whether to retain the detection point 3.

[0221] In some cases, more or fewer matching conditions can be designed for the associated detection points, which are not described one by one here.

[0222] In S603, the data processing apparatus determines at least one angle information based on the channel data corresponding to the detection points in the first associated point group.

[0223] The first associated point group belongs to at least one associated point group. Here, the first associated point group is taken as an example for description. In some cases, the data processing apparatus can perform angle measurement processing on each associated point group to obtain one or more angle information corresponding to each associated point group. That is, the data processing apparatus can determine one or more angle information based on the channel data corresponding to the multiple detection points in each associated point group.

[0224] The data reported by the detection apparatus includes channel data corresponding to the detection points, which reflects the amplitude and phase data of multiple channels of the detection apparatus at the detection points. The channel data corresponding to the detection points can be used to detect one or more angle information corresponding to the detection points, so as to distinguish multiple targets with the same distance and the same speed. On this basis, the data processing apparatus determines the associated detection points of the multiple pieces of reporting data, and performs angle measurement on the channel data of the associated detection points to obtain the angle information (such as the angle value or the angle value range) of the target. This is equivalent to combining the channels of multiple detection apparatuses, which can multiply the number of channels and significantly improve the angle resolution capability of the multi-detection apparatus system.

[0225] In one possible implementation, the data processing apparatus obtains multiple frequency domain data respectively based on the channel data corresponding to the multiple detection points in the first group of associated points, obtains summary data based on the multiple frequency domain data, and performs spectrum analysis based on the summary data to obtain at least one angle information. The above implementation gives an exemplary joint angle measurement process. The data processing apparatus converts the channel data to the frequency domain, and aggregates the frequency domain data, and obtains one or more angle values (or angle value ranges) using the aggregated data.

[0226] Of course, the above angle measurement process is only an example. In some possible implementations, the angle measurement algorithm using the channel data of the associated points can also use existing angle measurement algorithms, such as fast iterative interpolated beamforming (FIIB), multiple signal classification (MUSIC) algorithm, estimating signal parameters via rotational invariance techniques (ESPRIT) algorithm, Capon algorithm (a kind of statistical algorithm), deterministic maximum likelihood (DML) algorithm, RELAX algorithm, etc., but the input of the angle measurement algorithm is the channel data of the associated points from multiple detection apparatuses.

[0227] In some possible implementations, the data processing apparatus can generate joint point cloud data, and the joint point cloud data includes information of multiple points, and the information of each point includes velocity information of the point, distance information of the point, and angle information of the point. Wherein, the velocity of the point in the point cloud data is obtained by the velocity information of the detection point in the multiple pieces of reported data, the distance information of the point is obtained by the distance information in the multiple pieces of reported data, and the angle information of the point is related to the at least one angle information. The point cloud data reflects the coordinates of the point and the related information of the point, and can facilitate target identification, fusion perception and other processing of the back end, and the point cloud data has high ranging, velocity measurement and angle measurement accuracy, high value and good usability.

[0228] See Figure 12 , Figure 12is a schematic diagram of joint point cloud data provided by an embodiment of the present application. The joint point cloud data includes points G1, G2 and G3, each of which has corresponding distance information, speed information and angle information, which is calculated based on channel data of an associated point group. For example, taking an associated point group of L2 and M2 as an example, two angle information can be calculated, which can be used to obtain the angle information of points G2 and G3. Similarly, taking an associated point group of L1 and M1 as an example, one angle information can be calculated and used to obtain (e.g., directly as) the angle information of point G2.

[0229] In some cases, the point cloud data can be output as a detection result. In other cases, the data processing apparatus outputs the result of target recognition as a detection result based on the joint point cloud data.

[0230] In some possible solutions, the coordinate system of the joint point cloud data is a relative coordinate system, for example, the coordinate system of the second detection device. In other possible solutions, the coordinate system of the joint point cloud data is an absolute coordinate system, for example, the coordinate system of the ground.

[0231] In the latter case, the absolute pose information of some or all of the detection devices, such as absolute position and absolute orientation angle, is also used in the process of generating the joint point cloud data. For example, the data processing apparatus first obtains a relative fusion point cloud data based on the multiple reported data and the angle value obtained by joint angle measurement. The coordinate system of the relative fusion point cloud data is the coordinate system of the second detection device. The data processing apparatus converts the coordinate system of the relative fusion point cloud data to the absolute coordinate system using the absolute pose information of the second detection device, and obtains the joint point cloud data.

[0232] Optionally, the absolute pose information includes an absolute orientation angle. The absolute orientation angle can be determined by the following method: the data processing apparatus determines the absolute orientation angle of the first detection device based on the calibrated orientation angle of at least one detection device, the pose information of the terminal, the joint point cloud data and the independent point cloud data of the first detection device, the first detection device belonging to the at least one detection device. The independent point cloud data of the first detection device is the point cloud data obtained based on the reported data of the first detection device. For example, the pose information of the terminal includes one or more of the position, heading angle, roll angle, pitch angle, etc.

[0233] Optionally, in some cases, the result of target recognition can also be used in the process of determining the absolute orientation angle of the detection device.

[0234] In Figure 6In the embodiment shown, the data processing device can jointly measure the angle of the associated channels in multiple detection devices, greatly improving the angle measurement capability of the multiple detection device system, significantly improving the detection accuracy of the detection system, and significantly improving the confidence of the detection results.

[0235] The foregoing solutions introduce a variety of possible implementations, and the following introduces two possible implementation designs.

[0236] In a possible implementation design, the multiple detection devices in the detection system are radars. The detection system can perform the following steps:

[0237] Step 11, the multiple radars transmit detection signals and receive echo signals, and process the ADC data based on the echo signals.

[0238] Step 12, the multiple radars respectively perform 2D FFT and CFAR processing based on the ADC data to obtain RV spectrum data and channel data. The RV spectrum data includes velocity information and distance information of the detection points, but does not include angle information. The channel data includes channel data corresponding to the detection points.

[0239] Step 13, the multiple radars send the RV spectrum data and the channel data to the data processing device (or radar host). Correspondingly, the data processing device receives the RV spectrum data and the channel data reported by the multiple radars.

[0240] Optionally, the multiple radars and the data processing device are connected through Ethernet. Of course, the present application is also applicable to the case of using other communication modes.

[0241] Step 14, the data processing device performs joint calibration using the data reported by the multiple radars to obtain the calibrated orientation angle of the radars.

[0242] In a possible implementation, the data processing device performs position conversion on the data reported by the multiple radars to obtain RV spectrum data and channel data consistent with the reference coordinate system, i.e., multiple calibration data. The data processing device performs distance and velocity matching based on the multiple calibration data to obtain a calibrated associated point group and a calibrated management point group, wherein the calibrated management point group includes detection points with matched distance and velocity.

[0243] For the channel data in the matched at least one calibration associated point group, the data processing apparatus compensates the angle measurement deviation introduced by the installation position and angle of the radar based on the installation position and angle of the radar. The data processing apparatus performs global channel correlation matching on the channel data (optionally, the data after sin domain conversion) of multiple radars in an angle scanning matching manner, and takes the angle with the global maximum channel correlation as the relative calibration result, such as the calibration orientation angle of the radar and / or the calibration deflection angle of the radar. For example, the channel data of radar 1 and radar 2 are matched to obtain the calibration orientation angle (or the calibration deflection angle) of radar 1 relative to radar 2. For another example, the channel data of radar 3 and radar 2 are matched to obtain the calibration orientation angle (or the calibration deflection angle) of radar 3. Similarly, the channel data of radar 1, radar 2 and radar 2 are matched to obtain one or more of the calibration orientation angle (or the calibration deflection angle) of radar 1, the calibration orientation angle (or the calibration deflection angle) of radar 2 and the calibration orientation angle (or the calibration deflection angle) of radar 3.

[0244] Further, in some schemes, based on the relative calibration result and the attitude information of the terminal (such as a vehicle), the data processing apparatus further utilizes the multi-radar channel data, the newly added point cloud data (i.e., the joint point cloud data), the original point cloud (i.e., the independent point cloud data of the radar) and the target (i.e., the record after target recognition) to perform joint calibration, and obtains the absolute orientation angle of the radar. The absolute orientation angle can be taken as the result of the joint calibration of the radar.

[0245] It should be understood that there can be multiple groups of joint point cloud data, original point cloud data and targets, for example, multiple groups of joint point cloud data, original point cloud data and targets in a period of time.

[0246] Optionally, step S14 is not necessarily performed every time this embodiment is implemented. For example, step S14 is performed periodically or aperiodically.

[0247] In step 15, the data processing apparatus performs channel-level association on the data reported by multiple radars.

[0248] Specifically, the data processing apparatus converts the RV spectrum data into a first reference system, such as a vehicle body coordinate or a coordinate of radar 2. The data processing apparatus performs matching on the converted RV spectrum data to obtain an alternative associated point group, and the speed information and distance information of the detection points in the alternative associated point group are matched. The data processing apparatus performs channel consistency test on the alternative associated point group, removes the associated points with poor channel consistency, and obtains an associated point group.

[0249] In step 16, the data processing apparatus utilizes the channel data of the associated detection points in the associated point group to perform joint angle measurement, and obtains a point cloud angle value.

[0250] Exemplarily, the data processing apparatus adds the frequency amplitudes of the channel data of the associated detection points, and obtains the point cloud angle value with higher confidence and angle resolution based on peak angle measurement.

[0251] Step 17, the data processing apparatus outputs the joint point cloud data.

[0252] The joint point cloud data includes the distance information, velocity information and point cloud angle value of the points.

[0253] In the above embodiment, the data processing apparatus combines the channel data with high channel consistency from multiple radars, performs joint angle measurement, and can improve the angle measurement resolution of the detection system in the horizontal and elevation angles. In addition, joint angle measurement of data from multiple radars can detect the object space from multiple angles, reduce the impact of multipath in the horizontal and elevation directions, and improve the point cloud confidence.

[0254] In addition, using joint online calibration to determine the position information of the detection device can allow the calibration scene of the radar to be not limited by the type of moving target in the environment, and can speed up the calibration convergence speed.

[0255] In a possible implementation design, the multiple detection devices in the detection system are FMCW lidars. The detection system can perform the following steps:

[0256] Step 21, the multiple FMCW lidars emit detection signals (i.e., FMCW beams) and receive echo signals, and process the ADC data based on the echo signals.

[0257] Step 22, the multiple FMCW lidars respectively perform 2D FFT and CFAR processing based on the ADC data to obtain RV spectrum data and channel data. The RV spectrum data includes the velocity information and distance information of the detection points, but does not include the angle information. The channel data includes the channel data corresponding to the detection points.

[0258] Step 23, the multiple FMCW lidars send the RV spectrum data and the channel data to the data processing apparatus (or radar host). Correspondingly, the data processing apparatus receives the RV spectrum data and the channel data reported by the multiple FMCW lidars.

[0259] Optionally, the multiple FMCW lidars are connected with the data processing apparatus through Ethernet. Of course, the present application is also applicable to the case of using other communication modes.

[0260] Step 24, the data processing utilizes the data reported by the multiple FMCW lidars to perform joint calibration, and obtains the calibrated orientation angle of the FMCW lidar.

[0261] Optionally, step S24 is not necessarily performed each time the embodiment is implemented. For example, step S24 is performed periodically or aperiodically.

[0262] At step 25, the data processing apparatus performs channel-level association on the data reported by the plurality of radars.

[0263] At step 26, the data processing apparatus performs joint angle measurement using the channel data of the associated detection points in the association point group to obtain point cloud angle values.

[0264] At step 27, the data processing apparatus outputs the joint point cloud data.

[0265] The embodiment of the application can also refer to the descriptions of steps 11 to 17.

[0266] The above describes the method of the embodiment of the application in detail. The apparatus of the embodiment of the application is provided below.

[0267] It should be understood that the apparatus provided in the embodiment of the application is logically divided into units, and all or part of the units can be integrated into a physical entity, or can be physically separated. In addition, the units in the apparatus can be implemented in the form of processor calling software. For example, the apparatus includes a processor, the processor is connected with a memory, the memory stores instructions, and the processor calls the instructions stored in the memory to implement any one of the above methods or to implement the functions of the units of the apparatus, wherein the processor is a general processor such as a CPU or an MPU, and the memory is an internal memory of the apparatus or an external memory of the apparatus.

[0268] Alternatively, the units in the apparatus can be implemented in the form of hardware circuit, and the functions of part or all of the units can be implemented by designing the hardware circuit, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of part or all of the units are implemented by designing the logical relationship of elements in the circuit. For another example, in another implementation, the hardware circuit is a PLD, and taking an FPGA as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by a configuration file, so as to implement the functions of part or all of the units.

[0269] In the embodiment of the application, each unit in the apparatus can be one or more processors (or processing circuits) configured to implement the above method, such as a CPU, a GPU, an NPU, a TPU, a DPU, an MPU, a digital signal processor (DSP), an ASIC, an FPGA, or a combination of at least two of these processor forms.

[0270] Furthermore, all or some of the units in the above apparatus can be integrated or can be independent. In one implementation, these units are integrated to be in the form of an SOC. The SOC can include at least one processor for implementing the functions of the above method or the units of the apparatus. The at least one processor can be of different types, such as including a CPU and an FPGA, or including a CPU and a MCU, or including a CPU and a GPU, and the like. The following lists several possible apparatuses.

[0271] See Figure 13 , Figure 13 is a structural schematic diagram of a data processing apparatus provided by an embodiment of the present application. Optionally, the data processing apparatus 130 can be a stand-alone device, such as a vehicle, a computing device (for example, a DC), and the like. Alternatively, the data processing apparatus 130 can also be a component in a stand-alone device (such as a vehicle, a DC), for example, a chip or an integrated circuit, and the like. The data processing apparatus 130 is configured to implement the foregoing data processing method, for example, the data processing method shown in Figure 6 and possible implementation manners thereof.

[0272] Exemplarily, the data processing apparatus 130 includes an acquisition unit 1301 and a processing unit 1302. The acquisition unit 1301 is configured to implement one or more operations of receiving, acquiring, and the like, and the processing unit 1302 is configured to implement one or more operations of processing, determining, generating, calculating, aligning, correlating, and the like.

[0273] In a possible implementation manner, the acquisition unit 1301 is configured to acquire a plurality of pieces of reported data from a plurality of detection apparatuses, and the processing unit 1302 is configured to determine at least one associated point group based on pose information of at least one detection apparatus, and determine at least one angle information based on channel data corresponding to a detection point in the first associated point group. For specific actions performed by the above units, refer to the descriptions in the foregoing embodiments.

[0274] In yet another possible implementation manner, the plurality of pieces of reported data include first reported data and second reported data, and the first associated point group includes a first detection point derived from the first reported data and a second detection point derived from the second reported data. The channel data of the first detection point and the channel data of the second detection point satisfy a channel consistency condition, including: a first correlation angle is greater than or equal to a first threshold value, and the first correlation angle is related to the channel data of the first detection point and the channel data of the second detection point. When an orientation angle of the first detection apparatus is a calibrated orientation angle, the channel data of the detection point in the first reported data and the channel data of the detection point in the second reported data have maximum correlation.

[0275] In yet another possible implementation, the processing unit 1302 is further configured to align the reference systems of the first reported data and the second reported data to obtain first calibrated data and second calibrated data, and determine a calibrated orientation angle of the first detection device according to the channel data of the detection points in the first calibrated data and the channel data of the detection points in the second calibrated data.

[0276] In yet another possible implementation, the processing unit 1302 is further configured to compensate the channel data of the detection points in the first calibrated data by using the installation position of the first detection device and the installation orientation angle of the first detection device, and determine a calibrated deflection angle of the first detection device according to the channel data of the detection points in the compensated first calibrated data and the channel data of the detection points in the second calibrated data. In some cases, the calibrated orientation angle of the first detection device is related to the installation orientation angle of the first detection device and the calibrated deflection angle of the first detection device.

[0277] In yet another possible implementation, the processing unit 1302 is further configured to determine at least one calibrated correlation point group based on the first calibrated data and the second calibrated data, each calibrated correlation point group including distance information and a plurality of detection points matching the distance information, and each calibrated correlation point group including detection points in the first calibrated data and detection points in the second calibrated data. The processing unit 1302 is further configured to compensate the channel data of the detection points from the first reported data in the at least one calibrated correlation point group by using the installation position of the first detection device and the installation orientation angle of the first detection device.

[0278] In a possible implementation, the processing unit 1302 is further configured to align the reference systems of the first reported data and the second reported data to obtain first calibrated data and second calibrated data, and compensate the channel data of the detection points in the first calibrated data by using the installation position of the first detection device. The processing unit 1302 is further configured to determine a calibrated orientation angle of the first detection device according to the channel data of the detection points in the compensated first calibrated data and the channel data of the detection points in the second calibrated data.

[0279] In a possible implementation, the processing unit 1302 is further configured to align the reference systems of the plurality of reported data based on the pose information of the at least one detection device to obtain a plurality of aligned data. Each aligned data corresponds to one reported data, and each aligned data includes aligned distance information of the detection points and aligned velocity information of the detection points. The plurality of aligned data are aligned with reference to the first coordinate system. The processing unit 1302 is further configured to perform detection point correlation based on at least the plurality of aligned data to obtain at least one correlation point group.

[0280] In a possible implementation, the multiple pieces of reported data include first reported data, and the first reported data is from a first probe device of the multiple probe devices. The processing unit 1302 is further configured to perform position alignment on the first reported data by using the pose information of the first probe device to obtain first aligned data. The first aligned data includes aligned distance information of the detection points and aligned velocity information of the detection points.

[0281] In a possible implementation, the processing unit 1302 is further configured to perform position alignment on the first reported data by using the pose information of the first probe device to obtain first aligned data, and the first aligned data includes aligned distance information of the detection points and aligned velocity information of the detection points. The processing unit 1302 is further configured to perform detection point association based at least on the second aligned data and the first aligned data to obtain at least one associated point group.

[0282] In a possible implementation, the processing unit 1302 is further configured to match the velocity information and the distance information of the detection points in the multiple pieces of aligned data to obtain an alternative associated point group in which the distance and the velocity are matched. The alternative associated point group can be directly used as the associated point group, or the alternative associated point group needs to be further screened to obtain the associated point group. For example, in the latter case, the processing unit 1302 is further configured to perform channel consistency verification on the detection points in the alternative associated point group, screen the detection points in the alternative associated point group that meet the channel consistency condition, and obtain at least one associated point group.

[0283] In a possible implementation, the processing unit 1302 is further configured to obtain multiple frequency domain data based on the channel data corresponding to the multiple detection points in the first associated point group respectively, obtain summary data based on the multiple frequency domain data, and obtain at least one angle information based on the summary data.

[0284] In a possible implementation, the processing unit 1302 is further configured to generate joint point cloud data, and the joint point cloud data includes information of multiple points. The information of each point includes velocity information of the point, distance information of the point, and angle information of the point. The angle information of the point is related to the at least one angle information.

[0285] In a possible implementation, the processing unit 1302 is further configured to generate the joint point cloud data based on the absolute pose information of the at least one probe device, and a reference system of the joint point cloud data is an absolute coordinate system.

[0286] In a possible implementation, the processing unit 1302 is further configured to determine an absolute orientation angle of a first detection device based on the calibrated orientation angle of the at least one detection device, the pose information of the terminal, the joint point cloud data, and independent point cloud data of the first detection device, the first detection device being the at least one detection device, and the independent point cloud data of the first detection device being point cloud data obtained based on reported data of the first detection device.

[0287] Figure 13 The specific operations performed by the data processing apparatus shown can also refer to the descriptions in the foregoing embodiments.

[0288] Please refer to Figure 14 , Figure 14 is a structural schematic diagram of another data processing apparatus provided by an embodiment of the present application, as Figure 14 The data processing apparatus 140 can be a standalone device, for example, a vehicle, a computing device (for example, a DC), or the like. Alternatively, the data processing apparatus 140 can also be a component in a standalone device (for example, a vehicle, a DC), such as a chip or an integrated circuit, or the like. The data processing apparatus 140 is configured to implement the foregoing data processing method, for example, the data processing method shown in Figure 6 and possible implementations thereof.

[0289] The data processing apparatus 140 can include at least one processor 1401 and a memory 1403. Optionally, it can also include a communication interface 1402. Further optionally, it can also include a connection line 1404, wherein the processor 1401, the communication interface 1402, and / or the memory 1403 are connected by the connection line 1404, and / or communicate with each other through the connection line 1404 to transfer control signals and / or data signals.

[0290] Wherein:

[0291] The processor 1401 is a module for performing arithmetic operations and / or logical operations, and can specifically include one or more of the following modules: a CPU, an application processor (AP), an MCU, an ECU, a GPU, an MPU, an ASIC, an image signal processor (ISP), a DSP, an FPGA, a complex programmable logic device (CPLD), or a co-processor, and the like.

[0292] The communication interface 1402 can be configured to provide information input or output for at least one processor, or to receive a signal transmitted from outside and / or transmit a signal to the outside. For example, the communication interface 1402 can include an interface circuit. For example, the communication interface 1402 can include a wired link interface such as an Ethernet cable, and can also be a wireless link (Wi-Fi, Bluetooth, universal wireless transmission, vehicle-mounted short-range communication technology, and other short-range wireless communication technologies) interface. Optionally, the communication interface 1402 can also include a radio frequency transmitter, an antenna, and the like. In the case where the communication interface 1402 includes an antenna, the number of antennas can be one or more.

[0293] As a possible design, if the data processing apparatus 140 is a stand-alone device, the communication interface 1402 can include a receiver and a transmitter. The receiver and the transmitter can be the same component, or different components. When the receiver and the transmitter are the same component, the component can be referred to as a transceiver.

[0294] As another possible design, if the data processing apparatus 140 is a chip or a circuit, the communication interface 1402 can include an input interface and an output interface, which can be the same interface, or can be different interfaces.

[0295] Optionally, the functions of the communication interface 1402 can be implemented by a transceiver circuit or a dedicated chip of the transceiver.

[0296] The memory 1403 is configured to provide a storage space, in which data such as an operating system and a computer program can be stored. The memory 1403 can be one or a combination of a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a compact disc read-only memory (CD-ROM), and the like.

[0297] It should be noted that the functions and actions of the modules or units in the above-mentioned data processing apparatus 140 are only exemplary.

[0298] The functional units in the data processing apparatus 140 can be used to implement the above-mentioned data processing method, for example Figure 6 The data processing method and its possible implementation.

[0299] Optionally, the processor 1401 can be a processor specially used for executing the foregoing method (conveniently distinguished as a special-purpose processor), or a processor for executing the foregoing method by calling a computer program (conveniently distinguished as a special-purpose processor). Optionally, the at least one processor can include both a special-purpose processor and a general-purpose processor.

[0300] Optionally, in the case where the data processing apparatus 140 includes at least one memory 1403, if the processor 1401 implements the foregoing data processing method by calling a computer program, the computer program can be stored in the memory 1403.

[0301] The embodiments of the present application also provide a chip, which includes a logic circuit and a communication interface. The communication interface is used for receiving and / or sending information, or for inputting and / or outputting information. The logic circuit is used for processing information. The chip is used for implementing the foregoing data processing method, for example Figure 6 the data processing method and possible implementation manners thereof.

[0302] The embodiments of the present application also provide a computer readable storage medium, which stores instructions. When the instructions are run on at least one processor (or data processing apparatus), the foregoing data processing method is implemented, for example Figure 6 the data processing method and possible implementation manners thereof.

[0303] The embodiments of the present application also provide a computer program product, which includes computer instructions. The computer instructions are used for implementing the foregoing data processing method, for example Figure 6 the data processing method and possible implementation manners thereof.

[0304] In addition, the following several points need to be additionally explained:

[0305] I. The foregoing embodiments are only used to illustrate the technical solutions of the present application, rather than limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the protection scope of the technical solutions of the embodiments of the present application.

[0306] II. Unless otherwise specified, the meaning of "multiple" is two or more.

[0307] III. If there is no special description and no logical conflict, the terms and / or descriptions of different embodiments of the present application are consistent and can be mutually referenced. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0308] Fourth, various digital numbers involved in the present application are only used for distinguishing, and do not limit the protection scope of the present application. The size of the serial numbers involved in the present application does not mean the execution order, and the execution order of each process should be determined according to its function and inherent logic. For example, the terms "first", "second", "third", "fourth" and other various terms in the specification and claims of the present application and the drawings (if any) are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. Among them, the data thus used can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0309] Meanwhile, any embodiment or design scheme described as "exemplary" or "for example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the use of "exemplary" or "for example" and the like is intended to present the relevant concept in a specific manner for ease of understanding.

[0310] Fifth, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or modules does not have to be limited to the steps or modules clearly listed, but can include other steps or modules not clearly listed or inherent to these processes, methods, products or devices.

[0311] Sixth, in the present application, "for indicating" can be understood as "enabling", and "enabling" can include direct enabling and indirect enabling. When describing that a certain information is used to enable A, it can include that the information directly enables A or indirectly enables A, and it does not mean that A must be carried in the information.

[0312] The information enabled by the information is called to-be-enabled information, and there are many ways to enable the to-be-enabled information in the implementation process, for example, but not limited to, the to-be-enabled information can be directly enabled, such as the to-be-enabled information itself or the index of the to-be-enabled information. The to-be-enabled information can also be indirectly enabled by enabling other information, where the other information and the to-be-enabled information have an association relationship. The to-be-enabled information can also be enabled only in part, and the other part of the to-be-enabled information is known or agreed in advance. For example, the enabling of a specific information can also be realized by means of the arrangement order of each information agreed in advance (such as a protocol), thereby reducing the enabling cost to a certain extent. Meanwhile, the common part of each information can be identified and uniformly enabled to reduce the enabling cost caused by separately enabling the same information.

[0313] Seven, in the present application, "predefined" can include pre-configuration, for example, a certain information is predefined, which means that a certain information is calculated or received in advance before performing an action using a certain information. Wherein, "predefined" can be realized by pre-storing corresponding code, table or other means for indicating related information in the device (for example, controller or vehicle), and the specific implementation manner is not limited in the present application.

[0314] Eight, the "storage" or "save" in the present application can mean saving in one or more memories. One or more memories can be separately arranged or integrated in the encoder or decoder, processor or communication device. One or more memories can be partially separately arranged and partially integrated in the decoder, processor or communication device. The type of memory can be any form of storage medium, which is not limited.

[0315] Nine, the arrows or blocks shown by dashed lines in the schematic diagram of the drawing part of the present application specification represent optional steps or optional modules.

[0316] Ten, unless otherwise specified, " / " represents that the objects before and after are in an "or" relationship, for example, A / B can represent A or B; "and / or" in the present application is only a description of the relationship between the associated objects, which means that there can be three kinds of relationships, for example, A and / or B, which can represent three cases: A exists alone, A and B exist together, and B exists alone, wherein A and B can be singular or plural.

[0317] Eleven, unless otherwise specified, the names of devices, systems, modules and other information in the embodiments of the present application are only examples, and the devices, systems and modules are used to represent the possible bodies of realizing a certain function, and the meanings among them can be replaced with each other.

Claims

1. A data processing method, characterized in that, The method includes: Multiple reported data from multiple detection devices are acquired. Each reported data is data obtained by the corresponding detection device using detection signals to detect the object space. Each reported data includes distance information of the detection point, speed information of the detection point, and channel data corresponding to the detection point. At least one associated point group is determined based on the pose information of at least one detection device; wherein, the at least one detection device belongs to the plurality of detection devices, each associated point group includes a plurality of associated detection points, the plurality of detection points in each associated point group are derived from at least two sets of reported data, and the channel data corresponding to the plurality of detection points in each associated point group satisfies the channel consistency condition. At least one angle information is determined based on the channel data corresponding to multiple detection points in the first associated point group. The at least one angle information is used to indicate the angle of the target in the object space. The first associated point group belongs to the at least one associated point group.

2. The method according to claim 1, characterized in that, The multiple reported data include first reported data and second reported data, and the first associated point group includes a first detection point derived from the first reported data and a second detection point derived from the second reported data. The channel data at the first detection point and the channel data at the second detection point meet the channel consistency conditions, including: The first correlation angle is greater than or equal to the first threshold, and the first correlation angle is correlated with the channel data of the first detection point and the channel data of the second detection point.

3. The method according to claim 1 or 2, characterized in that, The plurality of detection devices include a first detection device and a second detection device, the plurality of reported data include first reported data from the first detection device and second reported data from the second detection device, and the pose information of the first detection device includes the calibrated orientation angle of the first detection device; The method further includes: Align the reference frames of the first reported data and the second reported data to obtain the first calibration data and the second calibration data; Based on the channel data of the detection points in the first calibration data and the channel data of the detection points in the second calibration data, the calibration orientation angle of the first detection device is determined.

4. The method according to claim 3, characterized in that, The step of using the channel data of the detection points in the first calibration data and the channel data of the detection points in the second calibration data includes: The channel data of the detection points in the first calibration data are compensated by utilizing the installation position and orientation angle of the first detection device. Based on the channel data of the detection points in the compensated first calibration data and the channel data of the detection points in the second calibration data, the calibration deviation angle of the first detection device is obtained. The calibration orientation angle of the first detection device is related to the calibration deflection angle of the first detection device and the installation orientation angle of the first detection device; When the deflection angle of the first detection device is the calibration deflection angle, the channel data of the detection point in the compensated first calibration data has the greatest correlation with the channel data of the second calibration data.

5. The method according to claim 4, characterized in that, The step of compensating for the channel data of the detection points in the first calibration data by utilizing the installation position and orientation angle of the first detection device includes: Based on the first calibration data and the second calibration data, at least one calibration association point group is determined. Each calibration association point group includes distance information and multiple detection points that match the distance information. Each calibration association point group includes detection points in the first calibration data and detection points in the second calibration data. Using the installation position and orientation angle of the first detection device, the channel data corresponding to the detection point from the first reported data in the at least one calibration association point group is compensated; The step of obtaining the calibration angle of the first detection device based on the channel data of the detection points in the compensated first calibration data and the channel data of the detection points in the second calibration data includes: The calibration angle of the first detection device is obtained based on the channel data in the at least one calibration association point group after compensation.

6. The method according to claim 3, characterized in that, The step of using the channel data of the detection points in the first calibration data and the channel data of the detection points in the second calibration data includes: Align the reference frames of the first reported data and the second reported data to obtain the first calibration data and the second calibration data; The channel data of the detection point in the first calibration data is compensated by utilizing the installation position of the first detection device. Based on the channel data of the detection points in the compensated first calibration data and the channel data of the detection points in the second calibration data, the calibration orientation angle of the first detection device is obtained; When the orientation angle of the first detection device is the calibration orientation angle, the channel data of the detection point in the compensated first calibration data has the greatest correlation with the channel data of the second calibration data.

7. The method according to any one of claims 1-6, characterized in that, Determining at least one group of associated points based on the pose information of at least one detection device includes: Based on the pose information of the at least one detection device, the reference system of the multiple reported data is aligned to obtain multiple aligned data, wherein each of the aligned data corresponds to one of the reported data, and each of the aligned data includes the aligned distance information and the aligned velocity information of the detection point. The multiple aligned data are aligned with the first coordinate system. Based on the multiple alignment data, the detection points are associated to obtain at least one group of associated points.

8. The method according to claim 7, characterized in that, The plurality of detection devices includes a first detection device and a second detection device; The at least one detection device includes the first detection device, and the first coordinate system is the coordinate system of the second detection device.

9. The method according to claim 7, characterized in that, The first coordinate system is different from the coordinate systems of the plurality of detection devices.

10. The method according to any one of claims 7-9, characterized in that, The at least one detection device includes a first detection device, the plurality of reported data includes first reported data from the first detection device, and the plurality of aligned data includes first aligned data corresponding to the first reported data. The pose information of the first detection device includes the installation position of the first detection device and the attitude information of the first detection device; The aligned distance information of the detection points in the first alignment data is related to the distance information of the detection points in the first reported data, the attitude information of the first detection device, and the installation position of the first detection device. The aligned velocity information of the detection points in the first aligned data is related to the velocity information of the detection points in the first reported data and the installation position of the first detection device.

11. The method according to claim 10, characterized in that, The attitude information of the first detection device includes one or more of the following: the calibration orientation angle of the first detection device, the installation orientation angle of the first detection device, the calibration deflection angle of the first detection device, the installation orientation angle of the first detection device, and the measurement target deflection angle of the first detection device.

12. The method according to any one of claims 7-11, characterized in that, The distance and speed information corresponding to multiple detection points in the associated point group are matched.

13. The method according to claim 12, characterized in that, The multiple reported data include first reported data and second reported data, and the first associated point group includes a first detection point derived from the first reported data and a second detection point derived from the second reported data; The distance information includes a distance bin index, and the speed information includes a speed bin index. The distance information and speed information of the first detection point match those of the second detection point if the following two conditions are met: Condition 1: The distance bin index after the first detection point is aligned is the same as the distance bin index after the second detection point is aligned; or, the difference between the distance bin index after the first detection point is aligned and the distance bin index after the second detection point is aligned is less than the first distance threshold. Condition 2: The velocity bin index after the first detection point is aligned is the same as the velocity bin index after the second detection point is aligned, or the difference between the velocity bin index after the first detection point is aligned and the velocity bin index after the second detection point is aligned is less than the first velocity threshold.

14. The method according to any one of claims 1-13, characterized in that, The determination of at least one angle information based on channel data corresponding to multiple detection points in the first associated point group includes: Based on the channel data corresponding to multiple detection points in the first associated point group, multiple frequency domain data are obtained respectively; The aggregated data is obtained based on the multiple frequency domain data; The at least one angle information is obtained based on the summarized data.

15. The method according to any one of claims 1-14, characterized in that, The method further includes: Generate joint point cloud data, which includes information about multiple points. The information about each point includes the point's velocity information, the point's distance information, and the point's angle information. The point's angle information is related to at least one angle information.

16. The method according to any one of claims 1-15, characterized in that, The multiple detection devices are installed on the terminal, and the generation of joint point cloud data includes: Based on the absolute pose information of the at least one detection device, joint point cloud data is generated, wherein the reference system of the joint point cloud data is an absolute coordinate system.

17. The method according to claim 16, characterized in that, The absolute pose information of the at least one detection device includes the absolute orientation angle of the at least one detection device, and the method further includes: Based on the calibrated orientation angle of the at least one detection device, the pose information of the terminal, the joint point cloud data, and the independent point cloud data of the first detection device, the absolute orientation angle of the first detection device is determined. The first detection device belongs to the at least one detection device, and the independent point cloud data of the first detection device is point cloud data obtained based on the data reported by the first detection device.

18. The method according to any one of claims 1-17, characterized in that, The detection signal is a radio electromagnetic wave. Alternatively, the detection signal may be a frequency-modulated continuous wave (FMCW) laser beam.

19. A detection system, characterized in that, The detection system includes a data processing device and multiple detection devices; The plurality of detection devices are used to provide reported data to the data processing device, and each of the reported data is data obtained by the corresponding detection device using detection signals to detect the object space; The data processing device is used to implement the method according to any one of claims 1-18.

20. A data processing apparatus, characterized in that, The data processing device includes a data acquisition module and a processing module. The data acquisition module is used to transmit data with multiple detection devices, and the processing module is used to process the data. The data processing device is used to implement the method according to any one of claims 1-18.

21. A data processing apparatus, characterized in that, The data processing apparatus includes a memory and at least one processor, the memory being used to store computer instructions, and the at least one processor being used to invoke the computer instructions stored in the memory to implement the method according to any one of claims 1-17.

22. A chip, characterized in that, The chip includes a communication interface and at least one processor, the communication interface being used to input data, and the at least one processor being used to execute computer instructions to implement the method according to any one of claims 1-18.

23. A terminal, characterized in that, The terminal includes the detection system of claim 19, or the data processing device of claim 20 or 21, or the chip of claim 22.

24. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by at least one processor, implement the method as claimed in any one of claims 1-18.

25. A computer program product containing instructions, characterized in that, When the instructions are executed by at least one processor, the method as described in any one of claims 1-18 is implemented.

Images