Dynamic Weight Allocation Method and System, and Information Acquisition Method for Target Object

Abstract

The present application discloses a dynamic weight allocation method and system, and an information acquisition method for a target object. The dynamic weight allocation method includes: obtaining a plurality of first valid points within a preset area of the target object based on a lidar; then obtaining a first target distance mean value and a first variance average deviation; obtaining a plurality of second valid points within the preset area of the target object based on a 4D millimeter-wave radar; then obtaining a second target distance mean value and a second variance average deviation; then obtaining a first difference, a second difference, a third difference, and a fourth difference; then obtaining a first lidar weight and a second lidar weight; then obtaining a first millimeter-wave radar weight and a second millimeter-wave radar weight; and finally obtaining a lidar fusion weight and a millimeter-wave radar fusion weight. This method can achieve adaptive weight adjustment, improve the accuracy of data acquisition, improve the stability of the target recognition system, has low cost, high efficiency, saves resources, and meets the requirements under different complex environments.

Description

Technical Field

[0001] The present disclosure relates to the field of driverless technology, and in particular to a method and system for dynamic weight allocation, and a method for obtaining information of a target object. Background Art

[0002] In the environmental perception of driverless, sensors such as cameras, lidars, and millimeter-wave radars are usually used to separately implement environmental perception tasks. Generally, lidars are used to detect moving or static targets such as obstacles and pedestrians, and point cloud information including three-dimensional spatial coordinates and reflection intensity can be output from the lidar. This type of sensor has high accuracy, high resolution, and a relatively long detection distance, but it is easily affected by the environment and has a high cost; millimeter-wave radars can directly obtain a list of information about obstacles, including obstacle ID categories, three-dimensional position coordinates of obstacles, relative speeds of obstacles, position contrasts, speed contrasts, false alarm probabilities, Doppler effective solutions, and other data; this type of sensor has a long detection distance and strong penetration ability, but the data is unstable.

[0003] In the prior art, the disclosed target detection method based on the feature fusion of lidar and millimeter-wave radar requires different processing and conversion of the scan data of the lidar and 4D millimeter-wave radar to obtain a pseudo-image representation and feature extraction; this involves complex data processing processes and algorithms, greatly increasing the complexity of the system and the difficulty of development; using a convolutional neural network for feature extraction and a feature attention mechanism network for weight adjustment requires a large amount of computing resources and computing power support, which will impose a burden on embedded devices or systems with limited computing resources and limit the application of this technology in resource-constrained environments. Summary of the Invention

[0004] In view of this, the embodiments of the present disclosure provide a method and system for dynamic weight allocation, and a method for obtaining information of a target object, which can realize the fusion of lidar and millimeter-wave data, improve the accuracy of data acquisition, improve the stability of the target recognition system, and have low cost, high efficiency, and resource saving.

[0005] In a first aspect, the embodiments of the present disclosure provide a method for dynamic weight allocation, including:

[0006] Obtaining first point cloud data within a preset area of a target object based on a lidar; the first point cloud data includes a number of first valid points;

[0007] Obtaining a first target distance mean value and a first variance average deviation based on the number of the first valid points;

[0008] Obtaining second point cloud data within a preset area of the target object based on a 4D millimeter-wave radar; the second point cloud data includes a number of second valid points;

[0009] Based on a plurality of the second valid points, obtain the second target distance mean value and the second variance average deviation;

[0010] Based on the first target distance mean value, the first variance average deviation, the second target distance mean value, and the second variance average deviation, obtain a first difference value, a second difference value, a third difference value, and a fourth difference value;

[0011] Based on the first variance average deviation, the second variance average deviation, the first difference value, the second difference value, the third difference value, and the fourth difference value, obtain a first lidar weight and a second lidar weight;

[0012] Based on the first target distance mean value, the second target distance mean value, the first difference value, the second difference value, the third difference value, and the fourth difference value, obtain a first millimeter-wave radar weight and a second millimeter-wave radar weight;

[0013] Based on the first lidar weight and the second lidar weight, obtain a lidar fusion weight;

[0014] Based on the first millimeter-wave radar weight and the second millimeter-wave radar weight, obtain a millimeter-wave radar fusion weight.

[0015] Optionally, the first target distance mean value is l L , and the first variance average deviation is d L ;

[0016]

[0017]

[0018] where R is the number of the first valid points, d(P, C R ) is the distance from any point P in the preset area to the geometric center point C R of the preset area, is the average distance from a plurality of the first valid points in the preset area to the geometric center point of the preset area, is the variance of a plurality of the first valid points in the preset area.

[0019] Optionally, the second target distance mean value is l R , and the second variance average deviation is d R ;

[0020]

[0021]

[0022] where N(R) is the number of the second valid points, di is the distance to the i-th second valid point.

[0023] Optionally, the first difference is D L , the second difference is D R , the third difference is L L , the fourth difference is L R ;

[0024] D L = P1d L + P2d R - d L ;

[0025] D R = P1d L + P2d R - d R ;

[0026] L L = N1l L + N2l R - l L ;

[0027] L R = N1l L + N2l R - l R ;

[0028] 0 ≤ P1 ≤ 1, 0 ≤ P2 ≤ 1 and P1 + P2 = 1;

[0029] 0 ≤ N1 ≤ 1, 0 ≤ N2 ≤ 1 and N1 + N2 = 1.

[0030] Optionally, P1 = P2 = N1 = N2 = 0.5.

[0031] Optionally, the first weight of the lidar is W L1 , the second weight of the lidar is W L2 ;

[0032]

[0033]

[0034] Optionally, the first weight of the millimeter-wave radar is W R1 , the second weight of the millimeter-wave radar is W R2 ;

[0035]

[0036]

[0037] Optionally, the lidar fusion weight is W L , and the millimeter-wave radar weight is W R ;

[0038] W L =W L1 +W L2 ;

[0039] W R =W R1 +W R2 .

[0040] The second aspect of the present application discloses a dynamic weight allocation system, including:

[0041] A first acquisition module configured to acquire first point cloud data within a preset area of a target object based on a lidar; the first point cloud data includes a number of first valid points;

[0042] A second acquisition module configured to acquire a first target distance mean value and a first variance average deviation based on a number of the first valid points;

[0043] A third acquisition module configured to acquire second point cloud data within a preset area of a target object based on a 4D millimeter-wave radar; the second point cloud data includes a number of second valid points;

[0044] A fourth acquisition module configured to acquire a second target distance mean value and a second variance average deviation based on a number of the second valid points;

[0045] A fifth acquisition module configured to obtain a first difference, a second difference, a third difference, and a fourth difference based on the first target distance mean value, the first variance average deviation, the second target distance mean value, and the second variance average deviation;

[0046] A sixth acquisition module configured to obtain a first lidar weight and a second lidar weight based on the first variance average deviation, the second variance average deviation, the first difference, the second difference, the third difference, and the fourth difference;

[0047] A seventh acquisition module configured to obtain a first millimeter-wave radar weight and a second millimeter-wave radar weight based on the first target distance mean value, the second target distance mean value, the first difference, the second difference, the third difference, and the fourth difference;

[0048] An eighth acquisition module configured to obtain a lidar fusion weight based on the first lidar weight and the second lidar weight;

[0049] Obtain the millimeter-wave radar fusion weight based on the first weight of the millimeter-wave radar and the second weight of the millimeter-wave radar.

[0050] The third aspect of this application discloses a method for obtaining information about a target object. Based on the dynamic weight allocation method described above, the method further includes:

[0051] Obtain the target object information according to the lidar fusion weight, the millimeter-wave radar fusion weight, and a preset formula;

[0052] The target object information includes the relative angular velocity α, the relative velocity v, the x coordinate value x of the target object, the y coordinate value y of the target object, and the z coordinate value z of the target object;

[0053] α = α L W L + α R W R ;

[0054] v = v L W L + v R W R ;

[0055]

[0056]

[0057]

[0058]

[0059]

[0060]

[0061]

[0062]

[0063]

[0064] where α L is the relative angular velocity between the lidar and the target object, α R is the relative angular velocity between the millimeter-wave radar and the target object, v L is the relative velocity between the lidar and the target object, v R is the relative velocity between the millimeter-wave radar and the target object, is the maximum value in the x direction of the lidar effective point set, is the maximum value in the x direction of the millimeter-wave radar effective point set, is the minimum value in the x - direction of the valid point set of the lidar, is the minimum value in the x - direction of the valid point set of the millimeter - wave radar, is the maximum value in the y - direction of the valid point set of the lidar, is the maximum value in the y - direction of the valid point set of the millimeter - wave radar, is the minimum value in the y - direction of the valid point set of the lidar, is the minimum value in the y - direction of the valid point set of the millimeter - wave radar, is the maximum value in the z - direction of the valid point set of the lidar, is the maximum value in the z - direction of the valid point set of the millimeter - wave radar, is the minimum value in the z - direction of the valid point set of the lidar, is the minimum value in the z - direction of the valid point set of the millimeter - wave radar.

[0065] In a fourth aspect, embodiments of the present disclosure further provide an electronic device, adopting the following technical solution:

[0066] The electronic device includes:

[0067] At least one processor; and,

[0068] A memory communicatively connected to the at least one processor; wherein,

[0069] The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute any one of the above - mentioned dynamic weight assignment methods.

[0070] In a fourth aspect, embodiments of the present disclosure further provide a computer - readable storage medium, which stores computer instructions for causing a computer to execute any one of the above - mentioned dynamic weight assignment methods.

[0071] The dynamic weight allocation method disclosed in this application can improve the accuracy and reliability of target detection and ranging by simultaneously using the point cloud data obtained from lidar and 4D millimeter-wave radar, giving full play to the advantages of different sensors and enhancing the performance of the perception system. Weight allocation based on multi-dimensional features such as the average target distance, variance average deviation, etc. can more comprehensively consider the spatial distribution characteristics of the target within the preset area, making the weight more representative and accurate. By dynamically adjusting the weights of lidar and millimeter-wave radar according to the weights calculated from the average target distance and variance average deviation, it can adapt to the requirements of target detection in different environments and scenarios, improving the flexibility and robustness of the system. Calculating the weights of lidar and millimeter-wave radar based on multiple sets of differences can reduce the influence of data noise to a certain extent and improve the robustness of the system against different interferences, thereby improving the accuracy and stability of target detection.

[0072] The above description is only an overview of the technical solution of the present disclosure. In order to understand the technical means of the present disclosure more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present disclosure more obvious and understandable, the following specific preferred embodiments are given, and detailed descriptions are provided in conjunction with the accompanying drawings as follows. Brief Description of the Drawings

[0073] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the accompanying drawings required for use in the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0074] Figure 1 It is a flowchart of the dynamic weight allocation method in this application.

[0075] Figure 2 For Figure 1 It is a flowchart of the method for obtaining the first point cloud data.

[0076] Figure 3 For Figure 1 It is a flowchart of the method for obtaining the second point cloud data.

[0077] Figure 4 It is a principle block diagram of the dynamic weight allocation system provided by the embodiments of the present disclosure.

[0078] Figure 5 It is a flowchart of the method for obtaining information of the target object in this application.

[0079] Figure 6 It is a principle block diagram of the information acquisition of the target object provided by the embodiments of the present disclosure.

[0080] Figure 7 A structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. Specific embodiments

[0081] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0082] It should be clear that the following specific examples illustrate the implementation manners of the present disclosure, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific implementation manners, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present disclosure without creative efforts belong to the scope of protection of the present disclosure.

[0083] It should be noted that the following describes various aspects of the embodiments within the scope of the appended claims. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is illustrative only. Based on the present disclosure, those skilled in the art should understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. In addition, this device and / or this method can be implemented using other structures and / or functions in addition to one or more of the aspects described herein.

[0084] It should also be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present disclosure in a schematic manner, and only the components related to the present disclosure are shown in the diagrams, rather than being drawn according to the number, shape and size of the components in actual implementation. The type, quantity and proportion of each component in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0085] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the aspects can be practiced without these specific details.

[0086] Refer to Figure 1 , this application discloses a dynamic weight allocation method, which is a dynamic weight allocation method based on the fusion of lidar and millimeter-wave radar. The method includes the following steps:

[0087] S100, obtain the first point cloud data within the preset area of the target object based on the lidar; wherein, the first point cloud data includes a number of first valid points.

[0088] S200, obtain the first target distance mean value and the first variance average deviation based on a number of first valid points.

[0089] The first target distance mean value is l L , the first variance average deviation is d L ;

[0090]

[0091]

[0092] wherein, R is the number of first valid points, d(P, C R ) is the distance from any point P within the preset area to the geometric center point C of the preset area R of, is the average distance from a number of first valid points within the preset area to the geometric center point of the preset area, is the variance of a number of first valid points within the preset area.

[0093] Such a mean value result can more stably reflect the distance relationship between the target object and the lidar, reduce the influence of individual points, improve the reliability of the distance measurement result, and the mean value of the first target distance can provide the average distance information between the target object and the lidar, which is very important for positioning the position of the target object; through the accurately measured and calculated distance mean value, the position of the target object relative to the lidar can be more accurately grasped, providing a basis for subsequent target tracking and detection.

[0094] S300, obtain the second point cloud data within the preset area of the target object based on the 4D millimeter wave radar; wherein, the second point cloud data includes a number of second valid points.

[0095] S400, obtain the second target distance mean value and the second variance average deviation based on a number of second valid points.

[0096] The second target distance mean value is l R , the second variance average deviation is d R ;

[0097]

[0098]

[0099] wherein, N(R) is the number of second valid points, d iis the distance to the i-th second valid point.

[0100] The millimeter-wave radar can provide distance data different from that of lidar, including better adaptability to target objects of different materials and shapes, and can provide effective target object detection information in these complex environments, thereby enhancing the adaptability and robustness of the system. By obtaining the second target distance mean and the second variance mean deviation, these information can be comprehensively utilized to improve the comprehensive evaluation of the target object distance information.

[0101] S500, based on the first target distance mean, the first variance mean deviation, the second target distance mean, and the second variance mean deviation, obtains the first difference, the second difference, the third difference, and the fourth difference.

[0102] Among them, the first difference (i.e., the difference between the lidar mean deviation and the mean of the variance mean deviations of the two sensors) is D L , the second difference (i.e., the difference between the millimeter-wave radar mean deviation and the mean of the variance mean deviations of the two sensors is D R , the third difference (i.e., the difference between the lidar target distance and the mean of the target distances of the two sensors) is L L , the fourth difference (i.e., the difference between the millimeter-wave radar target distance and the mean of the target distances of the two sensors) is L R .

[0103] D L = P1d L + P2d R - d L ;

[0104] D R = P1d L + P2d R - d R ;

[0105] L L = N1l L + N2l R - l L ;

[0106] L R = N1l L + N2l R - l R .

[0107] In this embodiment, 0 ≤ P1 ≤ 1, 0 ≤ P2 ≤ 1 and P1 + P2 = 1.

[0108] In this embodiment, 0 ≤ N1 ≤ 1, 0 ≤ N2 ≤ 1 and N1 + N2 = 1.

[0109] This solution can provide a more comprehensive description of distance features. These differences can reflect the changes, uncertainties, and stabilities of the target object's distance, thus providing more information in target feature description and analysis, which is beneficial for more accurate identification and classification of the target object; multi-dimensional data fusion helps to comprehensively consider the changes in distance data under different measurement conditions, improving the comprehensiveness and reliability of the distance information of the target object; it can adapt to different application scenarios and provide more general support for target object recognition and tracking. Different combinations of differences can be applied to different types of target objects and environmental conditions, which is beneficial for enhancing the adaptability and generalization ability of the system.

[0110] S600. Based on the first variance mean deviation, the second variance mean deviation, the first difference, the second difference, the third difference, and the fourth difference, obtain the first lidar weight and the second lidar weight.

[0111] In this step, perform normalization processing on the weights.

[0112] Among them, the first lidar weight is W L1 , and the second lidar weight is W L2 .

[0113]

[0114]

[0115] By obtaining weights based on the above multiple parameters, the lidar data can be weighted on the basis of different parameters. Such weight allocation can adjust the influence of lidar data in data fusion according to different distance features and difference situations, making the fused data more in line with the actual application needs. At the same time, it helps to more comprehensively utilize the information brought by lidar data, improving the comprehensive utilization rate and overall effect of the data; in different application environments and target characteristics, the system can be made more flexible and adaptable according to the way of weight adjustment.

[0116] S700. Based on the first target distance mean, the second target distance mean, the first difference, the second difference, the third difference, and the fourth difference, obtain the first millimeter-wave radar weight and the second millimeter-wave radar weight.

[0117] The first millimeter-wave radar weight is W R1 , and the second millimeter-wave radar weight is W R2 , that is, in this embodiment, the first millimeter-wave radar weight W R1 allocated according to the variance mean deviation, and the second millimeter-wave radar weight W R2 allocated according to the target distance.

[0118]

[0119]

[0120] By obtaining the first weight and the second weight of the millimeter-wave radar, data fusion can be made more accurate and reliable. Through weight control, the impact of unstable or unreliable data on the overall fusion result can be reduced, and the reliability of the system's recognition and tracking of targets can be enhanced; by optimizing the application of millimeter-wave radar data in the system according to the weight values obtained from different parameters, the system performance can be improved. The reasonable allocation of weights can enable the system to more accurately locate, track, and predict targets, thereby further improving the overall performance of the system.

[0121] S800, based on the first lidar weight and the second lidar weight, obtains the lidar fusion weight; based on the first millimeter-wave radar weight and the second millimeter-wave radar weight, obtains the millimeter-wave radar fusion weight.

[0122] The lidar fusion weight is W L , and the millimeter-wave radar weight is W R .

[0123] W L = W L1 + W L2 ;

[0124] W R = W R1 + W R2 .

[0125] By obtaining the weights of the lidar and the millimeter-wave radar, the data of the two sensors can be optimally fused. Through reasonable weight allocation, the advantages of each sensor can be fully utilized and its disadvantages can be compensated, thereby improving the quality and accuracy of the overall data fusion; by using the sensor weights, the diverse information of the lidar and the millimeter-wave radar can be comprehensively utilized. This comprehensive use can improve the robustness of target detection and tracking, reduce the false alarm rate, and improve the accuracy of target positioning; by adjusting the fusion weight based on the weights, the fusion result can be made more adaptable to different scenarios according to different environmental conditions and target characteristics. This adaptability can improve the reliability of the system and make it applicable to diverse application scenarios and target objects; by adjusting the weights, the controllability and flexibility of data fusion can be enhanced. According to actual needs, the weight values can be flexibly adjusted to achieve precise control of the relative importance of lidar and millimeter-wave radar data in fusion.

[0126] In summary, the dynamic weight allocation method disclosed in this application can improve the accuracy and reliability of target detection and ranging by simultaneously using the point cloud data obtained from lidar and 4D millimeter-wave radar, giving full play to the advantages of different sensors and enhancing the performance of the perception system: weight allocation based on multi-dimensional features such as the mean target distance, variance average deviation, etc. can more comprehensively consider the spatial distribution characteristics of the target within the preset area, making the weight more representative and accurate; through the weights calculated based on the mean target distance and variance average deviation, the dynamic adjustment of the weights of lidar and millimeter-wave radar is realized, which can adapt to the requirements of target detection in different environments and scenarios, improving the flexibility and robustness of the system; calculating the weights of lidar and millimeter-wave radar based on multiple groups of differences can, to a certain extent, reduce the influence of data noise and improve the robustness of the system to different interferences, thereby improving the accuracy and stability of target detection.

[0127] Referring to Figure 2 , the method for obtaining the first point cloud data includes:

[0128] S110, obtaining the first initial data of the preset area of the target object based on lidar;

[0129] S120, preprocessing the first initial data to obtain the first point cloud data.

[0130] The scanning technology based on lidar can provide high-precision three-dimensional point cloud data, which can more accurately capture the shape and contour of the target object in space; lidar can accurately obtain the data of the target object under various complex environmental conditions, with high reliability and stability; by preprocessing the first initial data, the original point cloud data can be transformed into a form that is easier to process and analyze, providing a better data basis for subsequent algorithms and applications.

[0131] In this embodiment, the preprocessing preferably includes: filtering, denoising, and amplification processing.

[0132] Further preferably, P1 = P2 = N1 = N2 = 0.5, that is, the weight allocation in the average of the variance average deviations of lidar and millimeter-wave radar is 0.5.

[0133] Referring to Figure 3 , the method for obtaining the second point cloud data includes:

[0134] S310, obtaining the second initial data of the preset area of the target object based on millimeter-wave radar;

[0135] S320, preprocessing the second initial data to obtain the second point cloud data.

[0136] Further, electromagnetic wave signals can be transmitted and received by a millimeter-wave radar to obtain reflected signals. The signals are denoised, filtered, and amplified; the processed signals are subjected to a fast Fourier transform (FFT) to obtain frequency-domain information; by setting a threshold, target echoes are identified. The Doppler information is used to estimate the velocity of the target; the distance of the target is estimated through the time delay of the signal. Corresponding point cloud data is generated according to the estimated parameters; the point cloud is clustered to distinguish different targets.

[0137] Millimeter-wave radars can operate under different weather conditions, including environments such as rain, snow, fog, and strong light, which gives them an advantage in outdoor applications because they can provide reliable data without being restricted by the external environment; compared with optical sensors, millimeter-wave radars have better penetration in turbid environments and can penetrate haze, dust, and smoke, thus providing a more reliable data acquisition ability; millimeter-wave radars do not rely on the optical contrast of the target surface, so they can also provide stable detection results in cases where the surface material and color change greatly;: by using the velocity change of an object relative to the radar, millimeter-wave radars can provide the velocity information of the target, further enriching the content of the acquired point cloud data and facilitating the tracking and motion analysis of the target; millimeter-wave radars do not require additional light sources, so they can operate in low-light or completely dark environments, which gives them a unique advantage in applications at night or in closed-door environments.

[0138] Further, the solution disclosed in this application realizes the adaptive redundancy of multi-sensor targets. During the operation of the system, the redundancy weights of each sensor are adjusted according to the real-time situation, thereby affecting the accuracy of the fused target; when the target tracking of one or more sensors has problems, the sensor will withdraw from the fusion, not participate in the sensor fusion, and do not trust the targets detected by the sensor. This method can avoid perception errors caused by sensor failures, and thus provide more reliable protection to prevent traffic accidents caused by perception errors and protect people's lives and property safety.

[0139] Referring to Figure 4 , the second aspect of this application discloses a dynamic weight allocation system, including:

[0140] A first acquisition module configured to obtain first point cloud data within a preset area of a target object based on a lidar; the first point cloud data includes a number of first valid points;

[0141] A second acquisition module configured to obtain a first target distance mean and a first variance average deviation based on a number of first valid points;

[0142] A third acquisition module configured to obtain second point cloud data within a preset area of a target object based on a 4D millimeter-wave radar; the second point cloud data includes a number of second valid points;

[0143] A fourth acquisition module, configured to acquire a second target distance mean value and a second variance average deviation based on a plurality of second valid points;

[0144] A fifth acquisition module, configured to obtain a first difference, a second difference, a third difference, and a fourth difference based on a first target distance mean value, a first variance average deviation, a second target distance mean value, and a second variance average deviation;

[0145] A sixth acquisition module, configured to obtain a first lidar weight and a second lidar weight based on a first variance average deviation, a second variance average deviation, a first difference, a second difference, a third difference, and a fourth difference;

[0146] A seventh acquisition module, configured to obtain a first millimeter-wave radar weight and a second millimeter-wave radar weight based on a first target distance mean value, a second target distance mean value, a first difference, a second difference, a third difference, and a fourth difference;

[0147] An eighth acquisition module, configured to obtain a lidar fusion weight based on a first lidar weight and a second lidar weight;

[0148] Based on a first millimeter-wave radar weight and a second millimeter-wave radar weight, obtain a millimeter-wave radar fusion weight.

[0149] Refer to Figure 5 , based on the dynamic weight allocation method disclosed in the first aspect of the present application, the third aspect of the present application discloses a method for acquiring information of a target object, including:

[0150] S100, acquiring first point cloud data within a preset area of a target object based on a lidar; wherein, the first point cloud data includes a plurality of first valid points.

[0151] S200, acquiring a first target distance mean value and a first variance average deviation based on a plurality of first valid points.

[0152] S300, acquiring second point cloud data within a preset area of a target object based on a 4D millimeter-wave radar; wherein, the second point cloud data includes a plurality of second valid points.

[0153] S400, acquiring a second target distance mean value and a second variance average deviation based on a plurality of second valid points.

[0154] S500, obtaining a first difference, a second difference, a third difference, and a fourth difference based on a first target distance mean value, a first variance average deviation, a second target distance mean value, and a second variance average deviation.

[0155] S600. Obtain the first lidar weight and the second lidar weight based on the first variance average deviation, the second variance average deviation, the first difference, the second difference, the third difference, and the fourth difference.

[0156] S700. Obtain the first millimeter-wave radar weight and the second millimeter-wave radar weight based on the first target distance mean, the second target distance mean, the first difference, the second difference, the third difference, and the fourth difference.

[0157] S800. Obtain the lidar fusion weight based on the first lidar weight and the second lidar weight; obtain the millimeter-wave radar fusion weight based on the first millimeter-wave radar weight and the second millimeter-wave radar weight.

[0158] S900. Obtain the target object information according to the lidar fusion weight, the millimeter-wave radar fusion weight, and a preset formula.

[0159] The method for obtaining the information of the target object disclosed in this application combines the data of the lidar and the 4D millimeter-wave radar. By fusing their weights, the advantages of the two sensors can be comprehensively utilized to improve the accuracy and reliability of obtaining the target object information; by obtaining the target distance mean and the variance average deviation, more detailed and accurate target object information can be obtained. It can not only obtain the position of the target, but also understand the size and shape distribution of the target; by obtaining the first difference, the second difference, the third difference, and the fourth difference, a more in-depth analysis of the target object can be carried out to master the changes in its motion and shape, which is helpful for further target recognition and tracking; through the weight fusion of the lidar and the millimeter-wave radar, an appropriate trade-off can be made according to the performance and reliability of the two sensors to obtain a more reasonable fusion weight and improve the comprehensive evaluation ability of the target object information; this solution processes and analyzes the data collected by the sensor and uses a preset formula to obtain the target object information, so it has good real-time performance and stability and is applicable to real-time application scenarios and long-term continuous monitoring.

[0160] Specifically, the target object information includes the relative angular velocity α, the relative velocity v, the x coordinate value x of the target object, the y coordinate value y of the target object, and the z coordinate value z of the target object;

[0161] α = α L W L + α R W R ;

[0162] v = v L W L + v R W R ;

[0163]

[0164]

[0165]

[0166]

[0167]

[0168]

[0169]

[0170]

[0171]

[0172] where α L is the relative angular velocity between the lidar and the target object, and α R is the relative angular velocity between the millimeter-wave radar and the target object, v L is the relative velocity between the lidar and the target object, and v R is the relative velocity between the millimeter-wave radar and the target object, is the maximum value in the x-direction of the lidar effective point set, is the maximum value in the x-direction of the millimeter-wave radar effective point set, is the minimum value in the x-direction of the lidar effective point set, is the minimum value in the x-direction of the millimeter-wave radar effective point set, is the maximum value in the y-direction of the lidar effective point set, is the maximum value in the y-direction of the millimeter-wave radar effective point set, is the minimum value in the y-direction of the lidar effective point set, is the minimum value in the y-direction of the millimeter-wave radar effective point set, is the maximum value in the z-direction of the lidar effective point set, is the maximum value in the z-direction of the millimeter-wave radar effective point set, is the minimum value in the z-direction of the lidar effective point set, is the minimum value in the z-direction of the millimeter-wave radar effective point set.

[0173] The solution disclosed in this application can significantly increase the number and probability of successfully fused targets. The multi-sensor target fusion system can meet the requirements of driverless vehicles for target perception, and can make timely adjustments when some sensors have problems, excluding some sensors to obtain the best perception effect. Compared with directly using static redundant weights for fusion, using dynamic redundant weights can significantly increase the number of frames successfully fused into targets.

[0174] Referring to Figure 6 , the fourth aspect of this application discloses an information acquisition system for target objects, including:

[0175] The first acquisition module is configured to acquire first point cloud data within a preset area of the target object based on a lidar; the first point cloud data includes a number of first valid points;

[0176] The second acquisition module is configured to acquire a first target distance mean value and a first variance average deviation based on a number of first valid points;

[0177] The third acquisition module is configured to acquire second point cloud data within a preset area of the target object based on a 4D millimeter-wave radar; the second point cloud data includes a number of second valid points;

[0178] The fourth acquisition module is configured to acquire a second target distance mean value and a second variance average deviation based on a number of second valid points;

[0179] The fifth acquisition module is configured to obtain a first difference, a second difference, a third difference, and a fourth difference based on the first target distance mean value, the first variance average deviation, the second target distance mean value, and the second variance average deviation;

[0180] The sixth acquisition module is configured to obtain a first lidar weight and a second lidar weight based on the first variance average deviation, the second variance average deviation, the first difference, the second difference, the third difference, and the fourth difference;

[0181] The seventh acquisition module is configured to obtain a first millimeter-wave radar weight and a second millimeter-wave radar weight based on the first target distance mean value, the second target distance mean value, the first difference, the second difference, the third difference, and the fourth difference;

[0182] The eighth acquisition module is configured to obtain a lidar fusion weight based on the first lidar weight and the second lidar weight;

[0183] Based on the first millimeter-wave radar weight and the second millimeter-wave radar weight, obtain a millimeter-wave radar fusion weight;

[0184] The ninth acquisition module is configured to obtain target object information according to the lidar fusion weight, the millimeter-wave radar fusion weight, and a preset formula.

[0185] An electronic device according to an embodiment of the present disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0186] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions. In an embodiment of the present disclosure, the processor is used to run the computer-readable instructions stored in the memory, so that the electronic device executes all or part of the steps of the dynamic weight allocation method of the foregoing embodiments of the present disclosure.

[0187] Those skilled in the art should understand that, in order to solve the technical problem of how to obtain good user experience effects, known structures such as communication buses and interfaces may also be included in this embodiment, and these known structures should also be included in the protection scope of the present disclosure.

[0188] As Figure 7 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. It shows a schematic structural diagram of an electronic device suitable for implementing the electronic device in the embodiments of the present disclosure. Figure 7 The illustrated electronic device is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0189] As Figure 7 As shown, the electronic device may include a processor (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the electronic device are also stored. The processor, ROM, and RAM are connected to each other through a bus. The input / output (I / O) interface is also connected to the bus.

[0190] Generally, the following devices may be connected to the I / O interface: an input device including, for example, a sensor or a visual information acquisition device; an output device including, for example, a display screen; a storage device including, for example, a magnetic tape, a hard disk, etc.; and a communication device. The communication device may allow the electronic device to communicate with other devices (such as edge computing devices) wirelessly or wiredly to exchange data. Although Figure 7An electronic device having various devices is shown, but it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had.

[0191] In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program product that includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the dynamic weight allocation method according to the embodiments of the present disclosure are performed.

[0192] For a detailed description of this embodiment, reference may be made to the corresponding descriptions in the foregoing embodiments, and details are not described herein again.

[0193] A computer-readable storage medium according to an embodiment of the present disclosure has non-temporary computer-readable instructions stored thereon. When the non-temporary computer-readable instructions are run by a processor, all or part of the steps of the dynamic weight allocation method according to the foregoing embodiments of the present disclosure are performed.

[0194] The above-mentioned computer-readable storage medium includes, but is not limited to: optical storage media (such as CD-ROMs and DVDs), magneto-optical storage media (such as MOs), magnetic storage media (such as magnetic tapes or external hard drives), media having built-in rewritable non-volatile memories (such as memory cards), and media having built-in ROMs (such as ROM cartridges).

[0195] For a detailed description of this embodiment, reference may be made to the corresponding descriptions in the foregoing embodiments, and details are not described herein again.

[0196] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present disclosure. In addition, the above-mentioned specific details are only for illustrative and easy-to-understand purposes, rather than limitations, and the above details do not limit the present disclosure to necessarily adopt the above specific details for implementation.

[0197] In this disclosure, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and do not intend to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any way. Words such as "including", "comprising", "having", etc. are open-ended terms, meaning "including but not limited to", and can be used interchangeably with each other. The words "or" and "and" used herein refer to the word "and / or", and can be used interchangeably with each other, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to", and can be used interchangeably with each other.

[0198] In addition, as used herein, the "or" used in the listing of items starting with "at least one" indicates a disjunctive listing, so that for example, the listing of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the term "exemplary" does not mean that the examples described are preferred or better than other examples.

[0199] It should also be noted that in the systems and methods of this disclosure, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of this disclosure.

[0200] Various changes, substitutions, and alterations to the technologies described herein can be made without departing from the teachings defined by the appended claims. In addition, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and acts described above. Current or later-developed processes, machines, manufactures, compositions of events, means, methods, or acts that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or acts within their scope.

[0201] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

[0202] The foregoing description has been presented for purposes of illustration and description. In addition, the description is not intended to limit embodiments of the present disclosure to the form disclosed herein. Although several example aspects and embodiments have been discussed above, those skilled in the art will recognize some variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A dynamic weight allocation method, characterized in that, Including: Obtaining first point cloud data within a preset area of a target object based on a lidar; The first point cloud data includes a number of first valid points; Based on the number of the first valid points, obtaining a first target distance mean value and a first variance average deviation; Obtaining second point cloud data within a preset area of a target object based on a 4D millimeter wave radar; the second point cloud data includes a number of second valid points; Based on the number of the second valid points, obtaining a second target distance mean value and a second variance average deviation; Based on the first target distance mean value, the first variance average deviation, the second target distance mean value, and the second variance average deviation, obtaining a first difference value, a second difference value, a third difference value, and a fourth difference value; Based on the first variance average deviation, the second variance average deviation, the first difference value, the second difference value, the third difference value, and the fourth difference value, obtaining a first lidar weight and a second lidar weight; Based on the first target distance mean value, the second target distance mean value, the first difference value, the second difference value, the third difference value, and the fourth difference value, obtaining a first millimeter wave radar weight and a second millimeter wave radar weight; Based on the first lidar weight and the second lidar weight, obtaining a lidar fusion weight; Based on the first millimeter wave radar weight and the second millimeter wave radar weight, obtaining a millimeter wave radar fusion weight.

2. The dynamic weight allocation method according to claim 1, wherein The mean of the first target distance is l L , and the mean deviation of the first variance is d L ; Where R is the number of the first effective points, d(P,C R ) is any point P in the preset area to the geometric center point C of the preset area R distance, is the average distance between the first effective points in the preset area and the geometric center point of the preset area, is the variance of several first effective points in the preset area.

3. The dynamic weight allocation method according to claim 2, wherein The mean of the second target distance is l R , and the average deviation of the second variance is d R ; where N(R) is the number of the second valid points, and d i is the distance of the i-th second valid point.

4. The dynamic weight allocation method according to claim 3, wherein The first difference is D L , the second difference is D R , the third difference is L L , the fourth difference is L R ; D L = P1d L + P2d R - d L ; D R = P1d L + P2d R - d R ; L L = N1l L + N2l R - l L ; L R =N1l L +N2l R -l R ; 0 ≤ P1 ≤ 1, 0 ≤ P2 ≤ 1 and P1 + P2 = 1; 0 ≤ N1 ≤ 1, 0 ≤ N2 ≤ 1 and N1 + N2 = 1.

5. The dynamic weight allocation method according to claim 4, wherein P1 = P2 = N1 = N2 = 0.

5.

6. The dynamic weight allocation method according to claim 4, wherein The first weight of the lidar is W L1 , and the second weight of the lidar is W L2 ; 7. The dynamic weight allocation method according to claim 6, wherein The first weight of the millimeter-wave radar is W R1 , and the second weight of the millimeter-wave radar is W R2 ; 8. The dynamic weight allocation method according to claim 7, wherein The laser radar fusion weight is W L , the millimeter wave radar weight is W R ; W L = W L1 + W L2 ; W R = W R1 + W R2 。 9. A dynamic weight allocation system, characterized in that: Including: A first obtaining module configured to obtain first point cloud data within a preset area of a target object based on a lidar; The first point cloud data includes a number of first valid points; A second obtaining module configured to obtain a first target distance mean value and a first variance average deviation based on the number of the first valid points; A third obtaining module configured to obtain second point cloud data within a preset area of a target object based on a 4D millimeter wave radar; the second point cloud data includes a number of second valid points; A fourth obtaining module configured to obtain a second target distance mean value and a second variance average deviation based on the number of the second valid points; A fifth obtaining module configured to obtain a first difference value, a second difference value, a third difference value, and a fourth difference value based on the first target distance mean value, the first variance average deviation, the second target distance mean value, and the second variance average deviation; A sixth obtaining module configured to obtain a first lidar weight and a second lidar weight based on the first variance average deviation, the second variance average deviation, the first difference value, the second difference value, the third difference value, and the fourth difference value; A seventh obtaining module configured to obtain a first millimeter wave radar weight and a second millimeter wave radar weight based on the first target distance mean value, the second target distance mean value, the first difference value, the second difference value, the third difference value, and the fourth difference value; An eighth obtaining module configured to obtain a lidar fusion weight based on the first lidar weight and the second lidar weight; Obtain the millimeter-wave radar fusion weight based on the first weight of the millimeter-wave radar and the second weight of the millimeter-wave radar.

10. A method for acquiring information of a target object, characterized in that: This method is based on the dynamic weight allocation method described in claim 8, and further includes: Obtain the target object information according to the lidar fusion weight, the millimeter-wave radar fusion weight, and a preset formula; The target object information includes the relative angular velocity α, the relative velocity v, the x coordinate value x of the target object, the y coordinate value y of the target object, and the z coordinate value z of the target object. α = α L W L +α R W R ; v = v L W L +v R W R ; Among them, α L is the relative angular velocity between the lidar and the target object, and α R is the relative angular velocity between the millimeter-wave radar and the target object, v L is the relative velocity between the lidar and the target object, v R is the relative velocity between the millimeter-wave radar and the target object, is the maximum value in the x direction of the lidar effective point set, is the maximum value in the x direction of the millimeter-wave radar effective point set, is the minimum value in the x direction of the lidar effective point set, is the minimum value in the x direction of the millimeter-wave radar effective point set, is the maximum value in the y direction of the lidar effective point set, is the maximum value in the y direction of the millimeter-wave radar effective point set, is the minimum value in the y direction of the lidar effective point set, is the minimum value in the y direction of the millimeter-wave radar effective point set, is the maximum value in the z direction of the lidar effective point set, is the maximum value in the z direction of the millimeter-wave radar effective point set, is the minimum value in the z direction of the lidar effective point set, is the minimum value in the z direction of the millimeter-wave radar effective point set.

Images