Method for implementing velocity feature fusion for visual target and millimeter-wave radar point cloud

Abstract

The present invention relates to a method for implementing velocity feature fusion for a visual target and a millimeter-wave radar point cloud. The method comprises: within the same frame period, acquiring host vehicle state information, millimeter-wave radar point cloud data, and target information that is output by a visual detection network model; calculating an actual radial velocity of a point cloud to the ground on the basis of the host vehicle state information; associating the millimeter-wave radar point cloud data with a visual target; using height information and dynamic and static information of the visual target to perform secondary non-target point cloud screening on an initially selected point cloud cluster, so as to acquire a final associated point cloud cluster; performing preliminary verification on the currently acquired visual target, and determining whether a velocity vector of the target is accurate; and if the velocity vector is not accurate, on the basis of a training result obtained by the visual detection network model, determining the accuracy degrees of the current target velocity direction and magnitude, and using a target point cloud cluster to calculate a target velocity magnitude and an angular velocity magnitude, so as to implement fusion processing of velocity features.

Description

A method for fusing velocity features of visual targets and millimeter-wave radar point clouds

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202410934454.X, filed on July 12, 2024, the contents of which are incorporated herein by reference. Technical Field

[0003] This invention relates to the field of intelligent driving technology, and particularly to the field of multi-sensor fusion, specifically to a method, apparatus, processor, and computer-readable storage medium for fusing speed features of visual targets and millimeter-wave radar point clouds. Background Technology

[0004] To compensate for the limitations of single-type sensors, the environmental perception system in the intelligent driving industry often employs multi-sensor fusion solutions. Visual and millimeter-wave radar fusion has become one of the mainstream approaches. Visual perception has advantages in target shape, orientation, and type, and its technology is relatively mature. However, due to sensor characteristics, the reliability of detected velocity dimensions is not high. Millimeter-wave radar, on the other hand, has the advantage of high velocity measurement accuracy, but it only measures one-dimensional radial velocity. Therefore, the two can complement each other. Current fusion methods often use visually perceived targets and radar-sensed targets as independent inputs, directly fusing them, which cannot guarantee the accuracy of the input targets.

[0005] In response to this situation, it is necessary to propose a velocity fusion method for visual targets and radar point clouds based on the characteristics of fusion sensors. The two methods mutually verify and complement each other. The radial ground velocity of the point cloud is used to verify and correct the velocity information of the visual target, and the visual target is used to supplement the actual heading velocity of the target that cannot be directly measured by the radar point cloud, thereby improving the accuracy of target fusion. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, apparatus, processor and computer-readable storage medium for fusing velocity features of visual targets and millimeter-wave radar point clouds.

[0007] To achieve the above objectives, the present invention provides a method, apparatus, processor, and computer-readable storage medium for fusing velocity features of visual targets and millimeter-wave radar point clouds, as follows:

[0008] The method for fusing velocity features of visual targets and millimeter-wave radar point clouds is characterized by the following steps:

[0009] (1) Within the same frame period, acquire vehicle status information, millimeter-wave radar point cloud data and target information output by the visual detection network model; and calculate the actual radial velocity of the point cloud relative to the ground based on the vehicle status information to distinguish between dynamic and static states;

[0010] (2) The millimeter-wave radar point cloud data is associated with the visual target, and the point cloud cluster to which the visual target belongs is initially determined;

[0011] (3) Using the height information and dynamic and static information of the visual target, perform a second screening of non-target point clouds on the initial selected point cloud clusters to obtain the final associated point cloud clusters;

[0012] (4) Perform a preliminary verification on the currently acquired visual target and determine whether the target’s velocity vector is accurate. If it is accurate, use it directly as an accurate measurement value; otherwise, proceed to step (5).

[0013] (5) Based on the training results obtained from the visual detection network model, determine the accuracy of the current target velocity direction and magnitude, and use the target point cloud cluster and the target velocity direction to calculate the target velocity magnitude or direction, thereby realizing the fusion processing of velocity features.

[0014] Preferably, step (2) specifically includes:

[0015] Select millimeter-wave radar point clouds located within the visual target bounding box for position association; or

[0016] Position and velocity correlation are performed on visual targets after clustering millimeter-wave radar point clouds.

[0017] Preferably, step (4) includes the following steps:

[0018] (4.1) Calculate the radial velocity component of the target output by the visual detection network model in the point cloud cluster for each point cloud, and calculate the difference between this component and the radial velocity relative to the ground in the point cloud. The average difference, D_vr_mean, is then obtained.

[0019] Where n is the number of points in the point cloud cluster, vr_point is the radial velocity of the point cloud relative to the ground, α is the azimuth angle of the point cloud, v is the magnitude of the output target velocity, and θ is the direction of the output target velocity.

[0020] (4.2) Define the current tolerable velocity fluctuation deviation threshold according to the actual situation. If the average difference D_vr_mean is greater than the threshold, the preliminary verification is considered to be unsuccessful and proceed to step (5). Otherwise, the velocity vector of the target output by the visual detection network model is considered to be correct and can be used as an accurate measurement value.

[0021] Preferably, step (5) includes:

[0022] Based on the training results obtained from the aforementioned visual detection network model, assuming that the currently measured target velocity direction θ is more accurate, the target velocity magnitude v and target angular velocity ω are solved as follows:

[0023] (5.1) Assume the target center point (x, y) is the rotation center, and the point cloud position in the target point cloud cluster is (x... i ,y i If the coordinate difference between the point cloud and the target center point is (Δx), then the coordinate difference between the point cloud and the target center point is (Δx). i ,Δy i );

[0024] (5.2) Utilizing the mathematical relationship between the target point cloud cluster and the target: vr_point i =(v-ωΔy) i )×cos(α i -θ)+ωΔx i ×sin(α i -θ) = v×cos(α) i -θ)+ω(Δx i ×sin(α i -θ)-Δy i ×cos(α i -θ))

[0025] Where, Δx i =x i -x, Δy i =y i ×(-y);

[0026] Construct the matrix expression Vr = X × Y as follows:

[0027] Where Vr is an n×1 matrix, X is an n×2 matrix, and Y is a 2×1 matrix;

[0028] (5.3) The above formula is further transformed into:

[0029] Here, the Y matrix is ​​a matrix composed of the variables to be determined, and Y = (X) is obtained using the least squares method. T X) -1 X T Vr

[0030] By substituting the variable data, the target velocity magnitude v and the target angular velocity ω can be obtained.

[0031] Preferably, step (5) further includes:

[0032] When the target's own rotational angular velocity is ignored, the target velocity magnitude v is calculated according to the following formula: vr_point i =v×cos(α) i -θ)

[0033] Then, the least squares method can be used to find:

[0034] Preferably, step (5) further includes:

[0035] Based on the training results obtained from the aforementioned visual detection network model, if the currently measured target velocity magnitude v is more accurate, then it is assumed that the currently stated target velocity magnitude v is correct, and the target velocity direction θ is solved by constructing the following loss function:

[0036] And set constraints: cos 2 (θ)+sin 2 (θ) = 1.

[0037] The device for fusing velocity characteristics of visual targets and millimeter-wave radar point clouds is characterized by the following:

[0038] A processor is configured to execute computer-executable instructions;

[0039] The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the method described above for fusing velocity features of visual targets and millimeter-wave radar point clouds.

[0040] The processor for fusing velocity features of visual targets and millimeter-wave radar point clouds is characterized in that the processor is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the method for fusing velocity features of visual targets and millimeter-wave radar point clouds described above.

[0041] The computer-readable storage medium is characterized in that it stores a computer program thereon, which can be executed by a processor to implement the steps of the method described above for fusing velocity features of visual targets and millimeter-wave radar point clouds.

[0042] The present invention employs a method, apparatus, processor, and computer-readable storage medium for velocity feature fusion of visual targets and millimeter-wave radar point clouds. By using millimeter-wave radar point clouds to verify the accuracy of the front-end input results, compared with the traditional method of directly using sensors to identify targets for fusion, more feature information can be obtained as much as possible while ensuring accuracy, thereby effectively improving the accuracy of the final perception result output. Attached Figure Description

[0043] Figure 1 is a flowchart of the method for fusing velocity features of visual targets and millimeter-wave radar point clouds according to the present invention. Detailed Implementation

[0044] To more clearly describe the technical content of the present invention, the following description is provided in conjunction with specific embodiments.

[0045] Before describing the embodiments of the present invention in detail, it should be noted that, in the following, the terms “comprising,” “including,” or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0046] Please refer to Figure 1, which illustrates a method for fusing velocity features of visual targets and millimeter-wave radar point clouds. The method includes the following steps:

[0047] (1) Within the same frame period, acquire vehicle status information, millimeter-wave radar point cloud data and target information output by the visual detection network model; and calculate the actual radial velocity of the point cloud relative to the ground based on the vehicle status information to distinguish between dynamic and static states;

[0048] (2) The millimeter-wave radar point cloud data is associated with the visual target, and the point cloud cluster to which the visual target belongs is initially determined;

[0049] (3) Using the height information and dynamic and static information of the visual target, perform a second screening of non-target point clouds on the initial selected point cloud clusters to obtain the final associated point cloud clusters;

[0050] (4) Perform a preliminary verification on the currently acquired visual target and determine whether the target’s velocity vector is accurate. If it is accurate, use it directly as an accurate measurement value; otherwise, proceed to step (5).

[0051] (5) Based on the training results obtained from the visual detection network model, determine the accuracy of the current target velocity direction and magnitude, and use the target point cloud cluster and the target velocity direction to calculate the target velocity magnitude or direction, thereby realizing the fusion processing of velocity features.

[0052] In a preferred embodiment of the present invention, step (2) specifically comprises:

[0053] Select millimeter-wave radar point clouds located within the visual target bounding box for position association; or

[0054] Position and velocity correlation are performed on visual targets after clustering millimeter-wave radar point clouds.

[0055] In a preferred embodiment of the present invention, step (4) includes the following steps:

[0056] (4.1) Calculate the radial velocity component of the target output by the visual detection network model in the point cloud cluster for each point cloud, and calculate the difference between this component and the radial velocity relative to the ground in the point cloud. The average difference, D_vr_mean, is then obtained.

[0057] Where n is the number of points in the point cloud cluster, vr_point is the radial velocity of the point cloud relative to the ground, α is the azimuth angle of the point cloud, v is the magnitude of the output target velocity, and θ is the direction of the output target velocity.

[0058] (4.2) Define the current tolerable velocity fluctuation deviation threshold according to the actual situation. If the average difference D_vr_mean is greater than the threshold, the preliminary verification is considered to be unsuccessful and proceed to step (5). Otherwise, the velocity vector of the target output by the visual detection network model is considered to be correct and can be used as an accurate measurement value.

[0059] In a preferred embodiment of the present invention, step (5) includes:

[0060] Based on the training results obtained from the aforementioned visual detection network model, assuming that the currently measured target velocity direction θ is more accurate, the target velocity magnitude v and target angular velocity ω are solved as follows:

[0061] (5.1) Assume the target center point (x, y) is the rotation center, and the point cloud position in the target point cloud cluster is (x... i ,y i If the coordinate difference between the point cloud and the target center point is (Δx), then the coordinate difference between the point cloud and the target center point is (Δx). i ,Δy i );

[0062] (5.2) Utilizing the mathematical relationship between the target point cloud cluster and the target: vr_point i =(v-ωΔy) i )×cos(α i -θ)+ωΔx i ×sin(α i-θ) = v×cos(α) i -θ)+ω(Δx i ×sin(α i -θ)-Δy i ×cos(α i -θ))

[0063] Where, Δx i =x i -x, Δy i =y i ×(-y);

[0064] Construct the matrix expression Vr = X × Y as follows:

[0065] Where Vr is an n×1 matrix, X is an n×2 matrix, and Y is a 2×1 matrix;

[0066] (5.3) The above formula is further transformed into:

[0067] Here, the Y matrix is ​​a matrix composed of the variables to be determined, and Y = (X) is obtained using the least squares method. T X) -1 X T Vr

[0068] By substituting the variable data, the target velocity magnitude v and the target angular velocity ω can be obtained.

[0069] In a preferred embodiment of the present invention, step (5) further includes:

[0070] When the target's own rotational angular velocity is ignored, the target velocity magnitude v is calculated according to the following formula: vr_point i =v×cos(α) i -θ)

[0071] Then, the least squares method can be used to find:

[0072] In a preferred embodiment of the present invention, step (5) further includes:

[0073] Based on the training results obtained from the aforementioned visual detection network model, if the currently measured target velocity magnitude v is more accurate, then it is assumed that the currently stated target velocity magnitude v is correct, and the target velocity direction θ is solved by constructing the following loss function:

[0074] And set constraints: cos 2 (θ)+sin 2 (θ) = 1.

[0075] This technical solution proposes a novel method for fusing the velocity characteristics of visual targets and millimeter-wave radar point clouds, which utilizes radar point clouds to correct the velocity characteristics of visual targets and simultaneously solves for the target's rotational angular velocity.

[0076] The method provided by this invention mainly includes the following steps:

[0077] 1. Within the same frame period, acquire the following input information: vehicle status information, millimeter-wave radar point cloud (denoted as point) data, and target information output by the visual detection network model (denoted as target); and calculate the dynamic and static information of each point cloud, actual radial velocity to the ground, etc., based on the vehicle status information.

[0078] 2. Based on the detection characteristics of the visual model, the radar point cloud is associated with the visual target. The association methods are as follows: ① The point cloud is located within the bounding box of the visual target and associated with its position; ② The visual target is associated with its position and velocity after the point cloud is clustered. All associated point cloud clusters are used as the initial point cloud clusters of their respective targets.

[0079] 3. Using the target's height and dynamic / static information, perform a second round of non-target point cloud filtering on the initially selected point cloud clusters, filtering out incorrectly selected ground point clouds and high-altitude point clouds, to obtain the final associated point cloud clusters.

[0080] 4. Preliminary verification:

[0081] Calculate the radial velocity component of the target in each point cloud cluster and the difference between it and the radial velocity of the point cloud relative to the ground. Calculate the average difference D_vr_mean. Define a tolerable velocity fluctuation deviation threshold according to the actual situation. If the value of D_vr_mean is greater than the threshold, the verification is considered to have failed and proceed to step 5. Otherwise, the velocity vector of the target is considered to be correct and can be used as an accurate measurement.

[0082] Where n is the number of point clouds in the point cloud cluster, vr_point is the radial velocity of the point cloud relative to the ground, α is the azimuth angle of the point cloud, v is the magnitude of the target velocity, and θ is the direction of the target velocity.

[0083] 5. Based on the training results of the visual detection model, it can be seen that the target velocity orientation angle detected by the model is more accurate, that is, the accuracy of the output target velocity direction θ is higher. Therefore, it is assumed that the current target orientation angle θ is correct.

[0084] (1) Solve for the target velocity magnitude v and the target angular velocity ω. Assume the target center point (x, y) is the center of rotation, and the point cloud position in the target point cloud cluster is (x, y). i ,y i Then the difference between the point cloud and the coordinates is (Δx).i ,Δy i Using the mathematical relationship between the target point cloud cluster and the target: vr_point i =(v-ωΔy) i )×cos(α i -θ)+ωΔx i ×sin(α i -θ) = v×cos(α) i -θ)+ω(Δx i ×sin(α i -θ)-Δy i ×cos(α i -θ))

[0085] Where, Δx i =x i -x, Δy i =y i ×(-y);

[0086] Construct the matrix expression Vr = X × Y, where Vr is an n × 1 dimensional matrix, X is an n × 2 dimensional matrix, and Y is a 2 × 1 dimensional matrix:

[0087] Where the Y matrix is ​​the matrix composed of the variables to be determined, it is obtained using the least squares method: Y = (X T X) -1 X T Vr

[0088] By substituting all the data, the magnitude and angular velocity of the target can be obtained.

[0089] (2) If the target's own rotational angular velocity is ignored, the mathematical relationship simplifies to vr_point i =v×cos(α) i -θ)

[0090] Even if obtained using least squares

[0091] (3) Alternatively, if the training results of the detection model show that the accuracy of the output target velocity magnitude v is higher, then assume that the detected target velocity magnitude v is correct, solve for the θ value, and construct the following loss function:

[0092] And there are constraints: cos 2 (θ)+sin 2 (θ)=1

[0093] The above equation is transformed into an optimization problem with only equality constraints. The optimal solution θ can be obtained by conventional mathematical methods such as elimination method and Lagrange multiplier method.

[0094] In a specific embodiment of the present invention, the method for fusing velocity features of visual targets and millimeter-wave radar point clouds is implemented in practical applications as follows:

[0095] Step 1: Obtain three pieces of information: vehicle status information, millimeter-wave radar point cloud (denoted as point) data, and network model output target information (denoted as target), and calculate the dynamic and static information of the point cloud and the actual radial velocity to the ground.

[0096] Step 2: Use the network model to output the target's location and size bounding box information. Select point cloud clusters that are located within the bounding box, adjacent to the bounding box, and closely connected to the point cloud positions within the box as initial point cloud clusters belonging to the target. For example: If a target is obtained with a height range of 0.1 to 1.4 m, a ground velocity vector with a velocity magnitude of v = 6 m / s and a direction of θ = 0 degrees, the corresponding selected point cloud cluster contains three dynamic points and one static point, all with an azimuth angle α of 0 degrees. The coordinate positions (x, y, z) and radial ground velocities vr_point are ((40 m, 0 m, 1 m), 8 m / s), ((40.1 m, 0 m, 1 m), 8 m / s), ((40.2 m, 0 m, 1 m), 8 m / s), and ((40.3 m, 0.1 m, 7 m), 0.01 m / s).

[0097] Step 3: Using the target's height and dynamic / static information, perform a second screening of non-target point clouds on the initially selected point cloud clusters, filtering out ground point clouds and high-altitude point clouds that were mistakenly selected. For example, the static point ((40.3m, 0.1m, 7m), 0.01m / s) is obviously at an incorrect height and is therefore excluded.

[0098] Step 4: Preliminary verification:

[0099] In the example, the number of point clouds within the point cloud cluster is n=3. Substituting this into the formula, the average difference is calculated to be D_vr_mean = 2m / s.

[0100] Choosing threshold = 0.5 m / s results in a large difference and should not be used directly.

[0101] Step 5: Given that the target velocity direction output by the network model is more accurate, assuming that the obtained θ = 0° is the correct value, and ignoring the target angular velocity, only the v value needs to be solved. Substituting it into the formula, we get:

[0102] This device for fusing velocity features of visual targets and millimeter-wave radar point clouds includes:

[0103] A processor is configured to execute computer-executable instructions;

[0104] The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the method described above for fusing velocity features of visual targets and millimeter-wave radar point clouds.

[0105] The processor for fusing velocity features of visual targets and millimeter-wave radar point clouds is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the method for fusing velocity features of visual targets and millimeter-wave radar point clouds described above.

[0106] The computer-readable storage medium contains a computer program that can be executed by a processor to implement the steps of the method described above for fusing velocity features of visual targets with millimeter-wave radar point clouds.

[0107] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0108] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution device.

[0109] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0110] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0111] In the description of this specification, references to terms such as "an embodiment," "some embodiments," "example," "specific example," or "embodiment," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0112] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

[0113] The present invention employs a method, apparatus, processor, and computer-readable storage medium for velocity feature fusion of visual targets and millimeter-wave radar point clouds. By using millimeter-wave radar point clouds to verify the accuracy of the front-end input results, compared with the traditional method of directly using sensors to identify targets for fusion, more feature information can be obtained as much as possible while ensuring accuracy, thereby effectively improving the accuracy of the final perception result output.

[0114] In this specification, the invention has been described with reference to specific embodiments thereof. However, it will be apparent that various modifications and variations can be made without departing from the spirit and scope of the invention. Therefore, the specification and drawings should be considered illustrative rather than restrictive.

Claims

1. A method for fusing velocity features of visual targets and millimeter-wave radar point clouds, characterized in that, The method includes the following steps: (1) Within the same frame period, acquire vehicle status information, millimeter-wave radar point cloud data and target information output by the visual detection network model; and calculate the actual radial velocity of the point cloud relative to the ground based on the vehicle status information to distinguish between dynamic and static states; (2) The millimeter-wave radar point cloud data is associated with the visual target, and the point cloud cluster to which the visual target belongs is initially determined; (3) Using the height information and dynamic and static information of the visual target, perform a second screening of non-target point clouds on the initial selected point cloud clusters to obtain the final associated point cloud clusters; (4) Perform a preliminary verification on the currently acquired visual target and determine whether the target’s velocity vector is accurate. If it is accurate, use it directly as an accurate measurement value; otherwise, proceed to step (5). (5) Based on the training results obtained from the visual detection network model, determine the accuracy of the current target velocity direction, and use the target point cloud cluster and the target velocity direction to calculate the target velocity magnitude or direction, thereby realizing the fusion processing of velocity features. Specifically, step (2) is as follows: Select millimeter-wave radar point clouds located within the visual target bounding box for position association; or Correlate the position and velocity of visual targets after clustering millimeter-wave radar point clouds; Step (4) includes the following steps: (4.1) Calculate the radial velocity component of the target output by the visual detection network model in the point cloud cluster for each point cloud, and calculate the difference between this component and the radial velocity relative to the ground in the point cloud. The average difference, D_vr_mean, is then obtained. Where n is the number of points in the point cloud cluster, vr_point is the radial velocity of the point cloud relative to the ground, α is the azimuth angle of the point cloud, v is the magnitude of the output target velocity, and θ is the direction of the output target velocity. (4.2) Define the current tolerable velocity fluctuation deviation threshold according to the actual situation. If the average difference D_vr_mean is greater than the threshold, the preliminary verification is considered to be unsuccessful and proceed to step (5). Otherwise, the velocity vector of the target output by the visual detection network model is considered to be correct and can be used as an accurate measurement value.

2. The method for fusing velocity features of visual targets and millimeter-wave radar point clouds according to claim 1, characterized in that, Step (5) includes: Based on the training results obtained from the aforementioned visual detection network model, assuming that the currently measured target velocity direction θ is more accurate, the target velocity magnitude v and target angular velocity ω are solved as follows: (5.1) Assume the target center point (x, y) is the rotation center, and the point cloud position in the target point cloud cluster is (x... i ,y i If the coordinate difference between the point cloud and the target center point is (Δx), then the coordinate difference between the point cloud and the target center point is (Δx). i ,Δy i ); (5.2) Utilizing the mathematical relationship between the target point cloud cluster and the target: vr_point i =(v-ωΔy i )×cos(α i -θ)+ωΔx i ×sin(a i -i) =v×cos(α i -θ)+ω(Δx i ×sin(a i -i)-Δy i ×cos(α) i -i)) where, Δx i = x i - x, Δy i = y i × (-y); Construct the matrix expression Vr = X × Y as follows: Where Vr is an n×1 matrix, X is an n×2 matrix, and Y is a 2×1 matrix; (5.3) The above formula is further transformed into: Here, the Y matrix is ​​a matrix composed of the variables to be determined, obtained using the least squares method. Y=(X T X) -1 X T Vr By substituting the variable data, the target velocity magnitude v and the target angular velocity ω can be obtained.

3. The method for fusing velocity features of visual targets and millimeter-wave radar point clouds according to claim 2, characterized in that, Step (5) further includes: When the target's own rotational angular velocity is ignored, the magnitude of the target velocity v is calculated according to the following formula: vr_point i =v×cos(α i -i) Then, the least squares method can be used to find:

4. The method for fusing velocity features of visual targets and millimeter-wave radar point clouds according to claim 1, characterized in that, Step (5) further includes: Based on the training results obtained from the aforementioned visual detection network model, if the currently measured target velocity magnitude v is more accurate, then it is assumed that the currently stated target velocity magnitude v is correct, and the target velocity direction θ is solved by constructing the following loss function: And set constraints: cos 2 (θ)+sin 2 (θ) = 1.

5. A device for fusing velocity features of visual targets and millimeter-wave radar point clouds, characterized in that, The device includes: A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the method for fusing velocity features of a visual target and a millimeter-wave radar point cloud as described in any one of claims 1 to 4.

6. A processor for fusing velocity features of visual targets and millimeter-wave radar point clouds, characterized in that, The processor is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the method for velocity feature fusion of visual targets and millimeter-wave radar point clouds as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the method for fusing velocity features of a visual target and a millimeter-wave radar point cloud as described in any one of claims 1 to 4.

Images