Three-dimensional radar visual detection method and system

Abstract

The invention provides a three-dimensional radar visual detection method and system, and the method comprises the steps: carrying out the two-dimensional image detection of a target object, preliminarily judging the material attribute, transmitting a first electromagnetic wave to the target object, and carrying out the three-dimensional modeling; carrying out material parameter assignment on the target model, and carrying out electromagnetic wave propagation simulation to obtain a simulation radar waveform; performing ray tracing measurement on a target object through a radar to obtain an actually measured radar waveform; and comparing the simulation radar waveform with the actually measured radar waveform, if the difference between the simulation radar waveform and the actually measured radar waveform is within a first preset range, determining that the material of the target object is consistent with the corresponding material during material parameter assignment, and if the difference between the simulation radar waveform and the actually measured radar waveform is within the first preset range, updating the material parameters and repeating the above steps. According to the detection method, the three-dimensional form information of the target object can be obtained, the surface material attribute of the target object can be judged, the limitation of a system in the prior art is overcome, and the comprehensive object recognition and material analysis capability is provided.

Description

Technical Field

[0001] The present invention belongs to the field of radar detection technology, and more specifically, relates to a three-dimensional radar visual detection method and system. Background Art

[0002] In today's computer vision and object detection field, traditional two-dimensional image machine vision inspection systems have been widely used in industrial inspection, robot navigation and other scenarios. Through two-dimensional images, the system can identify surface features such as the shape, color and texture of objects, but cannot capture the three-dimensional structure or internal characteristics of objects, and is easily affected by changes in lighting, shadows and appearance. Moreover, because it relies on appearance information, the two-dimensional image recognition system is prone to errors when faced with objects that are similar in appearance but different in material (such as plastic products and real objects), and lacks in-depth analysis of the object's material.

[0003] At the same time, radar systems can identify the material properties of objects by detecting the reflection characteristics of electromagnetic waves, and are widely used in the military and aviation fields. However, traditional radar systems mainly rely on the interaction between electromagnetic waves and object materials, and cannot obtain detailed three-dimensional morphological information of objects, especially when dealing with complex geometric shapes. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a three-dimensional radar visual detection method and system to solve the technical problem that the two-dimensional visual detection system and radar system in the prior art cannot capture three-dimensional characteristics.

[0005] To achieve the above object, the technical solution adopted by the present invention is: to provide a three-dimensional radar visual detection method, comprising:

[0006] Perform two-dimensional image detection on the target object, preliminarily determine the material properties of the target object, and establish a material library and a point cloud library;

[0007] Transmitting a first electromagnetic signal to the target object at a plurality of detection points respectively, the first electromagnetic signal being received after being reflected by the target object, calculating the distance between the detection point and each pixel point of the target object according to the transmission and reception time of the first electromagnetic signal, and performing three-dimensional modeling to form a target model;

[0008] Assigning material parameters to the target model according to the material library, and performing electromagnetic wave propagation simulation to obtain a simulated radar waveform;

[0009] Performing ray tracing measurement on the target object by radar to obtain a measured radar waveform;

[0010] The simulated radar waveform is compared with the measured radar waveform: if the difference between the simulated radar waveform and the measured radar waveform is within a first preset range, the material of the target object is the corresponding material when the material parameter is assigned; if the difference between the simulated radar waveform and the measured radar waveform is outside the first preset range, the material parameters of the target model are updated, and after obtaining the simulated radar waveform again, this step is repeated.

[0011] Optionally, the step of assigning material parameters to the target model according to the material library and performing electromagnetic wave propagation simulation to obtain a simulated radar waveform includes:

[0012] Importing the target model into electromagnetic simulation software;

[0013] Assigning material parameters to the surface of the target model, wherein the corresponding materials when assigning material parameters are materials in the material library;

[0014] The propagation of electromagnetic waves on the surface and inside of the target model is calculated by a ray tracing algorithm to obtain a simulated radar waveform.

[0015] Optionally, the step of comparing the simulated radar waveform with the measured radar waveform includes:

[0016] Converting the simulated radar waveform and the measured radar waveform into the frequency domain, and obtaining the amplitude difference and phase difference between the simulated radar waveform and the measured radar waveform by calculating the frequency spectrum of the simulated radar waveform and the frequency spectrum of the measured radar waveform;

[0017] It is determined whether the difference between the simulated radar waveform and the measured radar waveform is within a first preset range according to the amplitude difference and the phase difference.

[0018] Optionally, the step of obtaining the amplitude difference and phase difference between the simulated radar waveform and the measured radar waveform by calculating the frequency spectrum of the simulated radar waveform and the frequency spectrum of the measured radar waveform includes:

[0019] The spectrum of the simulated radar waveform is X sim (f), the spectrum of the measured radar waveform is X meas (f);

[0020] The magnitude difference is Among them, A sim (f i )=|X sim (f i ), and A sim (f) is the amplitude spectrum of the simulated radar waveform, A meas (f i)=|X meas (f i )|, and A meas (f) is the amplitude spectrum of the measured radar waveform;

[0021] The phase difference is Among them, θ sim (f) is the phase spectrum of the simulated radar waveform, and θ meas (f) is the phase spectrum of the measured radar waveform.

[0022] Optionally, the step of determining whether the difference between the simulated radar waveform and the measured radar waveform is within a first preset range according to the amplitude difference and the phase difference includes:

[0023] Calculate the maximum value E of all amplitude differences mag,max and the minimum value E mag,min , and calculate the maximum value E of the phase difference phase,max and the minimum value E phase,min ,

[0024] Normalize the amplitude difference and the phase difference to obtain amplitude normalization And phase normalization

[0025] Calculate the comprehensive error Among them, α and β are weight coefficients, and the first preset range is E total ≤M.

[0026] Optionally, the step of updating the material parameters of the target model includes:

[0027] The current material parameters are The updated material parameters are: Among them, η is the learning rate, which is used to control the magnitude of each update. is the gradient of the first error function, by minimizing the first error function Update simulation parameters The difference between the simulated radar waveform and the measured radar waveform is reduced.

[0028] Optionally, the step of performing three-dimensional modeling to form a target model includes:

[0029] The original point cloud data is obtained by calculating the distance between the detection point and each pixel point of the target object, and outliers in the original point cloud data are filtered out to form a source point cloud;

[0030] Acquire a target point cloud according to the point cloud library, preliminarily align the plurality of source point clouds with the target point cloud, form a matching point pair with each point in the source point cloud and its nearest point in the target point cloud, calculate a transformation matrix according to the matching point pairs, apply the transformation matrix to the source point cloud, and gradually align the source point cloud with the target point cloud to form a modeling point cloud;

[0031] The modeling point cloud is three-dimensionally reconstructed using a three-dimensional mesh algorithm.

[0032] Optionally, the steps of forming a matching point pair between each point in the source point cloud and its nearest point in the target point cloud, calculating a transformation matrix based on the matching point pairs, applying the transformation matrix to the source point cloud, and gradually aligning the source point cloud with the target point cloud to form a modeling point cloud include:

[0033] Each of the source point clouds P={p1,p2...p n} and its nearest target point cloud Q = {q1,q2...q n The points qi in} form a matching point pair, and the rotation matrix R and translation vector t are calculated based on the matching point pair;

[0034] Applying the rotation matrix R and the translation vector t to the source point cloud to update the position of the source point cloud;

[0035] Repeat the above steps until the second error function is within the second preset range.

[0036] Optionally, the step of performing three-dimensional reconstruction on the modeling point cloud by using a three-dimensional mesh algorithm includes:

[0037] Selecting three points from the modeling point cloud to form an initial triangle;

[0038] Add the remaining points one by one, maintaining the empty circle property by edge flipping.

[0039] The present invention also provides a three-dimensional radar visual detection system, using the above three-dimensional radar visual detection method, comprising:

[0040] A radar module, used for transmitting and receiving radar signals to obtain measured radar waveforms;

[0041] A point cloud acquisition module, used for transmitting and receiving a first electromagnetic signal to obtain original point cloud data;

[0042] Storage and processing module, used for 3D modeling, electromagnetic simulation, data comparison and iterative backtracking.

[0043] The three-dimensional radar visual detection method and system provided by the present invention have the following beneficial effects: compared with the prior art, the three-dimensional radar visual detection method of the present invention performs two-dimensional image detection on the target object, preliminarily determines the material properties of the target object, and sends a first electromagnetic wave to the target object to calculate the distance between the detection point and each pixel point of the target object, and performs three-dimensional modeling; then the material parameters of the target model are assigned according to the determined material properties, and electromagnetic wave propagation simulation is performed to obtain a simulated radar waveform; the target object is ray-traced by radar to obtain a measured radar waveform; the simulated radar waveform and the measured radar waveform are compared, and if the difference between the two is within a first preset range, the material of the target object is consistent with the material corresponding to the material parameter assignment, and if the difference between the two is within a first preset range, the material parameters are updated and the above steps are repeated until the waveform difference between the two is within the first preset range. The above detection method can not only obtain the three-dimensional morphological information of the target object, but also determine the surface material properties of the target object, overcome the limitations of the prior art system, and provide comprehensive object recognition and material analysis capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A step diagram of a three-dimensional radar visual detection method provided by an embodiment of the present invention;

[0046] Figure 2 A schematic flow chart of a three-dimensional radar visual detection method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0048] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0049] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0050] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0051] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0052] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0053] The three-dimensional radar visual detection method provided by the embodiment of the present invention is now described. The three-dimensional radar visual detection method is to detect the three-dimensional geometric shape and material of the target object, and then visually detect the target object to more accurately determine the shape, material and category of the target object.

[0054] Please also read Figure 1 and Figure 2 , the three-dimensional radar visual detection method includes the following steps:

[0055] S10: Perform two-dimensional image detection on the target object, preliminarily determine the material properties of the target object, and establish a material library and a point cloud library;

[0056] S20: transmitting a first electromagnetic signal to the target object at a plurality of detection points respectively, the first electromagnetic signal being received after being reflected by the target object, calculating the distance between the detection point and each pixel point of the target object according to the emission and reception time of the first electromagnetic signal, and performing three-dimensional modeling to form a target model;

[0057] S30: assigning material parameters to the target model according to the material library, and performing electromagnetic wave propagation simulation to obtain a simulated radar waveform;

[0058] S40: performing ray tracing measurement on the target object through the radar to obtain a measured radar waveform;

[0059] S50: Compare the simulated radar waveform with the measured radar waveform: if the difference between the simulated radar waveform and the measured radar waveform is within a first preset range, the material of the target object is the corresponding material when the material parameters are assigned; if the difference between the simulated radar waveform and the measured radar waveform is outside the first preset range, update the material parameters of the target model, obtain the simulated radar waveform again, and repeat this step.

[0060] The target object is the object to be detected, and the three-dimensional radar visual detection method is used to perform visual detection on the target object.

[0061] In S10, two-dimensional image detection is to capture the image of the surface of the target object through a camera and classify the target object into object types. Specifically, two-dimensional image detection can detect and identify the appearance features of the target object, including parameters such as shape, texture, color, etc., and use the identified results to infer the material category of the object or the type of the target object. Through this process, the material and type of the target object can be preliminarily determined, and the corresponding material library and point cloud library can be formed to provide a reference for the subsequent material assignment in electromagnetic simulation. For example, through two-dimensional image detection, it can be preliminarily determined that the material of the target object is a cube with a metal surface. The established material library includes parameters of various metal materials, and the recommended point cloud library is a cube of various structural shapes.

[0062] In S20, by detecting the propagation time of the first electromagnetic signal, the transmission distance of the first electromagnetic signal can be calculated, and then the distance between each pixel point on the surface of the target object and the detection point can be obtained, and then the coordinates of each pixel point on the surface of the target object can be obtained.

[0063] In step S30, when simulating electromagnetic wave propagation of the target object, the material parameter setting of the target object is crucial, so assigning the material parameters according to the material library in step S10 is helpful to narrow the material selection range and improve the simulation and detection efficiency.

[0064] In step S40, the radar sends a radar signal to the target object, receives a signal reflected by the target object, and processes the echo signal to obtain a measured radar waveform.

[0065] In step S50, if the difference between the simulated radar waveform and the measured radar waveform is outside the first preset range, steps S30 and S50 are repeated and iterated until the difference between the simulated radar waveform and the measured radar waveform is within the first preset range.

[0066] It should be noted that step S10 may be implemented between step S20 and step S30, or before step S20, and step S40 may be implemented before step S10, between steps S10 and S20, or between steps S20 and S30.

[0067] The three-dimensional radar visual detection method in the above embodiment performs two-dimensional image detection on the target object, preliminarily determines the material properties of the target object, and sends a first electromagnetic wave to the target object to calculate the distance between the detection point and each pixel point of the target object to perform three-dimensional modeling; then assigns material parameters to the target model according to the determined material properties, and performs electromagnetic wave propagation simulation to obtain a simulated radar waveform; performs ray tracing measurement on the target object through radar to obtain a measured radar waveform; compares the simulated radar waveform with the measured radar waveform, if the difference between the two is within the first preset range, the material of the target object is consistent with the material corresponding to the material parameter assignment, if the difference between the two is within the first preset range, then the material parameters are updated and the above steps are repeated until the difference between the two waveforms is within the first preset range. The above detection method can not only obtain the three-dimensional morphological information of the target object, but also determine the surface material properties of the target object, overcome the limitations of the existing technical system, and provide comprehensive object recognition and material analysis capabilities.

[0068] In some embodiments of the present invention, step S10 includes:

[0069] S11: photographing the target object by using a camera to obtain appearance information of the target object;

[0070] S12: Use image recognition algorithm to identify the target object; the recognition process can use deep learning models (such as convolutional neural network, CNN) to extract features in the image.

[0071] S13: Classify the object type based on the recognition result and the feature matching algorithm, determine the material properties and category of the target object, and form a corresponding material library and point cloud library.

[0072] The camera used in S11 may be an RGB camera with a higher resolution. The RGB camera may be an independent camera or an RGB camera in a time-of-flight camera.

[0073] In some embodiments of the present invention, in step S20, the first electromagnetic signal is respectively transmitted to the target object at multiple detection points, the first electromagnetic signal is received after being reflected by the target object, and the distance between the detection point and each pixel point of the target object is calculated according to the transmission and reception time of the first electromagnetic signal as follows:

[0074] The point cloud acquisition module is located at the detection point. The first electromagnetic signal is emitted toward the target object through the point cloud acquisition module. After being reflected by the target object, the echo signal is received by the point cloud acquisition module, and then the distance between the detection point and each pixel point of the target object is calculated. Among them, the time difference from the emission to the reception of the first electromagnetic signal is t, and the distance between the detection point and the pixel point on the target object is d, d=c·t / 2, so d can be calculated through the time difference t. By detecting each pixel point of the target object, a "depth map" composed of countless points can be generated. These points represent the depth information of different surface areas of the target object. The "depth map" is actually a set of point cloud data, which contains the spatial coordinates (x, y, z) of each point. These coordinates together describe the surface structure of the object. Three-dimensional modeling is performed based on the above point cloud data to form a target model.

[0075] In some embodiments, the first electromagnetic signal may be infrared light, laser pulses, etc.

[0076] In some embodiments, the point cloud acquisition module may be a depth camera, etc., which can transmit and receive the first electromagnetic signal.

[0077] In some embodiments, the point cloud acquisition module further includes an RGB camera, which can be used in step S10 to perform two-dimensional image detection on the target object.

[0078] In some embodiments of the present invention, in step S20, the step of performing three-dimensional modeling to form a target model includes:

[0079] S21: obtaining original point cloud data by calculating the distance between the detection point and each pixel point of the target object, filtering outliers in the original point cloud data, and forming a source point cloud;

[0080] S22: acquiring a target point cloud according to the point cloud library, preliminarily registering multiple source point clouds with the target point cloud, forming a matching point pair between a point in each source point cloud and a point in the nearest target point cloud, calculating a transformation matrix according to the matching point pairs, applying the transformation matrix to the source point cloud, and gradually aligning the source point cloud with the target point cloud to form a modeling point cloud;

[0081] S23: Perform 3D reconstruction of the modeled point cloud using a 3D mesh algorithm.

[0082] In step S21, the original point cloud data can be understood as the above-mentioned "depth map" composed of countless points. This is the initial point cloud data corresponding to the target object. The original point cloud data may contain noise and erroneous data, so it needs to be processed to filter out outliers and form a source point cloud before it can be three-dimensionally modeled. In order to generate a complete three-dimensional model, it is necessary to scan the target object from multiple angles (the point cloud acquisition module sends a first electromagnetic signal toward the target object at multiple positions) to form multiple "depth maps". After filtering out outliers and other processing, multiple source point clouds are formed.

[0083] In step S22, the target point cloud may be a point cloud related to material simulation or other known models. Based on the preliminary two-dimensional image recognition results or certain feature information, the possible target point cloud set can be first narrowed down, that is, the point cloud library in step S10 can be used. For example, firstly, two-dimensional image recognition is used to infer that the object belongs to a certain type of material or shape, and then the standard point cloud of this type is selected from the point cloud library as the target point cloud. Since multiple source point clouds are formed by scanning the target object at various angles, it is necessary to align the multiple source point clouds. The method of finding and calculating the optimal transformation matrix can be used to perform rigid transformations such as rotation and translation on the source point cloud through the transformation matrix to minimize the alignment error of each group of source point clouds.

[0084] In some embodiments, step S21 includes:

[0085] Calculate the source point cloud P = {p1, p2...p n} the neighborhood of point pi within a predetermined radius;

[0086] Calculate the distance from all points in the neighborhood to the center point p of the area i The average distance d i ;

[0087] The average distance d to all points in the source point cloud i Statistical analysis is performed to obtain the average distance μ and standard deviation σ of the source point cloud;

[0088] If |d i -u|>k·σ, then point p i For outliers, point p needs to be filtered out i ; if |d i -u|≤k·σ, then point p i As a non-outlier point, retain point p i Among them, k is a parameter set by the user, which can be set and selected according to actual conditions.

[0089] In some embodiments, in step S22, multiple source point clouds and target point clouds are preliminarily aligned. Specifically, the preliminarily aligned can be performed by initial position estimation or by using feature point alignment. The initial alignment can make the distance between the source point cloud and the target point cloud within a reasonable range so that subsequent iterations can converge effectively.

[0090] In some embodiments, in step S22, each point in the source point cloud forms a matching point pair with its nearest point in the target point cloud, a transformation matrix is ​​calculated based on the matching point pairs, the transformation matrix is ​​applied to the source point cloud, and the source point cloud is gradually aligned with the target point cloud to form a modeling point cloud. The steps include:

[0091] S221: Each source point cloud P = {p1, p2...p n} and its nearest target point cloud Q = {q1,q2...q n Point q in i Form matching point pairs, and calculate the rotation matrix R and translation vector t based on the matching point pairs;

[0092] S222: Apply the rotation matrix R and the translation vector t to the source point cloud to update the position of the source point cloud;

[0093] S223: Repeat the above steps until the second error function is within the second preset range.

[0094] Each point in the source point cloud finds the nearest point corresponding to the target point cloud to form many matching point pairs. For each matching point pair, the rotation matrix R and translation vector t can be calculated, and then the rotation matrix R and translation vector t are applied to the source point cloud to make the positions of each point in the source point cloud close to the target point cloud. After each rotation and translation, the positions of each point in the source point cloud are updated, and steps S221 and S222 are repeated until the second error function is within the second preset range. In this process, the source point cloud is gradually aligned with the target point cloud to form a modeling point cloud. If the number of iterations exceeds the maximum number, it means that the matching point pair is incorrectly matched, and the point pair needs to be re-matched, and then steps S221 to S223 are performed. Among them, the second preset range is a pre-set range, which can be determined according to the detection accuracy requirements.

[0095] The rotation matrix R is a 3×3 rotation matrix used to describe the rotation operation in three-dimensional space. Each column or row (depending on the construction method) of the rotation matrix R represents the direction of the X, Y, and Z axes in the new coordinate system. The rotation matrix R maintains the shape and size of the source point cloud and only changes its direction. The translation vector t is a 3×1 translation vector used to describe the translation operation in three-dimensional space.

[0096] In some embodiments, step S23 includes:

[0097] S231: Select three points from the modeling point cloud to form an initial triangle;

[0098] S232: Add the remaining points one by one, and maintain the empty circle property by edge flipping.

[0099] After point cloud processing is performed in step S22 to form a modeling point cloud, the modeling point cloud is then three-dimensionally reconstructed using a triangular mesh algorithm. Specifically, discrete points may be connected into triangular facets through triangulation to form a smooth three-dimensional model.

[0100] The empty circle property means that in each generated triangle, its circumscribed circle does not contain any other points. This property ensures the stability and quality of triangulation. Edge flipping is a commonly used method to restore the empty circle property. Specifically, assume that two adjacent triangles ΔABC and ΔABD share an edge AD, and A, B, C, and D are four vertices. Calculate the circumscribed circles of the two triangles that share the edge AB and determine whether the two circles contain each other's vertices (such as C or D). If the circumscribed circle of one triangle contains the non-shared vertices of the other triangle, it means that the pair of triangles does not meet the Denoney condition. In order to restore the empty circle property, replace the current shared edge AB with the connecting line CD. This replacement will turn the original two triangles ΔABC and ΔABD into two new triangles ΔACD and ΔBCD. After edge flipping, the circumscribed circles of the new two triangles should satisfy the empty circle property.

[0101] For a triangle ΔABC, whose three vertices are A(x1,y1), B(x2,y2), and C(x3,y3), the center o(x,y) and radius R of its circumscribed circle can be calculated by the following formula:

[0102]

[0103] In order to quickly determine a point P(x p ,y p ) is inside the circumcircle of the triangle, we can use the following determinant:

[0104]

[0105] If the value of the determinant is less than zero, it means that point P is inside the circumscribed circle and needs to be adjusted; otherwise, it means that the triangle satisfies the empty circle property.

[0106] In some embodiments of the present invention, step S30 includes:

[0107] S31: importing the target model into electromagnetic simulation software;

[0108] S32: assigning material parameters to the surface of the target model, wherein the corresponding materials when assigning the material parameters are the materials in the material library;

[0109] S33: The propagation of electromagnetic waves on the surface and inside of the target model is calculated through a ray tracing algorithm to obtain a simulated radar waveform.

[0110] In step S31, the target model is generally a polygonal mesh (such as STL format). However, electromagnetic simulation software usually requires a more optimized geometric model, such as a CAD model (STEP, IGES format) for efficient calculation. Therefore, it is necessary to convert the target model (polygonal mesh model) into a CAD model before performing step S30. For example, software tools (such as Geomagic, MeshLab) can be used to convert polygonal meshes to CAD. These tools will simplify and reconstruct the surface of the mesh data to make it suitable for simulation applications.

[0111] Electromagnetic simulation software (such as Ansys HFSS, CST Microwave Studio, etc.) generally supports the import of various 3D model formats, including STEP, IGES, STL, etc. After the geometric structure of the model is assigned, the final model can be directly imported into these simulation software.

[0112] In step S32, material parameters are assigned to the surface of the target model according to the material library obtained by two-dimensional image recognition. The material parameters may include key parameters such as dielectric constant, conductivity, and magnetic permeability. For example, the type of material of the target object can be preliminarily identified according to the two-dimensional image. For example, if the target object is identified as metal, wood, or plastic, etc., then when assigning material parameters, the material parameters are assigned according to the corresponding material category.

[0113] In step S33, ray tracing is a technique for simulating the interaction between electromagnetic waves and objects. It simulates the propagation process of radar waves in three-dimensional space by tracking the path of the rays. The core idea of ​​ray tracing is based on the principle of geometric optics, which regards electromagnetic waves as rays and considers the reflection, refraction and absorption effects of rays on the surface of objects. The propagation of electromagnetic waves in the medium follows Maxwell's Equations, in which the reflection and refraction of waves can be calculated by Fresnel Equations. Through the ray tracing algorithm, the propagation of electromagnetic waves in the target model and inside is calculated to obtain the simulated radar waveform, which is a conventional technical means and will not be repeated here.

[0114] In some embodiments of the present invention, in step S40, a ray tracing measurement is performed on the target object by radar to obtain a measured radar waveform.

[0115] The radar can use frequency modulated continuous wave (FMCW) radar technology and work in the millimeter wave frequency band to achieve high-precision detection of target objects. The radar includes a transmitting module for transmitting radar waves with a frequency that varies with time (such as frequency modulated continuous wave); a receiving module for receiving radar waves reflected from the target object; and a signal processing module for processing the received signal and generating a radar waveform.

[0116] The signal transmitted by the radar transmitting module is a frequency modulated continuous wave signal (Chirp), whose frequency increases linearly from the initial frequency to the maximum frequency within a certain period of time (called the Chirp period).

[0117] First, the synthesizer generates a Chirp signal, which is then transmitted through the TX antenna (transmitting module) and a piece of data is sent to the mixer. After being reflected by the target object, the echo signal is received by the RX antenna (receiving module) to generate an intermediate frequency signal in the mixer.

[0118] Then, the radar mixes the transmitted signal with the received signal to obtain a signal containing an intermediate frequency, and uses an ADC (analog-to-digital converter) to sample the intermediate frequency signal to obtain a discrete time series. In order to reduce the interference of noise, preprocessing can be performed first, using a low-pass filter to remove high-frequency noise, or band-pass filtering the signal to focus on a specific frequency band. Apply window functions such as Hanning window or Heyman window to reduce spectrum leakage and improve the resolution of spectrum analysis. Perform FFT on the preprocessed intermediate frequency signal to obtain amplitude and phase information in the frequency domain.

[0119] In some embodiments of the present invention, in step S50, the step of comparing the simulated radar waveform with the measured radar waveform includes:

[0120] S51: converting the simulated radar waveform and the measured radar waveform into the frequency domain, and obtaining the amplitude difference and phase difference between the simulated radar waveform and the measured radar waveform by calculating the frequency spectrum of the simulated radar waveform and the frequency spectrum of the measured radar waveform;

[0121] S52: Determine whether the difference between the simulated radar waveform and the measured radar waveform is within a first preset range according to the amplitude difference and the phase difference.

[0122] By comparing the amplitude and phase differences between the simulated radar waveform and the measured radar waveform, it is possible to determine whether the material parameter assignment of the target model is accurate. If it is inaccurate, it is necessary to update the material parameter assignment, perform electromagnetic simulation on the target model again to obtain the simulated radar waveform, and compare it with the measured radar waveform.

[0123] In some embodiments, step S51 includes:

[0124] The spectrum of the simulated radar waveform is X sim (f), the spectrum of the measured radar waveform is X meas (f);

[0125] The difference in magnitude is Among them, A sim (f i )=|X sim (f i ), and A sim (f) is the amplitude spectrum of the simulated radar waveform, A meas (f i )=|X meas (f i )|, and A meas (f) is the amplitude spectrum of the measured radar waveform;

[0126] The phase difference is Among them, θ sim (f) is the phase spectrum of the simulated radar waveform, and θ meas (f) is the phase spectrum of the measured radar waveform.

[0127] In this way, the amplitude difference and phase difference between the simulated radar waveform and the measured radar waveform can be obtained through step S51. When the simulated radar waveform and the measured radar waveform are compared, the amplitude difference and phase difference between the two are actually compared.

[0128] In some embodiments, step S52 includes:

[0129] Calculate the maximum value E of all amplitude differences mag,max and the minimum value E mag,min , and calculate the maximum phase difference E phase,max and the minimum value E phase,min ,

[0130] Normalize the amplitude difference and phase difference to get the amplitude normalization And phase normalization

[0131] Calculate the comprehensive error Among them, α and β are weight coefficients, and the first preset range is E total ≤M.

[0132] Understandably, the comprehensive error E total ≤M, the difference between the simulated radar waveform and the measured radar waveform is within the first preset range, and the comprehensive error E total >M, the difference between the simulated radar waveform and the measured radar waveform is outside the first preset range. Wherein, M is a known quantity, and M can be between 0.05 and 0.1.

[0133] In some embodiments of the present invention, in step S50, when the difference between the simulated radar waveform and the measured radar waveform is outside the first preset range, it is necessary to update the material parameters of the target model, which can also be understood as the need to optimize the material parameters of the target model so that the simulated radar waveform gradually approaches the measured radar waveform.

[0134] In some embodiments, the simulation parameters are optimized using a gradient descent method, which is an optimization algorithm that updates parameters based on the gradient of a loss function. The goal of the gradient descent method is to find the optimal material parameters by minimizing the error function so that the difference between the simulated radar waveform and the measured radar waveform is minimized.

[0135] In some embodiments, the step of updating the material parameters of the target model includes:

[0136] The current material parameters are The updated material parameters are: Among them, η is the learning rate, which is used to control the magnitude of each update. is the gradient of the first error function, by minimizing the first error function Update simulation parameters Reduce the difference between simulated radar waveform and measured radar waveform.

[0137] The above optimization process will continue to iterate until one of the following stopping conditions is met:

[0138] When E total When ≤M, the simulated radar waveform is consistent with the measured radar waveform.

[0139] If the preset maximum number of iterations is reached, the optimization process stops, even if the error has not fully converged, the simulated radar waveform and the measured radar waveform are inconsistent, and the material parameters need to be updated (the material type needs to be changed).

[0140] If the parameter changes each time are small, it means that the optimization process has stabilized and can be stopped.

[0141] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0142] The present invention further provides a three-dimensional radar visual detection system, using the three-dimensional radar visual detection method in any of the above embodiments, the three-dimensional radar visual detection system comprises:

[0143] A radar module, used for transmitting and receiving radar signals to obtain measured radar waveforms;

[0144] A point cloud acquisition module, used for transmitting and receiving a first electromagnetic signal to obtain a source point cloud;

[0145] Storage and processing module, used for 3D modeling, electromagnetic simulation, data comparison and iterative backtracking.

[0146] The three-dimensional radar vision detection system provided by the present invention can not only obtain the three-dimensional morphological information of the target object, but also determine the surface material properties of the target object, overcoming the limitations of the existing technical systems and providing comprehensive object recognition and material analysis capabilities.

[0147] In some embodiments, the radar module acquires the measured radar signal of the target object by transmitting and receiving radar signals. The radar module may include a transmitting module (such as a radio frequency transmitter) and a receiving module (such as a radio frequency receiver), the transmitting module can transmit frequency modulated continuous wave or other forms of radar signals, and the receiving module can capture the echo signal of the target object. After measuring the echo of the target object, the radar module converts the echo signal into an intermediate frequency signal and converts it into a digital signal through an analog-to-digital converter, and then transmits it to the computer system module through a USB or Ethernet interface. This process ensures the accurate acquisition of the signal and provides real radar data for subsequent comparative analysis.

[0148] In some embodiments, the point cloud acquisition module is used to capture the three-dimensional structural information of the target object and generate high-precision raw point cloud data. The point cloud acquisition module actively emits infrared light or other light sources and uses light flight time (measurement technology) to obtain a depth map. In addition to three-dimensional modeling, the point cloud acquisition module can collect a two-dimensional RGB image of the target for preliminary identification of the surface material of the target object. The point cloud acquisition module transmits the raw point cloud data and RGB image data to the computer system module through a high-speed interface (such as USB or Camera Link), ensuring efficient and accurate data transmission.

[0149] In some embodiments, the storage and processing module includes a three-dimensional modeling module, a material parameter assignment module, an electromagnetic simulation module, a data comparison module, an iterative backtracking module, etc.

[0150] The 3D modeling module receives the original point cloud data from the point cloud acquisition module and converts it into a 3D model file of the target object using a point cloud processing algorithm. The 3D modeling process includes steps such as point cloud denoising, point cloud registration, and surface reconstruction to ensure the integrity and accuracy of the model. The 3D model file will be further imported into the electromagnetic simulation module.

[0151] The material parameter assignment module processes the 2D RGB image detected by the 2D graphics, identifies the type of the target object (such as metal, plastic, etc.) through the computer vision algorithm, and preliminarily assigns material parameters to the target material properties based on the identification results. The assigned material properties will be used as the initial conditions for the simulation. This module can use pre-trained image recognition models or deep learning algorithms to improve the accuracy of material assignment.

[0152] The electromagnetic simulation module uses the 3D model generated by the 3D modeling module and the preliminary material assignments of the material parameter assignment module as input, and simulates the material properties through ray tracing. The ray tracing method is used to simulate the propagation and reflection of electromagnetic waves on the surface and inside of the target object, and generate the simulated radar waveform of the target object (such as reflection coefficient, scattering coefficient, etc.). The output of the simulated radar waveform will be passed to the data comparison module for subsequent analysis.

[0153] The data comparison module receives the measured radar waveform from the radar module and the simulated radar waveform from the electromagnetic simulation module, and compares the two using signal analysis algorithms (such as spectrum analysis and signal feature matching). The comparison result can be the consistency or similarity score of the two. If the comparison result reaches the first preset range, the final material identification result is output; otherwise, the iterative backtracking process is entered to further optimize the material assignment.

[0154] When the data comparison module detects that the measured radar waveform and the simulated radar waveform are inconsistent, the iterative backtracking module will start the backtracking process. Through the feedback mechanism, this module transmits the inconsistent information to the material parameter assignment module, adjusts the material property assignment, and re-simulates the material property and compares the data until the simulation result with the consistent waveform is obtained.

[0155] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A three-dimensional radar visual detection method, characterized in that: include: Perform two-dimensional image detection on the target object, preliminarily determine the material properties of the target object, and establish a material library and a point cloud library; Transmitting a first electromagnetic signal to the target object at a plurality of detection points respectively, the first electromagnetic signal being received after being reflected by the target object, calculating the distance between the detection point and each pixel point of the target object according to the transmission and reception time of the first electromagnetic signal, and performing three-dimensional modeling to form a target model; Assigning material parameters to the target model according to the material library, and performing electromagnetic wave propagation simulation to obtain a simulated radar waveform; Performing ray tracing measurement on the target object by radar to obtain a measured radar waveform; The simulated radar waveform is compared with the measured radar waveform: if the difference between the simulated radar waveform and the measured radar waveform is within a first preset range, the material of the target object is the corresponding material when the material parameter is assigned; if the difference between the simulated radar waveform and the measured radar waveform is outside the first preset range, the material parameters of the target model are updated, and after obtaining the simulated radar waveform again, this step is repeated.

2. The three-dimensional radar visual detection method according to claim 1, characterized in that: The steps of assigning material parameters to the target model according to the material library and simulating electromagnetic wave propagation to obtain a simulated radar waveform include: Importing the target model into electromagnetic simulation software; Assigning material parameters to the surface of the target model, wherein the corresponding materials when assigning material parameters are materials in the material library; The propagation of electromagnetic waves on the surface and inside of the target model is calculated by a ray tracing algorithm to obtain a simulated radar waveform.

3. The three-dimensional radar visual detection method according to claim 1, characterized in that: The step of comparing the simulated radar waveform with the measured radar waveform comprises: Converting the simulated radar waveform and the measured radar waveform into the frequency domain, and obtaining the amplitude difference and phase difference between the simulated radar waveform and the measured radar waveform by calculating the frequency spectrum of the simulated radar waveform and the frequency spectrum of the measured radar waveform; It is determined whether the difference between the simulated radar waveform and the measured radar waveform is within a first preset range according to the amplitude difference and the phase difference.

4. The three-dimensional radar visual detection method according to claim 3, characterized in that: The step of obtaining the amplitude difference and phase difference between the simulated radar waveform and the measured radar waveform by calculating the frequency spectrum of the simulated radar waveform and the frequency spectrum of the measured radar waveform comprises: The spectrum of the simulated radar waveform is X sim (f), the spectrum of the measured radar waveform is X meas (f); The magnitude difference is Among them, A sim (f i )=|X sim (f i ), and A sim (f) is the amplitude spectrum of the simulated radar waveform, A meas (f i )=|X meas (f i ), and A meas (f) is the amplitude spectrum of the measured radar waveform; The phase difference is Among them, θ sim (f) is the phase spectrum of the simulated radar waveform, and θ meas (f) is the phase spectrum of the measured radar waveform.

5. The three-dimensional radar visual detection method according to claim 3, characterized in that: The step of determining whether the difference between the simulated radar waveform and the measured radar waveform is within a first preset range according to the amplitude difference and the phase difference comprises: Calculate the maximum value E of all amplitude differences mag,max and the minimum value E mag,min , and calculate the maximum value E of the phase difference phase,max and the minimum value E phase,min , Normalize the amplitude difference and the phase difference to obtain amplitude normalization And phase normalization Calculate the comprehensive error Among them, α and β are weight coefficients, and the first preset range is E total ≤M.

6. The three-dimensional radar visual detection method according to any one of claims 1 to 5, characterized in that: The step of updating the material parameters of the target model comprises: The current material parameters are The updated material parameters are: Among them, η is the learning rate, which is used to control the magnitude of each update. is the gradient of the first error function, by minimizing the first error function Update simulation parameters The difference between the simulated radar waveform and the measured radar waveform is reduced.

7. The three-dimensional radar visual detection method according to any one of claims 1 to 5, characterized in that: The steps of performing three-dimensional modeling to form a target model include: The original point cloud data is obtained by calculating the distance between the detection point and each pixel point of the target object, and outliers in the original point cloud data are filtered out to form a source point cloud; Acquire a target point cloud according to the point cloud library, preliminarily align the plurality of source point clouds with the target point cloud, form a matching point pair with each point in the source point cloud and its nearest point in the target point cloud, calculate a transformation matrix according to the matching point pairs, apply the transformation matrix to the source point cloud, and gradually align the source point cloud with the target point cloud to form a modeling point cloud; The modeling point cloud is three-dimensionally reconstructed using a three-dimensional mesh algorithm.

8. The three-dimensional radar visual detection method according to claim 7, characterized in that: The steps of forming a matching point pair between each point in the source point cloud and its nearest point in the target point cloud, calculating a transformation matrix based on the matching point pairs, applying the transformation matrix to the source point cloud, and gradually aligning the source point cloud with the target point cloud to form a modeling point cloud include: Each of the source point clouds P={p1,p2...p n } and its nearest target point cloud Q = {q1,q2...q n The points qi in} form a matching point pair, and the rotation matrix R and translation vector t are calculated based on the matching point pair; Applying the rotation matrix R and the translation vector t to the source point cloud to update the position of the source point cloud; Repeat the above steps until the second error function is within the second preset range.

9. The three-dimensional radar visual detection method according to claim 7, characterized in that: The step of three-dimensionally reconstructing the modeling point cloud by a three-dimensional mesh algorithm comprises: Selecting three points from the modeling point cloud to form an initial triangle; Add the remaining points one by one, maintaining the empty circle property by edge flipping.

10. A three-dimensional radar visual detection system, using the three-dimensional radar visual detection method according to any one of claims 1 to 9, characterized in that: include: A radar module, used for transmitting and receiving radar signals to obtain measured radar waveforms; A point cloud acquisition module, used for transmitting and receiving a first electromagnetic signal to obtain original point cloud data; Storage and processing module, used for 3D modeling, electromagnetic simulation, data comparison and iterative backtracking.

Images