3D Radar Visual Inspection Method and System

Abstract

This invention provides a three-dimensional radar vision inspection method and system. The method first performs two-dimensional image inspection on the target object to preliminarily determine its material properties, and then sends a first electromagnetic wave to the target object to perform three-dimensional modeling. Next, material parameters are assigned to the target model, and electromagnetic wave propagation simulation is performed to obtain a simulated radar waveform. Ray tracing measurements are then performed on the target object using radar to obtain a measured radar waveform. The simulated radar waveform and the measured radar waveform are compared. If the difference between the two is within a first preset range, the material of the target object matches the material corresponding to the assigned material parameters. If the difference is still within the first preset range, the material parameters are updated, and the above steps are repeated. This inspection method can not only acquire the three-dimensional morphological information of the target object but also determine its surface material properties, overcoming the limitations of existing technology systems and providing comprehensive object recognition and material analysis capabilities.

Description

Technical Field

[0001] This invention belongs to the field of radar detection technology, and more specifically, relates to a three-dimensional radar visual detection method and system. Background Technology

[0002] In the field of computer vision and object detection, traditional two-dimensional image machine vision inspection systems are widely used in industrial inspection, robot navigation, and other scenarios. While two-dimensional images allow systems to identify surface features such as shape, color, and texture, they cannot capture the three-dimensional structure or internal characteristics of objects and are easily affected by lighting, shadows, and changes in appearance. Furthermore, because they rely on appearance information, two-dimensional image recognition systems are prone to errors when dealing with objects that are similar in shape but different in material (such as plastic products versus real objects), and they lack in-depth analysis of the object's material.

[0003] Meanwhile, radar systems, by detecting the reflection characteristics of electromagnetic waves, can identify the material properties of objects and are widely used in military and aerospace fields. However, traditional radar systems mainly rely on the interaction between electromagnetic waves and the material of objects, and cannot obtain detailed three-dimensional morphological information of objects, especially showing limitations when dealing with complex geometries. Summary of the Invention

[0004] The purpose of this invention is to provide a three-dimensional radar visual inspection method and system to solve the technical problem that existing two-dimensional visual inspection systems and radar systems cannot capture three-dimensional characteristics.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is: to provide a three-dimensional radar visual inspection method, comprising:

[0006] Two-dimensional image detection is performed on the target object to preliminarily determine the material properties of the target object and establish a material library and a point cloud library.

[0007] A first electromagnetic signal is emitted towards the target object at multiple detection points. The first electromagnetic signal is received after being reflected by the target object. The distance between each pixel of the detection point and the target object is calculated by the emission and reception time of the first electromagnetic signal, and a three-dimensional model is formed to create the target model.

[0008] The target model is assigned material parameters based on the material library, and electromagnetic wave propagation simulation is performed to obtain the simulated radar waveform.

[0009] The target object is measured by ray tracing using radar to obtain the measured radar waveform.

[0010] The simulated radar waveform and the measured radar waveform are compared: if the difference between the simulated radar waveform and the measured radar waveform is within a first preset range, then the material of the target object is the material corresponding to the material parameter assignment; if the difference between the simulated radar waveform and the measured radar waveform is outside the first preset range, then the material parameters of the target model are updated, and after obtaining the simulated radar waveform again, this step is repeated.

[0011] Optionally, the steps of assigning material parameters to the target model according to the material library and performing electromagnetic wave propagation simulation to obtain the simulated radar waveform include:

[0012] Import the target model into the electromagnetic simulation software;

[0013] Material parameters are assigned to the surface of the target model, and the corresponding material used in the material library is used when assigning the material parameters.

[0014] The propagation of electromagnetic waves on the surface and inside the target model is calculated using a ray tracing algorithm to obtain simulated radar waveforms.

[0015] Optionally, the step of comparing the simulated radar waveform and the measured radar waveform includes:

[0016] Both the simulated radar waveform and the measured radar waveform are converted to the frequency domain. By calculating the spectrum of the simulated radar waveform and the spectrum of the measured radar waveform, the amplitude difference and phase difference between the simulated radar waveform and the measured radar waveform are obtained.

[0017] Based on the amplitude difference and the phase difference, determine whether the difference between the simulated radar waveform and the measured radar waveform is within a first preset range.

[0018] Optionally, the step of calculating the amplitude difference and phase difference between the simulated radar waveform and the measured radar waveform by analyzing the spectrum of the simulated radar waveform and the spectrum of the measured radar waveform includes:

[0019] The spectrum of the simulated radar waveform is X. sim (f) The spectrum of the measured radar waveform is X meas (f);

[0020] The magnitude difference is Among them, A sim (f i )=|X sim (f i ), and A sim (f) represents the amplitude spectrum of the simulated radar waveform, A meas (f i)=|X meas (f i )|, and A meas (f) represents the amplitude spectrum of the measured radar waveform;

[0021] The phase difference is Where, θ sim (f) is the phase spectrum of the simulated radar waveform, and θ meas (f) is the phase spectrum of the measured radar waveform.

[0022] Optionally, the step of determining whether the difference between the simulated radar waveform and the measured radar waveform is within a first preset range based on the amplitude difference and the phase difference includes:

[0023] Calculate the maximum value E of all amplitude differences. mag,max and minimum value E mag,min And calculate the maximum value E of the phase difference. phase,max and minimum value E phase,min ,

[0024] The amplitude difference and the phase difference are normalized to obtain the amplitude normalization. and phase normalization

[0025] The comprehensive error was calculated. Where α and β are weighting coefficients, and the first preset range is E. total ≤M.

[0026] Optionally, the step of updating the material parameters of the target model includes:

[0027] The current material parameters are The updated material parameters are as follows Where η is the learning rate, used to control the magnitude of each update. The gradient of the first error function is obtained by minimizing the first error function. Update simulation parameters Reduce the difference between the simulated radar waveform and the measured radar waveform.

[0028] Optionally, the steps for performing 3D modeling to form the target model include:

[0029] The original point cloud data is obtained by calculating the distance between the detection point and each pixel of the target object. Outliers in the original point cloud data are then filtered out to form the source point cloud.

[0030] The target point cloud is obtained according to the point cloud library. The multiple source point clouds and the target point cloud are initially registered. Each point in the source point cloud forms a matching point pair with the nearest point in the target point cloud. The transformation matrix is ​​calculated according to the matching point pair. The transformation matrix is ​​applied to the source point cloud. The source point cloud is gradually aligned with the target point cloud to form a modeling point cloud.

[0031] The modeled point cloud is reconstructed in three dimensions using a three-dimensional mesh algorithm.

[0032] Optionally, the steps of forming a matching point pair between each point in the source point cloud and its nearest point in the target point cloud, calculating a transformation matrix based on the matching point pair, applying the transformation matrix to the source point cloud, and gradually aligning the source point cloud with the target point cloud to form a modeling point cloud include:

[0033] Each of the source point clouds P = {p1, p2, ..., p} n A point pi in the target point cloud Q = {q1, q2, ..., q} is related to its nearest neighbor. n Points qi in the} form matching point pairs, and the rotation matrix R and translation vector t are calculated based on the matching point pairs;

[0034] The rotation matrix R and translation vector t are applied to the source point cloud to update the position of the source point cloud;

[0035] Repeat the above steps until the second error function is within the second preset range.

[0036] Optionally, the step of performing 3D reconstruction of the modeled point cloud using a 3D mesh algorithm includes:

[0037] Three points are selected from the modeled point cloud to form an initial triangle;

[0038] Add the remaining points one by one, and maintain the empty circle property by flipping the edges.

[0039] The present invention also provides a three-dimensional radar vision inspection system, which uses the above-described three-dimensional radar vision inspection method and includes:

[0040] A radar module is used to transmit and receive radar signals to obtain measured radar waveforms.

[0041] The point cloud acquisition module is used to transmit and receive the first electromagnetic signal to obtain raw point cloud data.

[0042] The storage and processing module is used for 3D modeling, electromagnetic simulation, data comparison, and iterative backtracking.

[0043] The beneficial effects of the three-dimensional radar visual inspection method and system provided by this invention are as follows: Compared with the prior art, the three-dimensional radar visual inspection method of this invention performs two-dimensional image detection on the target object, initially determines the material properties of the target object, and sends a first electromagnetic wave to the target object to calculate the distance between the detection point and each pixel of the target object, thus performing three-dimensional modeling; then, based on the determined material properties, material parameters are assigned to the target model, and electromagnetic wave propagation simulation is performed to obtain a simulated radar waveform; ray tracing measurement is performed on the target object using radar to obtain a measured radar waveform; the simulated radar waveform and the measured radar waveform are compared; if the difference between the two is within a first preset range, then the material of the target object matches the material corresponding to the material parameter assignment; if the difference is still within the first preset range, the material parameters are updated and the above steps are repeated until the difference between the two waveforms is within the first preset range. The above detection method can not only obtain the three-dimensional morphological information of the target object, but also determine the surface material properties of the target object, overcoming the limitations of existing technology systems and providing comprehensive object recognition and material analysis capabilities. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating the steps of the three-dimensional radar visual inspection method provided in an embodiment of the present invention.

[0046] Figure 2 This is a flowchart illustrating the three-dimensional radar visual inspection method provided in an embodiment of the present invention. Detailed Implementation

[0047] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0048] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0049] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0050] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0051] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0052] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0053] The three-dimensional radar visual inspection method provided in this embodiment of the invention will now be described. The three-dimensional radar visual inspection method is used to detect the three-dimensional geometry and material of a target object, thereby enabling visual inspection of the target object and more accurately determining its shape, material, and category.

[0054] Please refer to the following: Figure 1 and Figure 2 The three-dimensional radar visual inspection method includes the following steps:

[0055] S10: Perform two-dimensional image detection on the target object, make a preliminary judgment on the material properties of the target object, and establish a material library and a point cloud library;

[0056] S20: First electromagnetic signals are emitted towards the target object at multiple detection points. The first electromagnetic signals are received after being reflected by the target object. The distance between the detection points and each pixel of the target object is calculated by the emission and reception time of the first electromagnetic signals, and a three-dimensional model is formed to create the target model.

[0057] S30: Assign material parameters to the target model based on the material library, and perform electromagnetic wave propagation simulation to obtain the simulated radar waveform;

[0058] S40: Use radar to perform ray-tracking measurements on target objects and obtain measured radar waveforms;

[0059] S50: Compare the simulated radar waveform with the measured radar waveform: If the difference between the simulated radar waveform and the measured radar waveform is within a first preset range, then the material of the target object is the material corresponding to the material parameter assignment; if the difference between the simulated radar waveform and the measured radar waveform is outside the first preset range, then update the material parameters of the target model, obtain the simulated radar waveform again, and repeat this step.

[0060] In this context, the target object is the object being detected, and the three-dimensional radar visual detection method is used to perform visual detection on the target object.

[0061] In S10, 2D image detection involves capturing images of the target object's surface using a camera and classifying the object's type. Specifically, 2D image detection can detect and identify the target object's appearance features, including parameters such as shape, texture, and color, and use the identified results to infer the object's material category or type. Through this process, the material and type of the target object can be preliminarily determined, forming a corresponding material library and point cloud library, providing a reference for subsequent material assignment in electromagnetic simulations. For example, 2D image detection can preliminarily determine that the target object's material is a cube with a metallic surface, and the established material library includes parameters for various metallic materials, while the suggested point cloud library consists of cubes with various structural shapes.

[0062] In S20, by detecting the propagation time of the first electromagnetic signal, the transmission distance of the first electromagnetic signal can be calculated, and then the distance between each pixel on the surface of the target object and the detection point can be obtained, and thus the coordinates of each pixel on the surface of the target object can be obtained.

[0063] In step S30, when performing electromagnetic wave propagation simulation on the target object, the setting of the target object's material parameters is crucial. Therefore, assigning values ​​to the material parameters according to the material library in step S10 helps to narrow down the range of material selection and improve simulation and detection efficiency.

[0064] In step S40, the radar sends a radar signal to the target object and receives the signal reflected back by the target object. The echo signal is then processed to obtain the measured radar waveform.

[0065] In step S50, if the difference between the simulated radar waveform and the measured radar waveform is outside the first preset range, then steps S30 and S50 are repeated for iterative iteration until the difference between the simulated radar waveform and the measured radar waveform is within the first preset range.

[0066] It should be noted that step S10 can be implemented between steps S20 and S30, or before step S20, and step S40 can be implemented before step S10, between steps S10 and S20, or between steps S20 and S30.

[0067] The three-dimensional radar vision detection method in the above embodiments performs two-dimensional image detection on the target object, initially determines the material properties of the target object, and sends a first electromagnetic wave to the target object to calculate the distance between the detection point and each pixel of the target object, thus performing three-dimensional modeling. Then, based on the determined material properties, material parameters are assigned to the target model, and electromagnetic wave propagation simulation is performed to obtain a simulated radar waveform. Ray tracing measurement is performed on the target object using radar to obtain a measured radar waveform. The simulated radar waveform and the measured radar waveform are compared. If the difference between the two is within a first preset range, the material of the target object matches the material corresponding to the assigned material parameters. If the difference is still within the first preset range, the material parameters are updated and the above steps are repeated until the difference between the two waveforms is within the first preset range. The above detection method can not only obtain the three-dimensional morphological information of the target object, but also determine the surface material properties of the target object, overcoming the limitations of existing technology systems and providing comprehensive object recognition and material analysis capabilities.

[0068] In some embodiments of the present invention, step S10 includes:

[0069] S11: Take a picture of the target object with a camera to obtain information about the appearance of the target object;

[0070] S12: Use image recognition algorithms to identify target objects; the recognition process can utilize deep learning models (such as convolutional neural networks, CNN) to extract features from the image.

[0071] S13: Based on the recognition results and feature matching algorithms, classify the object type, determine the material properties and category of the target object, and form the corresponding material library and point cloud library.

[0072] The camera used in the S11 can be an RGB camera with high resolution. The RGB camera can be a standalone camera or an RGB camera within a time-of-flight camera.

[0073] In some embodiments of the present invention, in step S20, a first electromagnetic signal is emitted to the target object at multiple detection points, and the first electromagnetic signal is received after being reflected by the target object. The specific steps for calculating the distance between the detection points and each pixel of the target object based on the emission and reception time of the first electromagnetic signal are as follows:

[0074] The point cloud acquisition module is located at the detection point. It emits a first electromagnetic signal towards the target object, which is reflected by the object and then received by the module. The distance between the detection point and each pixel on the target object is then calculated. The time difference between the emission and reception of the first electromagnetic signal is t, and the distance between the detection point and the pixels on the target object is d, where d = c·t / 2. Thus, d can be calculated from the time difference t. Detecting each pixel on the target object generates a "depth map" composed of numerous points. These points represent the depth information of different surface regions of the target object. The "depth map" is essentially a set of point cloud data containing the spatial coordinates (x, y, z) of each point. These coordinates collectively describe the surface structure of the object. Based on this point cloud data, a 3D model is created to form the target model.

[0075] In some embodiments, the first electromagnetic signal may be infrared light, laser pulse, etc.

[0076] In some embodiments, the point cloud acquisition module may be a depth camera or the like, capable of transmitting and receiving a first electromagnetic signal.

[0077] In some embodiments, the point cloud acquisition module further includes an RGB camera, which can be used in step S10 to perform two-dimensional image detection on the target object.

[0078] In some embodiments of the present invention, step S20, which involves performing three-dimensional modeling to form the target model, includes:

[0079] S21: Obtain the original point cloud data by calculating the distance between the detection point and each pixel of the target object, filter out outliers in the original point cloud data, and form the source point cloud;

[0080] S22: Obtain the target point cloud from the point cloud library, perform preliminary registration of multiple source point clouds and target point clouds, form a matching point pair between each point in the source point cloud and its nearest point in the target point cloud, calculate the transformation matrix based on the matching point pair, apply the transformation matrix to the source point cloud, and gradually align the source point cloud with the target point cloud to form the modeling point cloud;

[0081] S23: Perform 3D reconstruction of the modeled point cloud using a 3D mesh algorithm.

[0082] In step S21, the raw point cloud data can be understood as the "depth map" composed of countless points mentioned above. This is the initial point cloud data corresponding to the target object. The raw point cloud data may contain noise and erroneous data, so it needs to be processed to filter out outliers and form source point clouds before it can be used for 3D modeling. In order to generate a complete 3D model, the target object needs to be scanned from multiple angles (the point cloud acquisition module emits the first electromagnetic signal toward the target object at multiple positions) to form multiple "depth maps". After filtering outliers and other processing, multiple source point clouds are formed.

[0083] In step S22, the target point cloud can be a point cloud related to material simulation or other known models. Based on preliminary 2D image recognition results or certain feature information, the possible set of target point clouds can be narrowed down first, i.e., the point cloud library in step S10 can be used. For example, the object can be inferred to belong to a certain type of material or shape through 2D image recognition, and then the standard point cloud of this type can be selected from the point cloud library as the target point cloud. Since multiple source point clouds are generated by scanning the target object from various angles, it is necessary to register multiple source point clouds. This can be achieved by finding and calculating the optimal transformation matrix, and then performing rigid transformations such as rotation and translation on the source point clouds to minimize the alignment error of each group of source point clouds.

[0084] In some embodiments, step S21 includes:

[0085] Calculate the source point cloud P = {p1, p2, ..., p...} n The neighborhood of point pi within a predetermined radius in};

[0086] Calculate the distance from all points in the neighborhood to the center point p of that neighborhood. i average distance d i ;

[0087] The average distance d of all points within the source point cloud i Statistical analysis was performed to obtain the mean distance μ and standard deviation σ of the source point cloud;

[0088] If |d i -u|>k·σ, then point p i Point p is an outlier and needs to be filtered out. i If |d i -u|≤k·σ, then point p i Since it is not an outlier, retain point p. i Where k is a user-defined parameter that can be set and selected according to actual needs.

[0089] In some embodiments, in step S22, multiple source point clouds and target point clouds are initially registered. Specifically, the initial registration can be performed by initial position estimation or by using feature point alignment. The initial registration can ensure that the distance between the source point cloud and the target point cloud is within a reasonable range so that subsequent iterations can converge effectively.

[0090] In some embodiments, step S22, where each point in the source point cloud forms a matching point pair with its nearest point in the target point cloud, a transformation matrix is ​​calculated based on the matching point pair, the transformation matrix is ​​applied to the source point cloud, and the source point cloud is gradually aligned with the target point cloud to form a modeling point cloud, includes the following steps:

[0091] S221: Each source point cloud P = {p1, p2... p...} n In the context of a point pi, its nearest target point cloud Q = {q1, q2, ..., q} is... n Point q in} i Form matching point pairs, and calculate the rotation matrix R and translation vector t based on the matching point pairs;

[0092] S222: Apply the rotation matrix R and translation vector t to the source point cloud to update the position of the source point cloud;

[0093] S223: Repeat the above steps until the second error function is within the second preset range.

[0094] For each point in the source point cloud, the nearest point in the target point cloud is found, forming many matching point pairs. For each matching point pair, a rotation matrix R and a translation vector t can be calculated. These are then applied to the source point cloud to make the positions of each point in the source point cloud approximate those in the target point cloud. After each rotation and translation, the positions of each point in the source point cloud are updated, and steps S221 and S222 are repeated until the second error function falls within a second preset range. During this process, the source point cloud gradually aligns with the target point cloud to form a modeled point cloud. If the number of iterations exceeds the maximum, it indicates an error in the matching point pairs, requiring rematching and repeating steps S221 to S223. The second preset range is a pre-defined range that can be determined based on the required detection accuracy.

[0095] The rotation matrix R is a 3×3 rotation matrix used to describe rotation operations in three-dimensional space. Each column or row of the rotation matrix R (depending on the construction method) represents the direction of the X, Y, and Z axes in the new coordinate system. The rotation matrix R preserves the shape and size of the source point cloud, changing only its orientation. The translation vector t is a 3×1 translation vector used to describe translation operations in three-dimensional space.

[0096] In some embodiments, step S23 includes:

[0097] S231: Select three points from the modeled point cloud to form an initial triangle;

[0098] S232: Add the remaining points one by one, and maintain the empty circle property by flipping the edges.

[0099] After point cloud processing in step S22, a modeling point cloud is formed. Then, a triangular mesh algorithm is used to reconstruct the 3D modeling point cloud. Specifically, triangulation is used to connect discrete points into triangular patches to form a smooth 3D model.

[0100] The empty circle property states that in every triangle generated, its circumcircle does not contain any other vertices. This property ensures the stability and quality of triangulation. Edge flipping is a commonly used method to restore the empty circle property. Specifically, suppose two adjacent triangles ΔABC and ΔABD share an edge AD, with vertices A, B, C, and D. Calculate the circumcircles of the two triangles sharing the edge AB and determine if these two circles contain each other's vertices (e.g., C or D). If the circumcircle of one triangle contains a non-shared vertex of the other triangle, then the pair of triangles does not meet the Denoné condition. To restore the empty circle property, replace the current shared edge AB with the connecting line CD. This replacement transforms the original two triangles ΔABC and ΔABD into two new triangles ΔACD and ΔBCD. After edge flipping, the circumcircles of the new two triangles should satisfy the empty circle property.

[0101] For a triangle ΔABC, with vertices A(x1,y1), B(x2,y2), and C(x3,y3), the center o(x,y) and radius R of its circumcircle can be calculated using the following formula:

[0102]

[0103] To quickly determine a point P(x) p ,y p Whether a triangle is inside its circumcircle can be determined using the following determinant:

[0104]

[0105] If the value of the determinant is less than zero, it means that point P is inside the circumcircle and needs to be adjusted; otherwise, it means that the triangle satisfies the empty circle property.

[0106] In some embodiments of the present invention, step S30 includes:

[0107] S31: Import the target model into the electromagnetic simulation software;

[0108] S32: Assign material parameters to the surface of the target model. The material used for assigning material parameters is a material from the material library.

[0109] S33: Using a ray tracing algorithm, the propagation of electromagnetic waves on the surface and inside the target model is calculated to obtain the simulated radar waveform.

[0110] In step S31, the target model is typically a polygonal mesh (e.g., STL format). However, electromagnetic simulation software often requires more optimized geometric models, such as CAD models (STEP, IGES formats), for efficient computation. Therefore, the target model (polygonal mesh model) needs to be converted to a CAD model before proceeding to step S30. For example, software tools (such as Geomagic, MeshLab) can be used to convert polygonal meshes to CAD models. These tools simplify the mesh data and reconstruct the surface to make it suitable for simulation applications.

[0111] Electromagnetic simulation software (such as Ansys HFSS, CST Microwave Studio, etc.) typically supports the import of various 3D model formats, including STEP, IGES, STL, etc. After the geometry of the model is assigned, the final model can be directly imported into these simulation software programs.

[0112] In step S32, material parameters are assigned to the surface of the target model based on the material library obtained from the two-dimensional image recognition. These material parameters may include key parameters such as dielectric constant, conductivity, and magnetic permeability. For example, the type of material of the target object can be preliminarily identified from the two-dimensional image, such as identifying the target object as metal, wood, or plastic, etc. Then, when assigning material parameters, the material parameters are assigned according to the corresponding material category.

[0113] In step S33, ray tracing is a technique used to simulate the interaction between electromagnetic waves and objects. It simulates the propagation of radar waves in three-dimensional space by tracing the path of rays. The core idea of ​​ray tracing is based on the principles of geometric optics, treating electromagnetic waves as rays and considering the reflection, refraction, and absorption effects of these rays on the object's surface. The propagation of electromagnetic waves in a medium follows Maxwell's equations, where wave reflection and refraction can be calculated using Fresnel equations. Using ray tracing algorithms to calculate the propagation of electromagnetic waves within the target model and its interior to obtain the simulated radar waveform is a conventional technique and will not be elaborated upon here.

[0114] In some embodiments of the present invention, in step S40, the target object is measured by ray tracing using radar to obtain the measured radar waveform.

[0115] The radar can employ frequency-modulated continuous wave (FMCW) radar technology, operating in the millimeter-wave band, to achieve high-precision detection of target objects. The radar includes a transmitting module for transmitting radar waves whose frequency varies over time (such as frequency-modulated continuous waves); a receiving module for receiving radar waves reflected from the target object; and a signal processing module for processing the received signals and generating radar waveforms.

[0116] The signal emitted by the radar transmitting module is a frequency-modulated continuous wave (Chirp) signal, whose frequency increases linearly from the initial frequency to the maximum frequency within a certain time period (called the Chirp period).

[0117] First, the synthesizer generates a chirp signal, which is then transmitted through the TX antenna (transmitting module). At the same time, a data copy is sent to the mixer. After being reflected by the target object, the echo signal is received through the RX antenna (receiving module) and used to generate an intermediate frequency signal in the mixer.

[0118] The radar then mixes the transmitted and received signals to obtain a signal containing an intermediate frequency (IF). An analog-to-digital converter (ADC) is used to sample this IF signal, yielding a discrete time series. To reduce noise interference, preprocessing can be performed, using a low-pass filter to remove high-frequency noise or a band-pass filter to focus on specific frequency bands. Window functions such as the Hanning or Heyman window are applied to reduce spectral leakage and improve the resolution of the spectral analysis. An effective fractional-four (FFT) is then performed on the preprocessed IF signal to obtain the amplitude and phase information in the frequency domain.

[0119] In some embodiments of the present invention, step S50, which compares the simulated radar waveform with the measured radar waveform, includes:

[0120] S51: Both the simulated radar waveform and the measured radar waveform are converted to the frequency domain. By calculating the spectrum of the simulated radar waveform and the spectrum of the measured radar waveform, the amplitude difference and phase difference between the simulated radar waveform and the measured radar waveform are obtained.

[0121] S52: Determine whether the difference between the simulated radar waveform and the measured radar waveform is within the first preset range based on the amplitude difference and phase difference.

[0122] By comparing the amplitude and phase differences between the simulated radar waveform and the measured radar waveform, it can be determined whether the material parameters assigned to the target model are accurate. If they are inaccurate, the material parameters need to be updated, and the target model needs to be subjected to electromagnetic simulation again to obtain the simulated radar waveform, which is then compared with the measured radar waveform.

[0123] In some embodiments, step S51 includes:

[0124] The spectrum of the simulated radar waveform is X. sim (f) The spectrum of the measured radar waveform is X. meas (f);

[0125] The difference in amplitude is Among them, A sim (f i )=|X sim (f i ), and A sim (f) represents the amplitude spectrum of the simulated radar waveform, A meas (f i )=|X meas (f i )|, and A meas (f) is the amplitude spectrum of the measured radar waveform;

[0126] Phase difference is Where, θ sim (f) is the phase spectrum of the simulated radar waveform, and θ meas (f) is the phase spectrum of the measured radar waveform.

[0127] Thus, step S51 can obtain the amplitude and phase differences between the simulated radar waveform and the measured radar waveform. When comparing the simulated radar waveform and the measured radar waveform, we are actually comparing the amplitude and phase differences between the two.

[0128] In some embodiments, step S52 includes:

[0129] Calculate the maximum value E of all amplitude differences. mag,max and minimum value E mag,min And calculate the maximum value of the phase difference E phase,max and minimum value E phase,min ,

[0130] Normalize the amplitude difference and phase difference to obtain the amplitude normalization. and phase normalization

[0131] The comprehensive error was calculated. Where α and β are weighting coefficients, and the first preset range is E. total ≤M.

[0132] Understandably, the overall error E total When M ≤ M, the difference between the simulated radar waveform and the measured radar waveform is within the first preset range, and the comprehensive error E total When M > 0, the difference between the simulated radar waveform and the measured radar waveform is outside the first preset range. Here, M is a known quantity, which can be between 0.05 and 0.1.

[0133] In some embodiments of the present invention, in step S50, when the difference between the simulated radar waveform and the measured radar waveform is outside a first preset range, it is necessary to update the material parameters of the target model. This can also be understood as needing to optimize the material parameters of the target model so that the simulated radar waveform gradually approaches the measured radar waveform.

[0134] In some embodiments, gradient descent is used to optimize simulation parameters. Gradient descent is an optimization algorithm that updates parameters based on the gradient of the loss function. The goal of gradient descent is to find the optimal material parameters by minimizing the error function, thereby minimizing the difference between the simulated radar waveform and the measured radar waveform.

[0135] In some embodiments, the step of updating the material parameters of the target model includes:

[0136] The current material parameters are The updated material parameters are as follows Where η is the learning rate, used to control the magnitude of each update. The gradient of the first error function is obtained by minimizing the first error function. Update simulation parameters Reduce the difference between simulated radar waveforms and measured radar waveforms.

[0137] The above optimization process will continue to iterate until one of the following stopping conditions is met:

[0138] When E total When M ≤ M, the simulated radar waveform is consistent with the measured radar waveform.

[0139] If the preset maximum number of iterations is reached, the optimization process stops, even if the error has not fully converged and the simulated radar waveform is inconsistent with the measured radar waveform. In this case, the material parameters need to be updated (by changing the type of material).

[0140] If the parameter changes are small with each update, it indicates that the optimization process has stabilized and can be stopped.

[0141] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0142] The present invention also provides a three-dimensional radar visual inspection system, using the three-dimensional radar visual inspection method in any of the above embodiments. The three-dimensional radar visual inspection system includes:

[0143] A radar module is used to transmit and receive radar signals to obtain measured radar waveforms.

[0144] The point cloud acquisition module is used to transmit and receive the first electromagnetic signal to obtain the source point cloud;

[0145] The storage and processing module is used for 3D modeling, electromagnetic simulation, data comparison, and iterative backtracking.

[0146] The three-dimensional radar vision inspection system provided by this invention can not only acquire the three-dimensional morphological information of the target object, but also determine the surface material properties of the target object, overcoming the limitations of existing technology systems and providing comprehensive object recognition and material analysis capabilities.

[0147] In some embodiments, the radar module acquires measured radar signals of a target object by transmitting and receiving radar signals. The radar module may include a transmitting module (such as an RF transmitter) and a receiving module (such as an RF receiver). The transmitting module is capable of transmitting frequency-modulated continuous waves or other forms of radar signals, and the receiving module is capable of capturing the echo signal of the target object. After measuring the echo from the target object, the radar module converts the echo signal into an intermediate frequency signal and then into a digital signal via an analog-to-digital converter, before transmitting it to the computer system module via a USB or Ethernet interface. This process ensures accurate signal acquisition, providing accurate radar data for subsequent comparative analysis.

[0148] In some embodiments, the point cloud acquisition module is used to capture the three-dimensional structural information of the target object and generate high-precision raw point cloud data. The point cloud acquisition module actively emits infrared light or other light sources and uses time-of-flight (TOF) measurement technology to obtain a depth map. In addition to 3D modeling, the point cloud acquisition module can acquire two-dimensional RGB images of the target for preliminary identification of the target object's surface material. The point cloud acquisition module transmits the raw point cloud data and RGB image data to the computer system module via a high-speed interface (such as USB or Camera Link), ensuring efficient and accurate data transmission.

[0149] In some embodiments, the storage processing module includes a 3D modeling module, a material parameter assignment module, an electromagnetic simulation module, a data comparison module, and an iterative backtracking module.

[0150] The 3D modeling module receives raw point cloud data from the point cloud acquisition module and uses point cloud processing algorithms to convert it into a 3D model file of the target object. The 3D modeling process includes steps such as point cloud denoising, point cloud registration, and surface reconstruction to ensure the integrity and accuracy of the model. The 3D model file will then be further imported into the electromagnetic simulation module.

[0151] The material parameter assignment module processes the 2D RGB image detected by the 2D graphics, identifies the type of the target object (such as metal, plastic, etc.) using computer vision algorithms, and assigns preliminary material parameter values ​​to the target material properties based on the identification results. These assigned material properties will serve as the initial conditions for the simulation. This module can use pre-trained image recognition models or deep learning algorithms to improve the accuracy of material parameter assignment.

[0152] The electromagnetic simulation module takes the 3D model generated by the 3D modeling module and the initial material values ​​assigned by the material parameter assignment module as input, and simulates material properties using the ray tracing method. Ray tracing is used to simulate the propagation and reflection of electromagnetic waves on and within the surface of a target object, generating a simulated radar waveform of the target object (such as reflection coefficient, scattering coefficient, etc.). The output of the simulated radar waveform is then transmitted to the data comparison module for subsequent analysis.

[0153] The data comparison module receives measured radar waveforms from the radar module and simulated radar waveforms from the electromagnetic simulation module, and compares them using signal analysis algorithms (such as spectrum analysis and signal feature matching). The comparison result can be the degree of consistency or similarity score between the two. If the comparison result reaches a first preset range, the final material identification result is output; otherwise, an iterative backtracking process is entered to further optimize the material assignment.

[0154] When the data comparison module detects an inconsistency between the measured radar waveform and the simulated radar waveform, the iterative backtracking module will initiate the backtracking process. This module uses a feedback mechanism to transmit the inconsistency information to the material parameter assignment module, adjusts the material property assignments, and re-performs the material property simulation and data comparison until a simulation result with consistent waveforms is obtained.

[0155] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A three-dimensional radar vision detection method, characterized in that, The method comprises the following steps: two-dimensional image detection is performed on the target object, the material properties of the target object are preliminarily judged, a material library and a point cloud library are established; a first electromagnetic signal is emitted to the target object at multiple detection points, the first electromagnetic signal is received after being reflected by the target object, the distance between the detection points and each pixel point of the target object is calculated through the emission and reception time of the first electromagnetic signal, and a three-dimensional model is formed to form a target model; material parameters of the target model are assigned according to the material library, electromagnetic wave propagation simulation is performed, and a simulation radar waveform is obtained; ray tracing measurement is performed on the target object by radar, and a measured radar waveform is obtained; the simulation radar waveform and the measured radar waveform are compared: if the difference between the simulation radar waveform and the measured radar waveform is within a first preset range, the material of the target object is the corresponding material during the material parameter assignment; if the difference between the simulation radar waveform and the measured radar waveform is outside the first preset range, the material parameters of the target model are updated, the simulation radar waveform is obtained again, and the step is repeated.

2. The three-dimensional radar vision inspection method of claim 1, wherein, The step of assigning material parameters to the target model according to the material library and performing electromagnetic wave propagation simulation to obtain a simulation radar waveform comprises: the target model is imported into electromagnetic simulation software; material parameters are assigned to the surface of the target model, and the corresponding material during the material parameter assignment is the material in the material library; the electromagnetic wave propagation on the surface and inside of the target model is calculated through a ray tracing algorithm to obtain a simulation radar waveform.

3. The three-dimensional radar vision inspection method of claim 1, wherein, The step of comparing the simulation radar waveform and the measured radar waveform comprises: the simulation radar waveform and the measured radar waveform are converted into frequency domain, the amplitude difference and the phase difference between the simulation radar waveform and the measured radar waveform are obtained by calculating the frequency spectrum of the simulation radar waveform and the frequency spectrum of the measured radar waveform; whether the difference between the simulation radar waveform and the measured radar waveform is within a first preset range is determined according to the amplitude difference and the phase difference.

4. The three-dimensional radar vision inspection method of claim 3, wherein, The step of calculating the frequency spectrum of the simulation radar waveform and the frequency spectrum of the measured radar waveform to obtain the amplitude difference and the phase difference between the simulation radar waveform and the measured radar waveform comprises: The spectrum of the simulated radar waveform is , and the spectrum of the measured radar waveform is ; said amplitude difference is wherein and is an amplitude spectrum of the simulated radar waveform, and is an amplitude spectrum of the measured radar waveform; The phase difference is wherein, is a phase spectrum of the simulated radar waveform, and is a phase spectrum of the measured radar waveform.

5. The three-dimensional radar vision inspection method of claim 3, wherein, The step of determining whether the difference between the simulation radar waveform and the measured radar waveform is within a first preset range according to the amplitude difference and the phase difference comprises: calculating a maximum value Emag,max and a minimum value Emag,min of all amplitude differences and a maximum value Ephase,max and a minimum value Ephase,min of the phase differences, normalizing the amplitude differences and the phase differences to obtain amplitude normalized and phase normalized ; The integrated error is calculated wherein, α and β is a weight coefficient, the first preset range is , M between 0.05 and 0.

1.

6. The three-dimensional radar vision inspection method of any one of claims 1-5, wherein, The step of updating the material parameters of the target model comprises: The current material parameters are φ old The updated material parameters are φ new , wherein η is a learning rate for controlling the magnitude of each update, is a gradient of the first error function, by minimizing the first error function updates the simulation parameters φ to reduce the difference between the simulated radar waveform and the measured radar waveform.

7. The three-dimensional radar vision inspection method of any one of claims 1-5, wherein, The step of forming a three-dimensional model to form a target model comprises: raw point cloud data is obtained by calculating the distance between the detection points and each pixel point of the target object, outliers in the raw point cloud data are filtered out, and source point cloud is formed; According to the point cloud library, a target point cloud is obtained, a plurality of source point clouds and the target point cloud are preliminarily registered, a point in each source point cloud and a point closest to it in the target point cloud form a matched point pair, a transformation matrix is calculated according to the matched point pair, the transformation matrix is applied to the source point cloud, and the source point cloud is gradually aligned with the target point cloud to form a modeling point cloud; The modeling point cloud is reconstructed in three dimensions through a three-dimensional mesh algorithm.

8. The three-dimensional radar vision inspection method of claim 7, wherein, The step of forming a modeling point cloud by aligning each point in the source point cloud with the closest point in the target point cloud to form a matched point pair, calculating a transformation matrix according to the matched point pair, applying the transformation matrix to the source point cloud, and gradually aligning the source point cloud with the target point cloud includes: Each of the aforementioned source clouds Points in The nearest target point cloud Points in Form matching point pairs, and calculate the rotation matrix based on the matching point pairs. R Translation vector t ; applying a rotation matrix R and a translation vector t to the source point cloud, updating a position of the source point cloud; The above steps are repeated until a second error function is within a second pre-set range, the second error function .

9. The three-dimensional radar vision inspection method of claim 7, wherein, The step of reconstructing the modeling point cloud in three dimensions through a three-dimensional mesh algorithm includes: Three points are selected from the modeling point cloud to form an initial triangle; The remaining points are added one by one, and the edge flipping is used to maintain the empty circle property.

10. A three-dimensional radar vision detection system using the three-dimensional radar vision detection method according to any one of claims 1 to 9, characterized in that, It includes: A radar module for transmitting and receiving radar signals to obtain a measured radar waveform; A point cloud acquisition module for transmitting and receiving a first electromagnetic signal to obtain raw point cloud data; A storage processing module for three-dimensional modeling, electromagnetic simulation, data comparison, and iterative backtracking.

Images