Method and system for rapid visual positioning assisted by local radar scatter imaging diagnostic device

Abstract

Disclosed is a local radar scattering imaging diagnostic device assisted rapid visual positioning system, characterized in that the system comprises a local radar scattering imaging diagnostic device, a camera module, a feature extraction detection module, a positioning judgment module and a display module. A local radar scattering imaging diagnostic device assisted rapid visual positioning method is also provided. The method extracts the feature of a mark point of a reference position image and a current position image of the reference position captured by a camera through a deep learning network, obtains a visual positioning point for matching calibration, compares the current visual positioning point with the reference positioning point, continuously adjusts the movement of the local radar scattering imaging diagnostic device, and restores the current measurement position to the reference position, thereby ensuring the consistency of measurement, and realizing the local radar scattering imaging diagnostic device assisted rapid visual positioning based on artificial intelligence.

Description

Technical Field

[0001] This invention belongs to the field of near-field radar scattering imaging measurement and target radar scattering feature diagnosis, specifically involving a method and system for rapid visual positioning assisted by a local radar scattering imaging diagnostic device based on artificial intelligence and machine vision technology. Background Technology

[0002] Currently, low radar scattering characteristics have become a hallmark of advanced weaponry. Numerous weapons systems, such as stealth aircraft, missiles, armored tanks, and warships, employ stealth technologies, including precision-fitted seams, radar-absorbing coatings / honeycomb structures, and electromagnetic shielding films. However, during the use of stealth equipment, environmental and human factors such as collisions, scratches, natural aging, and salt spray corrosion can lead to structural damage, such as step differences, or degradation of stealth material performance. Local radar scattering imaging diagnostic equipment utilizes near-field imaging technology to perform microwave imaging on the target. By analyzing the scattering highlights in the microwave image, it obtains the scattering distribution characteristics of the target and diagnoses whether the low radar detectability of the stealth equipment is intact. However, near-field microwave imaging is highly sensitive to measurement distance and azimuth angle (horizontal and elevation). To compare the relative changes in radar scattering characteristics of the measured area with its original state, radar scattering imaging diagnostic equipment must ensure, as accurately as possible, that both measurements are taken at the same measurement distance and azimuth angle during microwave imaging of the target. This improves the consistency of the measurement results and ensures the reliability of the diagnostic results.

[0003] Current radar scattering imaging diagnostic equipment mainly uses laser positioning technology, which can achieve high-precision positioning. It determines the measurement distance and azimuth angle through precise laser imaging technology, or uses relatively complex marking methods for the target. Although these positioning technologies can provide accurate position and orientation information, they have problems such as long positioning time, cumbersome positioning process, and complex and expensive equipment. They are no longer suitable for radar scattering imaging diagnostic scenarios that need to be conducted in confined spaces, with high frequency of use and time-sensitive testing. A rapid positioning method that balances accuracy, efficiency and cost is needed. Therefore, rapid visual positioning of portable local radar scattering imaging diagnostic equipment is a key technical issue for the reliable use of radar scattering imaging diagnostic equipment. Summary of the Invention

[0004] This invention addresses the shortcomings of the prior art by providing an artificial intelligence-based method for assisting rapid visual localization of local radar scattering imaging diagnostic equipment, enabling rapid and accurate local radar scattering imaging diagnostic equipment localization. The method includes the following steps:

[0005] A rapid visual positioning system assisted by a local radar scattering imaging diagnostic device is characterized in that the system comprises a local radar scattering imaging diagnostic device, a camera module, a feature extraction and detection module, a positioning judgment module, and a display module; wherein...

[0006] The local radar scattering imaging diagnostic equipment adopts a planar near-field scanning frame type radar scattering imaging diagnostic equipment to obtain the radar scattering image of the target under test in a certain angular domain, and then determines the distribution of the strong scattering area of ​​the target under test based on the scattering image; the local radar scattering imaging diagnostic equipment and the target under test maintain a relatively fixed position;

[0007] The camera module is located at a fixed position on the local radar scattering imaging diagnostic equipment. It is used to take pictures of the target under test, acquire visual images of the target under test, and output them to the feature extraction and detection module.

[0008] The feature extraction and detection module uses the YOLOv5s detection model and the CSPDarknet53 feature extraction network to extract features from the image received from the camera module. It receives the visual image information of the target position output by the camera module and extracts the second contour line and second key point information of the visual positioning reference part in the image acquired by the camera module in real time to guide the movement adjustment of the position and attitude of the local radar scattering imaging diagnostic device.

[0009] The positioning and judgment module receives the second contour line and the second key point information output by the feature extraction and detection module in real time. Based on the first contour line and the first key point information, it determines whether the device has reached the reference position. Once it has, the positioning and matching is completed.

[0010] The display module receives positioning matching information from the positioning judgment module and displays in real time the second contour line and second key point of the reference part of the target under test in the image captured by the camera module, as well as the first contour line and first key point of the reference part when the target under test is in the reference position. In addition, the display module also displays the visual image and positioning matching result when the target under test is in the reference position.

[0011] In one embodiment of the present invention, a camera module is fixed to the front of the shock-absorbing cabinet, a display module is installed on the upper part of the shock-absorbing cabinet, and a feature extraction and detection module and a positioning judgment module are integrated in the industrial control computer; at the optimal diagnostic position of the local radar scattering imaging diagnostic equipment, an optical image of the target under test is captured and used as a reference position image.

[0012] The present invention also provides a rapid visual positioning method assisted by a local radar scattering imaging diagnostic device. This method, based on the aforementioned rapid visual positioning system assisted by a local radar scattering imaging diagnostic device, specifically includes the following steps:

[0013] S1. A camera module is fixed at a fixed position on a local radar scattering imaging diagnostic device to obtain optical images; the images captured by the camera are displayed in real time using a display module.

[0014] S2. Based on the requirements of measuring the relative position of the local radar scattering imaging diagnostic equipment and the target under test for measuring the local radar scattering image, determine the reference position of the target under test relative to the local radar scattering imaging diagnostic equipment, manually move the local radar scattering imaging diagnostic equipment to the reference position, and use the onboard camera to take a picture of the target under test to obtain the reference position image of the target under test.

[0015] S3. Visually observe and analyze the reference position image of the target being measured, identify the parts with clear outlines as reference parts for visual positioning matching, and mark the outline edges as the first outline line and the first outline line as the first color; calculate the center point of the reference part, which is the first key point, through the circumscribed rectangle of the first outline line; the four sides of the circumscribed rectangle are parallel to the four sides of the image and tangent to the first outline line, and the coordinates of the first key point are calculated by the following formula, and the first key point is marked as the first color; fix the first outline line and the first key point on the display;

[0016]

[0017] Where (x, y) are the coordinates of the first key point, and (xmin, ymin) are the coordinates of the top-left corner of the circumscribed rectangle. max (ymax) represents the coordinates of the bottom right corner of the circumscribed rectangle;

[0018] S4. By moving the local radar scattering imaging diagnostic equipment forward, backward, left, and right, and rotating it, the position of the camera module relative to the target under test is changed, and visual images of the target under test at different angles and distances are acquired. During the acquisition process, it is necessary to ensure that the reference part for visual positioning is always within the camera's field of view.

[0019] S5. Using the optical image dataset of the target object acquired by the camera module, train the PSPNet semantic segmentation model and the YOLOv5s object detection model. During model training, the Adam optimization algorithm is used to optimize the model, and the learning rate is adjusted using simulated cosine annealing. The trained optimal PSPNet semantic segmentation model can detect the contour line of the reference part of the target object, i.e., the second contour line, and mark the second contour line with a second color, which is different from the first color. The trained optimal YOLOv5s object detection model can detect the center point of the reference part of the target object, i.e., the second key point, and mark the second key point with a second color. The second contour line and the second key point are then displayed on the display.

[0020] S6. Turn on the camera module and simultaneously move the local radar scattering imaging diagnostic device to the vicinity of the reference measurement position of the target. Observe the display module. When a complete second contour line of the visual positioning reference part appears on the display module, observe the first contour line on the display module. Move and adjust the local radar scattering imaging diagnostic device. After the second contour line and the first contour line basically coincide in the display module, determine whether the current position is basically matched with the reference position by the intersection-union ratio (IoU) of the areas enclosed by the second contour line and the areas enclosed by the first contour line. When the IoU is greater than the value A, it is determined that the current position is basically matched with the reference position. Then, fine-tune the local radar scattering imaging diagnostic device and determine whether the current position is accurately matched with the reference position by calculating the Euclidean distance between the second key point and the first key point. When the Euclidean distance between the three pairs of key points is less than the specified value, it is determined that the position is accurately matched, and the screen will automatically prompt that the positioning is matched, thus completing the accurate positioning.

[0021] In one embodiment of the present invention, the PSPNet semantic segmentation network consists of a backbone network, an enhanced feature extraction structure, and a head network. The input image is first processed by the backbone network to extract features and obtain a feature map. Then, the enhanced feature extraction structure further aggregates the features to enrich the feature information and improve the ability of the feature map to represent global information. Finally, the head network integrates and upsamples the features to achieve semantic segmentation of the image.

[0022] In a specific embodiment of the present invention, MobileNetV2 is selected as the backbone network of PSPNet, and ordinary convolution is replaced by depthwise separable convolution. In the inverse residual, the input image feature data is first enlarged using a 1×1 convolution, then the relevant features of the image feature data are further extracted using a 3×3 depthwise separable convolution, and finally the extracted features are reduced using a 1×1 convolution.

[0023] In another specific embodiment of the present invention, the backbone network of the YOLOv5s model adopts the CSPDarknet53 network. The CSPDarknet53 network consists of 6 modules, namely 3 "convolutional layer-batch normalization layer-activation function layer" modules and 3 residual modules. The "convolutional layer-batch normalization layer-activation function layer" module consists of a 3×3 convolutional layer, a batch normalization layer and a LeakyReLU activation function layer. The stride of the convolutional layer in the first two "convolutional layer-batch normalization layer-activation function layer" modules is 2, and the image is downsampled by sliding the convolutional kernel in the convolutional layer on the image.

[0024] In another specific embodiment of the present invention, the FPN consists of a "convolutional layer-batch normalization layer-activation function layer", an upsampling layer, a 1×1 convolutional layer, and a concatenation layer. The deep features extracted by the backbone network of the model are further integrated and upsampled after passing through the "convolutional layer-batch normalization layer-activation function layer". They are then stacked with the shallower features extracted by the backbone network in the concatenation layer. The integration and upsampling process is completed by the convolution operation of the convolutional layer. Stacking refers to the stacking of features in the channel dimension within the concatenation layer.

[0025] In another embodiment of the invention, the learning rate adjustment method uses simulated cosine annealing to adjust the learning rate.

[0026]

[0027] Among them, lr t To simulate cosine annealing for adjusting the learning rate, lr max The initial learning rate is lr min The minimum allowable learning rate, i.e., when the learning rate decays to lr min At that time, it will no longer continue to decay, but will remain at lr. min ,lr t Let T be the current learning rate, t be the current training epoch, and T be the current learning rate. max This refers to the learning rate decay period.

[0028] In yet another embodiment of the present invention, the model optimization algorithm adopts the Adam optimization method, as shown in the following formula;

[0029]

[0030] Where L is the loss function, w i Let b be the i-th weight parameter of the network. i Let be the i-th bias parameter of the network, t be the number of iterations, α be the learning rate, and β1 and β2 be exponential weighting parameters. In this invention, β1 = 0.9 and β2 = 0.999 are chosen. ε is used to prevent a small quantity with a denominator of zero; in this invention, ε = 1 × 10^25. -9 w i,t Let b be the i-th weight parameter of the network in the t-th iteration. i,t w is the i-th bias parameter of the network at the t-th iteration. i,t+1 Let b be the i-th weight parameter of the network in the (t+1)-th iteration. i,t+1 Let be the i-th bias parameter of the network at the (t+1)-th iteration. Let i be the second intermediate variable of the i-th weight parameter in the t-th iteration. Let i be the second intermediate variable of the i-th weight parameter in the (t-1)-th iteration. Let be the second intermediate variable of the i-th bias parameter in the t-th iteration. This is the second intermediate variable for the i-th bias parameter during the (t-1)-th iteration. Let i be the first intermediate variable of the i-th weight parameter in the t-th iteration. Let i be the first intermediate variable of the i-th weight parameter in the (t-1)-th iteration. Let i be the first intermediate variable of the i-th bias parameter in the t-th iteration. Let i be the first intermediate variable of the i-th bias parameter in the (t-1)-th iteration. Let be the third intermediate variable for the i-th weight parameter. Let be the third intermediate variable for the i-th bias parameter. Let be the fourth intermediate variable for the i-th weight parameter. It is the fourth intermediate variable for the i-th bias parameter; The superscript t indicates t raised to the power of t.

[0031] In another embodiment of the present invention, step S6 is specifically as follows:

[0032] S601. Continuously move and adjust the local radar scattering imaging diagnostic equipment to try to match the second contour line displayed on the screen in real time with the first contour line displayed at a fixed time. When the IoU is greater than 95%, it is determined that the position is basically matched.

[0033] The formula for calculating the IoU of the regions enclosed by the second contour line and the regions enclosed by the first contour line is:

[0034]

[0035] Among them, S g S represents the pixels in the area enclosed by the second contour line. r Represents the pixels in the area enclosed by the first outline;

[0036] S602. Calculate the Euclidean distance between the current second key point and the first key point. When the Euclidean distance between all three pairs of key points is less than the set error value, stop adjusting the local radar scattering imaging diagnosis. It is considered that the device has reached the reference position and the local radar scattering imaging diagnosis device is matched with the target being measured. The screen module displays "Location Matching".

[0037] Given the accuracy requirements for the error value, the formula for calculating the Euclidean distance between the current positioning point and the reference positioning point is as follows:

[0038]

[0039] In the formula, xi The x-coordinate of the second key point is represented by y. i The ordinate of the second key point, x i ' represents the x-coordinate of the first key point, y i ′ represents the ordinate of the first key point, and L represents the width of the image captured by the camera module.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] A rapid visual positioning method assisted by an AI-based local radar scattering imaging diagnostic device is proposed. This method employs a camera module fixed to the local radar scattering imaging diagnostic device to acquire optical images of a local area of ​​the aircraft, extracting the marker features of a reference image to detect a reference visual positioning point. When the aircraft moves to a new observation point, it only needs to compare the current visual positioning point with the reference positioning point, and then move and adjust the local radar scattering imaging diagnostic device to quickly and accurately match the device and restore it to the reference position. This visual positioning method is simple to operate, reducing the entire positioning time to approximately 10 seconds, and the impact of positioning accuracy on measurement consistency is within an acceptable range.

[0042] In summary, this invention extracts the marker features of the reference position image captured by the camera and the current position image through a deep learning network to obtain visual positioning points for matching and calibration. By comparing the current visual positioning point with the reference positioning point, the moving local radar scattering imaging diagnostic device is continuously adjusted to restore the current measurement position to the reference position, thereby ensuring measurement consistency and realizing rapid visual positioning assisted by artificial intelligence-based local radar scattering imaging diagnostic device. Attached Figure Description

[0043] Figure 1 Structural diagram of a rapid visual localization method to assist local radar scattering imaging diagnostic equipment;

[0044] Figure 2 Schematic diagram of a local radar scattering imaging diagnostic device and an auxiliary rapid visual positioning system;

[0045] Figure 3 Local scattering images obtained by local radar scattering imaging diagnostic equipment;

[0046] Figure 4 To display the visual image of the location matching;

[0047] Figure 5 To display visual images where the positioning does not match. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0050] This invention provides an artificial intelligence-based auxiliary rapid visual positioning system for local radar scattering imaging diagnostic equipment (hereinafter referred to as the "auxiliary rapid visual positioning system"). This system enables the implementation of the subsequent artificial intelligence-based auxiliary rapid visual positioning method for local radar scattering imaging diagnostic equipment. The system includes a local radar scattering imaging diagnostic equipment, a camera module, a feature extraction and detection module, a positioning judgment module, and a display module. The local radar scattering imaging diagnostic equipment mainly consists of a small radar, a planar near-field scanning frame, a vibration-damping cabinet, and an industrial control computer. This invention fixes the camera module to the front of the vibration-damping cabinet, mounts the display module on the upper part of the vibration-damping cabinet, and integrates the feature extraction and detection module and the positioning judgment module within the industrial control computer. The auxiliary rapid visual positioning system is used to improve the positioning accuracy of the local radar scattering imaging diagnostic equipment.

[0051] Local radar scattering imaging diagnostic equipment is a commonly used device for detecting the radar scattering characteristics of targets such as aircraft and tanks, and is well known to those skilled in the art. This embodiment employs a planar near-field scanning frame type radar scattering imaging diagnostic equipment. This equipment typically uses near-field scattering imaging measurement technology to obtain a radar scattering image of the target (aircraft, tank) within a specific angular region, and then determines the distribution of strong scattering areas based on the scattering image. To improve the consistency of measurement results, the local radar scattering imaging diagnostic equipment usually needs to maintain a relatively fixed position relative to the target.

[0052] The camera module is installed at a fixed location on the local radar scattering imaging diagnostic equipment to photograph the target (e.g., aircraft, tank), acquiring a visual image of the target and outputting it to the feature extraction and detection module. Optical images of the target are captured at the optimal diagnostic position on the local radar scattering imaging diagnostic equipment and used as a reference image. Regions with clearly defined outlines on the reference image are identified as reference areas for visual positioning.

[0053] The feature extraction and detection module adopts the YOLOv5s detection model and uses the CSPDarknet53 feature extraction network to extract features from the image received from the camera module. It receives the visual image information of the target position output by the camera module and extracts the contour information (second contour line) and key point information (second key point) of the visual positioning reference part in the image acquired by the camera module in real time, which facilitates the guidance of moving and adjusting the position and attitude of the local radar scattering imaging diagnostic equipment.

[0054] The positioning and judgment module receives contour information (second contour line) and key point information (second key points) output by the feature extraction and detection module in real time. First, coarse positioning matching is performed, calculating the IoU (Intersection over Union) between the region enclosed by the second contour line and the region enclosed by the first contour line (which is the reference contour line, manually annotated). When the IoU is greater than 95%, the position is considered basically matched. Then, precise positioning matching is performed, calculating the Euclidean distances between the three pairs of second key points and the first key point. As described later, when the Euclidean distances between the three pairs of key points are less than a set error value, the device is considered to have reached the reference position, and the positioning matching is complete.

[0055] The display module receives positioning matching information from the positioning judgment module and displays in real time the second contour line and second key point of the reference part of the target under test in the image captured by the camera module, as well as the contour line (first contour line) and key point (first key point) of the reference part of the target under test when it is in the reference position. In addition, the display module also displays the visual image of the target under test when it is in the reference position and the positioning matching result.

[0056] This invention also provides an artificial intelligence-based method for rapid visual positioning assisted by a local radar scattering imaging diagnostic device. This method is based on the aforementioned artificial intelligence-based rapid visual positioning system for local radar scattering imaging diagnostic devices. It extracts features from images of a reference position and the current position captured by a camera using a deep learning network to obtain contour lines and key points for matching and calibrating reference parts. Coarse positioning is achieved by comparing the IoU (Intersection over Union) of the area enclosed by the second contour line and the area enclosed by the first contour line. By calculating the Euclidean distance between the second and first key points, the local radar scattering imaging diagnostic device is continuously adjusted and moved to continuously reduce this Euclidean distance, restoring the current measurement position to the reference position, thereby ensuring measurement consistency and achieving rapid visual positioning assisted by an artificial intelligence-based local radar scattering imaging diagnostic device.

[0057] Please see the appendix Figure 1 As shown, the present invention provides a rapid visual localization method assisted by an artificial intelligence-based local radar scattering imaging diagnostic device, comprising the following steps:

[0058] S1. A camera module is fixed at a specific location on the local radar scattering imaging diagnostic equipment to acquire optical images; a display module is used to display the images captured by the camera in real time; the camera can be installed at any location on the local radar scattering imaging diagnostic equipment. The specific layout of the local radar scattering imaging diagnostic equipment based on rapid visual positioning is as follows: Figure 2 As shown;

[0059] S2. Based on the requirements of measuring the local radar scattering image for the relative position of the local radar scattering imaging diagnostic equipment and the target under test, determine the reference position of the target under test relative to the local radar scattering imaging diagnostic equipment (the reference position is determined according to the measurement requirements, mainly the strong scattering part of the target under test, which is the position specified when the target under test leaves the factory). Manually move the local radar scattering imaging diagnostic equipment to the reference position, and use the onboard camera to take pictures of the target under test to obtain the reference position image of the target under test.

[0060] S3. Visually observe and analyze the reference position image of the target being measured. Identify parts with clear outlines (such as a mask with clear edges, camouflage paint, lightning protection strips, etc.) as reference parts for visual positioning matching, and mark the outline edges as the first outline line (marked with the first color). Calculate the center point (first key point) of the reference part using the circumscribed rectangle of the first outline line. The four sides of the circumscribed rectangle are parallel to the four sides of the image and tangent to the first outline line. The coordinates of the first key point are calculated using the following formula, and the first key point is marked with the first color. Display the first outline line and the first key point on the monitor.

[0061]

[0062] Where (x, y) are the coordinates of the first key point, (x min y min (x) represents the coordinates of the top-left corner of the circumscribed rectangle. max y max () represents the coordinates of the bottom right corner of the circumscribed rectangle;

[0063] S4. By moving the local radar scattering imaging diagnostic equipment forward, backward, left, and right, and rotating it, the position of the camera module relative to the target under test is changed, and visual images of the target under test at different angles and distances are acquired. During the acquisition process, it is necessary to ensure that the reference part for visual positioning is always within the camera's shooting range (it does not have to be in the center of the camera's field of view).

[0064] S5. Using the optical image dataset (multiple images of the target object) acquired by the camera module, train a PSPNet semantic segmentation model and a YOLOv5s object detection model. During model training, the Adam optimization algorithm (formula described below) is used to optimize the model, and simulated cosine annealing is used to adjust the learning rate (formula described below). The optimal PSPNet semantic segmentation model can detect the contour line of the reference part of the target object, i.e., the second contour line, and mark it with a second color, different from the first color. The optimal YOLOv5s object detection model can detect the center point of the reference part of the target object, i.e., the second key point, and mark it with a second color. The second contour line and the second key point are displayed on the display. The use of the Adam optimization algorithm to train the PSPNet semantic segmentation model and the YOLOv5s object detection model, and the use of simulated cosine annealing to adjust the learning rate, are well-known to those skilled in the art and will not be elaborated further.

[0065] Furthermore, the PSPNet semantic segmentation network mainly consists of a backbone network, an enhanced feature extraction structure, and a head network. The input image first undergoes preliminary feature extraction through the backbone network to obtain a feature map. Then, the enhanced feature extraction structure further aggregates the features, enriches the feature information, and improves the ability of the feature map to represent global information. Finally, the head network integrates and upsamples the features to achieve semantic segmentation of the image. The specific implementation process of the PSPNet semantic segmentation network is well known to those skilled in the art and will not be described in detail here (for the specific implementation process, please refer to the paper "Pyramid Scene Parsing Network" Zhao H, Shi J, Qi X, Wang X, Jia J. Pyramid Scene Parsing Network [C]. ComputerVision and Pattern Recognition. IEEE, 2017: 2881-2890).

[0066] Furthermore, MobileNetV2 was chosen as the backbone network of PSPNet (see the paper "Mobilenetv2: Inverted Residuals and Linear Bottlenecks" Sandler M, Howard A, Zhu M, Zhmonginov A, Chen L C. Mobilenetv2: Inverted Residuals and Linear Bottlenecks[C]. ComputerVision and Pattern Recognition. IEEE, 2018: 4510-4520). MobileNetV2 is a lightweight deep network whose core design concept is to replace ordinary convolution with depthwise separable convolution. In Inverted Residuals, the input image feature data is first enlarged using a 1×1 convolution. Then, a 3×3 depthwise separable convolution is used to further extract relevant features from the image feature data. Finally, a 1×1 convolution is used to reduce the dimensionality of the extracted features (RGB images have three channels: R, G, and B. Enlargement increases the number of channels in the image through convolution operations, while reduction decreases the number of channels through convolution operations). The convolution operation process and MobileNetV2 are well known to those skilled in the art and will not be described in detail here.

[0067] Furthermore, the backbone network of the YOLOv5s model adopts the CSPDarknet53 network, which consists of 6 modules: 3 "convolutional layer-batch normalization layer-activation function layer" modules and 3 residual modules (see the paper "YOLOv4: Optimal Speed ​​and Accuracy of Object Detection" Bochkovskiy A, Wang CY, Liao HY M. YOLOv4: Optimal Speed ​​and Accuracy of Object Detection[J]. arXiv preprint arXiv,2020,2004.10934). The "convolutional layer-batch normalization layer-activation function layer" module consists of a 3×3 convolutional layer, a batch normalization layer, and a LeakyReLU activation function layer. The stride of the convolutional layers in the first two "convolutional layer-batch normalization layer-activation function layer" modules is 2, and the image is downsampled by sliding the convolutional kernel in the convolutional layer on the image. The CSPDarknet53 network is well known to those skilled in the art and will not be described further.

[0068] Furthermore, YOLOv5s is a lightweight version of YOLOv5, offering faster inference speeds. YOLOv5s consists of a backbone network, a Feature Pyramid Network (FPN), and a head network (see the paper). <FeaturePyramid Networks for Object Detection》。Lin T Y, Dollar P,Girshick R,He K,Hariharan B,Belongie S.Feature Pyramid Networks for Object Detection[C].Computer Vision and Pattern Recognition.IEEE,2017:2117-2125)。

[0069] Furthermore, FPN consists of a convolutional layer-batch normalization layer-activation function layer, an upsampling layer, a 1×1 convolutional layer, and a concatenation layer (see the paper "YOLOv4: Optimal Speed ​​and Accuracy of Object Detection"). The deep features extracted by the model's backbone network are further integrated and upsampled through the convolutional layer-batch normalization layer-activation function layer. These features are then stacked with the shallower features extracted by the backbone network in the concatenation layer. The integration and upsampling process is accomplished through convolutional operations in the convolutional layer. Stacking refers to the stacking of features along the channel dimension within the concatenation layer. This allows features from different levels to be fused, which is beneficial for extracting features with richer semantic information. FPN is well-known to those skilled in the art and will not be elaborated further.

[0070] Furthermore, the head network consists of a "convolutional layer-batch normalization layer-activation function layer" and a 1×1 convolutional layer, including two scales: 13×13 and 26×26. The receptive field of the input image is 32×32 for the 13×13 scale and 16×16 for the 26×26 scale, which has a good effect on detecting targets of different sizes.

[0071] Furthermore, the activation function uses the Mish function: Mish = x·tanh(ln(1+e)) x )).

[0072] Furthermore, the learning rate adjustment method uses simulated cosine annealing to adjust the learning rate.

[0073]

[0074] Among them, lr t To simulate cosine annealing for adjusting the learning rate, lr max The initial learning rate is lr min The minimum allowable learning rate, i.e., when the learning rate decays to lr min At that time, it will no longer continue to decay, but will remain at lr. min ,lr t Let T be the current learning rate, t be the current training epoch, and T be the current learning rate. max This is the learning rate decay period.

[0075] Furthermore, the model optimization algorithm adopts the Adam optimization method, as shown in the following equation.

[0076]

[0077] Where L is the loss function, w i Let b be the i-th weight parameter of the network. i Let be the i-th bias parameter of the network, t be the number of iterations, α be the learning rate, and β1 and β2 be exponential weighting parameters. In this invention, β1 = 0.9 and β2 = 0.999 are chosen. ε is used to prevent a small quantity with a denominator of zero; in this invention, ε = 1 × 10^25. -9 w i,t Let b be the i-th weight parameter of the network in the t-th iteration. i,t w is the i-th bias parameter of the network at the t-th iteration. i,t+1 Let b be the i-th weight parameter of the network in the (t+1)-th iteration. i,t+1 Let be the i-th bias parameter of the network at the (t+1)-th iteration. Let i be the second intermediate variable of the i-th weight parameter in the t-th iteration. Let i be the second intermediate variable of the i-th weight parameter in the (t-1)-th iteration. Let be the second intermediate variable of the i-th bias parameter in the t-th iteration. This is the second intermediate variable for the i-th bias parameter during the (t-1)-th iteration. Let i be the first intermediate variable of the i-th weight parameter in the t-th iteration. Let i be the first intermediate variable of the i-th weight parameter in the (t-1)-th iteration. Let i be the first intermediate variable of the i-th bias parameter in the t-th iteration. Let i be the first intermediate variable of the i-th bias parameter in the (t-1)-th iteration. Let be the third intermediate variable for the i-th weight parameter. Let be the third intermediate variable for the i-th bias parameter. Let be the fourth intermediate variable for the i-th weight parameter. It is the fourth intermediate variable for the i-th bias parameter. The superscript t indicates t raised to the power of t.

[0078] S6. Turn on the camera module and simultaneously move the local radar scattering imaging diagnostic device to the vicinity of the reference measurement position of the target. Observe the display module. When a complete second contour line of the visual positioning reference part appears on the display module, observe the first contour line on the display module. Move and adjust the local radar scattering imaging diagnostic device until the second contour line and the first contour line basically coincide in the display module. Determine whether the current position is basically matched with the reference position by the intersection-union ratio (IoU) of the areas enclosed by the second contour line and the areas enclosed by the first contour line. When the IoU is greater than 95%, it is determined that the current position is basically matched with the reference position. Then, fine-tune the local radar scattering imaging diagnostic device and determine whether the current position is accurately matched with the reference position by calculating the Euclidean distance between the second key point and the first key point. When the Euclidean distance between all three pairs of key points is less than the specified value, it is determined that the position is accurately matched, and the screen will automatically prompt that the positioning is matched, completing the accurate positioning. The details are as follows.

[0079] S601. Continuously move and adjust the local radar scattering imaging diagnostic equipment to try to match the second contour line displayed on the screen in real time with the first contour line displayed at a fixed time. When the IoU is greater than 95%, it is determined that the position is basically matched.

[0080] The formula for calculating the IoU of the regions enclosed by the second contour line and the regions enclosed by the first contour line is:

[0081]

[0082] Among them, S g S represents the pixels in the area enclosed by the second contour line. r This represents the number of pixels in the area enclosed by the first outline.

[0083] S602. Calculate the Euclidean distance between the current second key point and the first key point. When the Euclidean distance between all three pairs of key points is less than the set error value, stop adjusting the local radar scattering imaging diagnosis. It is considered that the device has reached the reference position and the local radar scattering imaging diagnosis device is positioned and matched relative to the target being measured. The screen module displays "Positioning Matching".

[0084] The set accuracy requirement is an error value of less than 0.01. The formula for calculating the Euclidean distance between the current positioning point and the reference positioning point is:

[0085]

[0086] In the formula, x iThe x-coordinate of the second key point is represented by y. i The ordinate of the second key point, x i ' represents the x-coordinate of the first key point, y i ′ represents the ordinate of the first key point, and L represents the width of the image captured by the camera module.

[0087] In summary, the present invention provides an artificial intelligence-based method and system for assisting rapid visual positioning of local radar scattering imaging diagnostic equipment. This method and system can achieve both efficiency and accuracy in visual positioning of local radar scattering imaging diagnostic equipment, enabling rapid and accurate positioning. Furthermore, it is simple to operate, reducing the entire positioning time to approximately 10 seconds, and the impact of positioning accuracy on measurement consistency is within an acceptable range.

[0088] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A local radar scatter imaging diagnostic device assisted rapid visual positioning system, characterized in that, The system comprises a local radar scattering imaging diagnosis device, a camera module, a feature extraction detection module, a positioning judgment module and a display module; wherein The local radar scattering imaging diagnosis device adopts a planar near-field scanning frame radar scattering imaging diagnosis device to obtain a radar scattering image of the measured target in a certain angular domain, and then determine the strong scattering region distribution of the measured target according to the scattering image; the local radar scattering imaging diagnosis device and the measured target keep a relatively fixed position; The camera module is arranged at a fixed position on the local radar scattering imaging diagnosis device, and is used for photographing the measured target, collecting the visual image of the measured target and outputting the visual image to the feature extraction detection module; The feature extraction detection module adopts a YOLOv5s detection model, adopts a feature extraction network CSPDarknet53 to extract the features of the image received from the camera module, receives the visual image information of the measured target position output by the camera module, and extracts the second contour line and the second key point information of the visual positioning reference part in the image collected by the camera module in real time, so as to guide the movement and adjustment of the position and attitude of the local radar scattering imaging diagnosis device; The positioning judgment module receives the second contour line and the second key point information output by the feature extraction detection module in real time, judges whether the device has reached the reference position according to the first contour line and the first key point information, and completes the positioning matching when reaching the reference position; The display module receives the positioning matching information from the positioning judgment module, displays the second contour line and the second key point of the reference part of the measured target in the image collected by the camera module in real time, and displays the first contour line and the first key point of the reference part of the measured target when the measured target is at the reference position, in addition, the display module also displays the visual image and the positioning matching result when the measured target is at the reference position.

2. The local radar scatterometry diagnostic equipment assisted rapid visual positioning system of claim 1, wherein, The camera module is fixed in front of the shockproof cabinet, the display module is installed on the upper part of the shockproof cabinet, the feature extraction detection module and the positioning judgment module are integrated in the industrial computer; at the best diagnosis position of the local radar scattering imaging diagnosis device, the optical image of the measured target is photographed and taken as the reference position image.

3. A method of fast visual positioning assisted by a local radar scatter imaging diagnostic device, based on the system of fast visual positioning assisted by a local radar scatter imaging diagnostic device according to claim 1 or 2, characterized in that, Specifically, the following steps are included: S1, fixing the camera module at a fixed position on the local radar scattering imaging diagnosis device to obtain an optical image; using the display module to display the picture photographed by the camera in real time; S2, according to the requirement of the relative position of the local radar scattering imaging diagnosis device and the measured target for measuring the local radar scattering image, determining the reference position of the measured target relative to the local radar scattering imaging diagnosis device, manually moving the local radar scattering imaging diagnosis device to the reference position, using the camera to take a picture of the measured target, and obtaining the reference position image of the measured target; S3, visually observing and analyzing the reference position image of the measured target, determining a part with a clear contour as a reference part matched by visual positioning, marking the contour edge as a first contour line, and marking the first contour line as a first color; calculating the center point of the reference part, that is, a first key point, through the circumscribed rectangle frame of the first contour line; the four sides of the circumscribed rectangle frame are respectively parallel to the four sides of the image and tangent to the first contour line, and the coordinates of the first key point are calculated by the following formula; the first contour line and the first key point are fixedly displayed on the display; Where (x, y) are the coordinates of the first key point, (x min y min (x) represents the coordinates of the top-left corner of the circumscribed rectangle. max y max () represents the coordinates of the bottom right corner of the circumscribed rectangle; S4, change the position of the camera module relative to the measured target by moving the local radar scattering imaging diagnostic equipment forward, backward, left and right, and rotating it, and collect visual images of the measured target at different angles and distances; during the collection process, the reference part for visual positioning needs to be ensured to be always within the shooting range of the camera; S5, for the optical image data set of the measured target collected by the camera module, train the PSPNet semantic segmentation model and the YOLOv5s target detection model, and use the Adam optimization algorithm to optimize the model during the model training process, and the learning rate adjustment method uses the method of simulating cosine annealing to adjust the learning rate; the trained PSPNet semantic segmentation optimal model can detect the contour line of the reference part of the measured target, that is, a second contour line, mark the second contour line as a second color, and the second color is different from the first color; the trained YOLOv5s target detection optimal model can detect the center point of the reference part of the measured target, that is, a second key point, mark the second key point as a second color; and display the second contour line and the second key point on the display; S6, open the camera module, move the local radar scattering imaging diagnostic equipment to the vicinity of the reference measurement position of the measured target, and observe the display module; when the complete second contour line of the visual positioning reference part appears on the display module, observe the first contour line on the display module, move and adjust the local radar scattering imaging diagnostic equipment, and after the second contour line and the first contour line are basically coincident in the display module, determine whether the current position and the reference position are basically matched through the intersection over union IoU of the second contour line surrounding area and the first contour line surrounding area; when the IoU is greater than a numerical value A, it is determined that the current position and the reference position are basically matched; then fine-tune the local radar scattering imaging diagnostic equipment, judge whether the current position and the reference position are accurately matched by calculating the Euclidean distance between the second key point and the first key point; when the Euclidean distances between the three pairs of key points are all less than a specified value, it is determined that the positions are accurately matched, the screen automatically prompts the positioning matching, and the accurate positioning is completed.

4. The local radar scatterometry diagnostic device assisted rapid visual positioning method of claim 3, wherein, The PSPNet semantic segmentation network is composed of a backbone network, a strengthened feature extraction structure and a head network. The input image is first preliminarily extracted by the backbone network to obtain a feature map, then the feature map is further aggregated by the strengthened feature extraction structure to enrich the feature information and improve the ability of the feature map to represent global information, and finally the features are integrated and up-sampled by the head network to realize semantic segmentation of the image.

5. The local radar scatterometry diagnostic device assisted rapid visual positioning method of claim 4, wherein, MobileNetV2 is selected as the backbone network of PSPNet, and the ordinary convolution is replaced by the depth separable convolution; in the reverse residual, first, the size of 1*1 convolution is used to upgrade the image feature data, then the size of 3*3 depth separable convolution is used to further extract the related features of the image feature data, and finally the size of 1*1 convolution is used to reduce the dimension of the extracted features.

6. The local radar scatterometry diagnostic device assisted rapid visual positioning method of claim 3, wherein, The backbone network of the YOLOv5s model adopts the CSPDarknet53 network, which is composed of 6 modules, namely 3 "convolution layer-batch normalization layer-activation function layer" modules and 3 residual modules, which are composed of the CSPDarknet53 network. The "convolution layer-batch normalization layer-activation function layer" module is composed of a 3*3 convolution layer, a batch normalization layer and a LeakyReLU activation function layer, wherein the convolution layer in the first two "convolution layer-batch normalization layer-activation function layer" modules has a step of 2, and the image is down-sampled by sliding the convolution kernel in the image.

7. The local radar scatterometry diagnostic device assisted rapid visual positioning method of claim 3, wherein, The FPN is composed of "convolution layer-batch normalization layer-activation function layer", up-sampling layer, 1*1 convolution layer and Concatenation layer; the deep features extracted by the backbone network of the model are further integrated after "convolution layer-batch normalization layer-activation function layer", and after up-sampling, the shallow features extracted by the backbone network are stacked in the Concatenation layer. The integration and up-sampling process is completed by convolution operation in the convolution layer, and the stacking refers to the stacking of the features in the channel dimension in the Concatenation layer.

8. The local radar scatterometry diagnostic device assisted rapid visual positioning method of claim 3, wherein, The learning rate adjustment method uses the cosine annealing learning rate adjustment (Cosine Annealing), where lr t is the simulated cosine annealing learning rate, lr max is the initial learning rate, lr min is the minimum learning rate allowed, i.e. when the learning rate is decayed to lr min , it will not continue to decay, but stay at lr min , lr t is the current learning rate, t is the current training epoch, T max is the learning rate decay period.

9. The local radar scatterometry diagnostic device assisted rapid visual positioning method of claim 3, wherein, The model optimization algorithm adopts the Adam optimization method, as shown in the following formula; wherein, L is a loss function, w i is the i-th weight parameter of the network, b i is the i-th bias parameter of the network, t is the iteration number, a is the learning rate, b1, b2 are exponential weighting parameters, the present application takes b1=0.9, b2=0.999, and e is a small amount to prevent the denominator from being zero, the present application takes e=1x10 -9 , w i,t is the i-th weight parameter of the network at the t-th iteration, b i，t is the i-th bias parameter of the network at the t-th iteration, w i，t+1 is the i-th weight parameter of the network at the t+1-th iteration, b i,t+1 is the i-th bias parameter of the network at the t+1-th iteration, is the second intermediate variable of the i-th weight parameter at the t-th iteration, is the second intermediate variable of the i-th weight parameter at the t-1-th iteration, is the second intermediate variable of the i-th bias parameter at the t-th iteration, is the second intermediate variable of the i-th bias parameter at the t-1-th iteration, is the first intermediate variable of the i-th weight parameter at the t-th iteration, is the first intermediate variable of the i-th weight parameter at the t-1-th iteration, is the first intermediate variable of the i-th bias parameter at the t-th iteration, is the first intermediate variable of the i-th bias parameter at the t-1-th iteration, is the third intermediate variable of the i-th weight parameter, is the third intermediate variable of the i-th bias parameter, is the fourth intermediate variable of the i-th weight parameter, is the fourth intermediate variable of the i-th bias parameter; The subscript t represents t power.

10. The method of claim 3, wherein the method further comprises: The step S6 is specifically as follows: S601, continuously move the local radar scattering imaging diagnosis equipment to try to match the second contour line displayed on the screen in real time with the first contour line displayed fixedly, and when the IoU is greater than 95%, it is determined that the positions are basically matched; The IoU formula for calculating the area surrounded by the second contour line and the first contour line is: wherein S g represents a pixel point surrounded by the second contour line, S r represents a pixel point surrounded by the first contour line; S602, calculate the Euclidean distance between the current second key point and the first key point, when the Euclidean distances between the three pairs of key points are all less than the set error value, stop adjusting the local radar scattering imaging diagnosis, and consider that the equipment has reached the reference position, complete the positioning matching of the local radar scattering imaging diagnosis equipment relative to the measured target, and the screen module displays "positioning matching"; Set the accuracy requirement for the error value, and the formula for calculating the Euclidean distance between the current positioning point and the reference positioning point is: In the formula, x i represents the horizontal coordinate of the second key point, y i represents the vertical coordinate of the second key point, x i ' represents the horizontal coordinate of the first key point, y i ' represents the vertical coordinate of the first key point, and L represents the width of the image collected by the camera module.

Images