AI identification method and system for millimeter wave image overlapping target, and model training method

Abstract

The invention discloses an AI identification method and system for a millimeter wave image overlapping target based on a three-branch network, and a model training method, and the method comprises the steps: carrying out the cooperative work of three independent branch networks: an analysis branch network, an aggregation branch network, and a prediction boundary branch network; the problem of security inspection target detection in millimeter wave images with low resolution, mutual interference or mutual overlapping is solved. The analysis branch network stores detail information in the high-resolution feature map; the aggregation branch network realizes feature position offset alignment of high-resolution and low-resolution feature maps through a recursion fusion module; and the prediction boundary branch network extracts high-frequency features, and enhances perception and prediction of the target edge. The multi-branch network structure effectively realizes image segmentation and identification through fusion among branches. While the segmentation performance of the model is improved, the model significantly reduces the calculation complexity, reduces the detection error and missed detection probability in the millimeter wave image, and finally achieves the precise detection of overlapped objects.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and more specifically, to an AI recognition method, system and model training method for overlapping targets in millimeter wave images. Background Art

[0002] Electromagnetic waves with a wavelength of 1 to 10 mm are called millimeter waves. They are located between microwaves and far-infrared waves and have the characteristics of both. The theory and technology of millimeter waves are the extension of microwaves to high frequencies and the development of light waves to low frequencies. As a type of radio wave, millimeter waves do not have the problem of ionizing radiation compared to rays, and low-power millimeter wave signals are harmless to the human body. In addition to the common characteristics of infrared and microwaves, millimeter waves also have their own unique properties. Compared with microwaves, millimeter waves have better directivity, stronger anti-interference ability, excellent detection performance and better plasma penetration ability. In human body security inspections, millimeter wave imaging technology can not only penetrate ordinary clothing and detect hidden metal contraband, but also detect non-metallic contraband. Since the millimeter wave band is harmless to the human body, millimeter waves show good application effects in human body detection. However, due to the limitation of the wavelength diffraction limit, the resolution of millimeter wave images is generally low. Moreover, due to the limitations of millimeter wave detector performance and imaging algorithms, the signal-to-noise ratio of millimeter wave images is much lower than that of optical images, the generated image pixels are large and the image is relatively blurred, making it difficult for the human eye to recognize the information in the image.

[0003] Therefore, if the existing technology is implemented in human body security inspection, there will be shortcomings. At this time, it is necessary to cooperate with appropriate algorithm models to further improve the accuracy and efficiency of detection. Summary of the invention

[0004] In view of the above-mentioned defects of the prior art, the present invention provides an AI recognition method, system and model training method for overlapping targets in millimeter wave images, which simultaneously completes multiple tasks such as retaining detail information in high-resolution feature maps, effectively aggregating contextual information, and extracting high-frequency features to predict boundary areas based on a multi-branch fusion network, thereby enhancing the perception of target edges.

[0005] To achieve the above objectives, in a first aspect, the present invention provides an AI recognition system for overlapping targets in millimeter wave images, characterized in that it includes three branch networks and their branch fusion modules:

[0006] The parsing branch network is used to parse branches and preserve the details in the high-resolution feature map. It is also fused with the two different-scale feature maps of the aggregation branch network to selectively learn semantic features from the aggregation branch network.

[0007] An aggregation branch network is used to aggregate context information to resolve long-range dependencies; the aggregation branch network includes a recursive fusion module, which realizes recursive alignment fusion of feature layers by learning feature position offsets of a single high-resolution feature map and multiple adjacent levels of low-resolution feature maps;

[0008] The prediction boundary branch network is merged with the feature maps of different sizes of the aggregation branch network to extract the high-frequency features of the image to predict the boundary area;

[0009] Branch fusion module, the mathematical expression of the fusion strategy is:

[0010]

[0011] In the formula, Represent the vectors mapped by the three branch networks, F out and f represents the collective function of convolution, normalization and ReLU.

[0012] The parsing branch network preserves high resolution and is combined with the rich and accurate semantic information provided by the aggregation branch network. The semantic information provided by the aggregation branch network can effectively prevent the parsing branch network from being suppressed by too much background or global context information, so that the detail branch can selectively learn high-level semantic information. The predicted boundary branch network is spliced ​​with the semantic information provided by the aggregation branch network, so that the boundary branch network directly adds the feature map provided by the aggregation branch network, and accurately predicts the changes in the boundary area through the sensitivity of the boundary branch network to changes, thereby compensating for the problem that the mask caused by the averaging when the conventional high-resolution detail features are fused with the low-resolution context semantics is not detailed enough to describe the boundary of the predicted target, so that the edge segmentation of the target can be more refined. The aggregation branch network is semantically rich and can provide more accurate semantics, but it loses too many spatial and geometric details, especially for boundary areas and small objects. The detail branch better retains the spatial details. Finally, when the features provided by the three branches are fused, the model trusts the detail branch more along the boundary area, and uses the context features to fill the area inside the object, so as to achieve more accurate recognition of the image.

[0013] Furthermore, the recursive fusion module of the aggregation branch network adopts multi-scale feature fusion to perform pooling, normalization and convolution operations on feature maps of different scales.

[0014] Furthermore, a first fusion module is provided in the parsing branch network for fusing with two different scale feature maps of the aggregation branch network; and a second fusion module is provided in the prediction boundary branch network for merging with different size feature maps of the aggregation branch network.

[0015] Furthermore, the branch fusion module trusts the parsing branch network and the prediction boundary branch network more along the boundary area, and fills the area inside the object with the context information features of the aggregated branch network.

[0016] In a second aspect, the present invention provides a model training method for a millimeter wave image overlapping target AI recognition system, characterized in that it includes the following steps:

[0017] S1. Collect millimeter wave image datasets and divide the images into training sets and test sets;

[0018] S2. Construct the AI ​​recognition system model as described above;

[0019] S3. Use the collected training set to train the system model: downsample the training set into a high-resolution feature image and output it to three branch networks; the parsing branch network maintains the scale of the high-resolution feature image, merges it with the two different-scale feature maps of the aggregation branch network, and selectively learns semantic features from the aggregation branch network; the aggregation branch network performs a series of downsampling and upsampling on the input image, merges the information of the three scales, and then realizes the recursive alignment fusion of the feature layer through the aggregation module; the prediction boundary branch network maintains the image scale as the high-resolution feature image scale, and merges it with the different-scale feature maps of the aggregation branch network to extract the high-frequency features of the image to predict the boundary area; after the processing of each branch network is completed, the three-branch fusion processing is performed, and the mathematical expression of the fusion strategy is:

[0020]

[0021] In the formula, Represent the vectors mapped by the three branch networks, F out and f represents the collective function of convolution, normalization and ReLU;

[0022] S4. Test the trained model with the test set and obtain the prediction effect diagram through the segmentation head module; the segmentation head module is the key part of extracting the final segmentation result from the feature map. The function uses upsampling operation to restore the image to the resolution of the input image. It contains several convolutional layers for further enhancing features and generating category predictions.

[0023] Furthermore, the aggregation branch network performs a series of downsampling and upsampling on the input image, merges the information of three scales, and then performs pooling, normalization and convolution operations through the aggregation module to achieve better aggregation of contextual information.

[0024] On the last aspect, the present invention provides an AI recognition method for overlapping targets in millimeter wave images, characterized in that the millimeter wave images are subjected to AI recognition of overlapping targets using an AI recognition system such as the above-mentioned AI recognition system or an AI recognition system trained by the above-mentioned model training method.

[0025] In the actual application of human body contraband detection, there is often more than one detection object, and multiple objects overlap at the same position, which increases the difficulty of image recognition. Compared with the prior art, the present invention has the following technical effects:

[0026] The recognition system and method of the present invention can accurately detect and segment overlapping targets in low-resolution millimeter wave images. The multi-branch network structure can simultaneously complete multiple tasks: retaining detail information in high-resolution feature maps; effectively aggregating context information; extracting high-frequency features to predict boundary areas, thereby enhancing the perception of target edges. The branch fusion module uses boundary features to guide the fusion of detail and context representations to achieve better segmentation effects. The present invention achieves rapid and accurate recognition of targets in millimeter wave overlapping images.

[0027] More specifically, the present invention solves the problem of security inspection target detection in low-resolution, mutually interfering or overlapping millimeter-wave images through the collaborative work of three independent branches: the parsing branch network, the aggregation branch network, and the prediction boundary branch network. The parsing branch network preserves the detail information in the high-resolution feature map; the aggregation branch network realizes the feature position offset alignment of the high-resolution and low-resolution feature maps through a recursive fusion module; the prediction boundary branch network extracts high-frequency features to enhance the perception and prediction of the target edge. The multi-branch network structure effectively realizes image segmentation and recognition through the fusion of branches. While improving the segmentation performance of the model, the model significantly reduces the computational complexity and reduces the detection errors and missed detection probabilities in millimeter-wave images, ultimately achieving accurate detection of overlapping objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Schematic diagram of the architecture of an AI recognition system in one embodiment of the present invention.

[0029] Figure 2 Schematic diagram of the architecture of the first fusion module in one embodiment of the present invention.

[0030] Figure 3 FIG. 4 is a schematic diagram of the architecture of the second fusion module in one embodiment of the present invention.

[0031] Figure 4 FIG. 4 is a schematic diagram of the architecture of a branch fusion module in an embodiment of the present invention.

[0032] Figure 5 This is a flow chart of a model training method for an AI recognition system in one embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0034] In the following detailed description, many specific details are set forth to provide a more thorough understanding of the present invention. However, it is obvious to those skilled in the art that well-known algorithms and models (such as pooling algorithms, convolutional networks, normalization algorithms) are not shown in detail to avoid obscuring the subject matter of the present invention.

[0035] In addition, the order of execution of actions, steps, etc. in the devices and methods shown in the claims, specifications and drawings can be implemented in any order as long as there is no special explicit limitation on the order and the output of the previous processing is not used in the subsequent processing.

[0036] Example

[0037] See also Figure 1 , this embodiment provides an AI recognition system for overlapping targets in millimeter wave images, including three branch networks and their branch fusion modules. First, the input image is downsampled to 1 / 8 of the original size, and then this feature map is used in the three branch networks respectively. The upper branch is a detail resolution branch network, which is used to branch resolve and save detail information in the high-resolution feature map, and at the same time fuse with two different scale feature maps of the aggregation branch network, selectively learn semantic features from the aggregation branch network, thereby enhancing the detail resolution capability. More specifically, a first fusion module is provided in the resolution branch network, which is used to fuse with two different scale feature maps of the aggregation branch network. See Figure 2 First, the two branches are residually connected. Then, the aggregation branch is processed by convolution, batch normalization and upsampling, and the parsing branch is processed by convolution and batch normalization. Then, the outputs of the two branches are multiplied and added to fuse the feature maps. The details and context information are effectively fused, the feature representation ability is enhanced, and the model performance is improved. In this way, the parsing branch network ensures that important semantic information is not lost when processing image details, and avoids the interference of background information on detail information.

[0038] The aggregation branch network is the main branch, which is used to aggregate context information to resolve long-range dependencies; the aggregation branch network includes a recursive fusion module, which realizes recursive alignment fusion of feature layers by learning the feature position offsets of a single high-resolution feature map and multiple adjacent levels of low-resolution feature maps. More specifically, in the aggregation branch network, it first undergoes two downsamplings to obtain scales of 1 / 16 and 1 / 32, respectively, acting as backup support for the other two branch networks and providing them with the required information. Then, a series of upsampling and downsampling are performed to merge the information of the three scales, and then the aggregation module performs pooling, normalization and convolution operations to achieve better aggregation of context information.

[0039] The lower branch is a prediction boundary branch network, which keeps the image size at 1 / 8 of the original size, merges with the different-size feature maps of the aggregation branch network, and extracts the high-frequency features of the image to predict the boundary area. More specifically, the prediction boundary branch network is provided with a second fusion module for merging with the different-size feature maps of the aggregation branch network. Figure 3 First, the two branches are residually connected. Then, the boundary branch is directly output, while the aggregation branch is processed by convolution, batch normalization and upsampling. Finally, the results of the two branches are added and output. This module retains the original information of the input features, enhances the feature representation, and improves the expressiveness and training efficiency of the model. The design of the boundary branch network enables the model to more accurately identify the boundary areas in the image, so that it can better handle the problem of blurred or discontinuous edges during segmentation.

[0040] See also Figure 4 , the three-branch fusion processing is performed after the processing of each branch network in the identification system is completed. The mathematical expression of the branch fusion module is:

[0041]

[0042] Through this fusion strategy, the model can comprehensively utilize the advantages of each branch to achieve more accurate image segmentation and target recognition. Represent the vectors mapped by the three branch networks, and F outand f represents the collection function of convolution, normalization and ReLU. T1 and T3 are the first and third branches that process themselves once, and then are assigned by the activation function of the third branch. The value is between 0 and 1, and the sum of the two items is 1. Then the two items are tensor-producted with the other two branches respectively, and then the two results are added to obtain the final vector value. Then, the collection function is used to obtain the result output by the final fusion module through a series of convolution normalization operations. The third branch is the boundary branch, which guides the allocation of fusion ratios by adjusting the weights of boundary features, thereby achieving dynamic feature fusion. It effectively enhances the feature representation capability and improves the performance and robustness of the model.

[0043] See also Figure 5 This embodiment also provides a model training method for the above AI recognition system, comprising the following steps:

[0044] S1. Collect millimeter wave image datasets and divide the images into training sets and test sets;

[0045] S2. Construct the AI ​​recognition system model as described above;

[0046] S3. Use the collected training set to train the system model: downsample the training set into a high-resolution feature image and output it to three branch networks; the parsing branch network maintains the scale of the high-resolution feature image, merges it with the two different-scale feature maps of the aggregation branch network, and selectively learns semantic features from the aggregation branch network; the aggregation branch network performs a series of downsampling and upsampling on the input image, merges the information of the three scales, and then realizes the recursive alignment fusion of the feature layer through the aggregation module; the prediction boundary branch network maintains the image scale as the high-resolution feature image scale, and merges it with the different-scale feature maps of the aggregation branch network to extract the high-frequency features of the image to predict the boundary area; after the processing of each branch network is completed, the three-branch fusion processing is performed, and the mathematical expression of the fusion strategy is:

[0047]

[0048] In the formula, Represent the vectors mapped by the three branch networks, F out and f represents the collective function of convolution, normalization and ReLU;

[0049] S4. Test the trained model with the test set and obtain the prediction effect diagram through the segmentation head module; the segmentation head module is the key part of extracting the final segmentation result from the feature map. The function uses upsampling operation to restore the image to the resolution of the input image. It contains several convolutional layers for further enhancing features and generating category predictions.

[0050] In order to facilitate those skilled in the art to understand the implementation effect of the three-branch network model in the present invention, we conducted experiments on the dataset of millimeter wave images for human body detection. Taking the dual-branch and single-branch network models as the baseline models, all models are evaluated by single-scale inference, and the input image resolution is 640×640. The parameters for testing network performance include mean intersection over union (MIoU), parameter count, and floating-point operations (FLOPs). MIoU is used to measure the similarity between the predicted results and the true labels in semantic segmentation tasks; parameter count indicates the number of parameters that need to be learned during training; FLOPs indicates the number of floating-point operations performed during inference, which is usually used to measure algorithm complexity.

[0051] The experimental results show that the MIoU value of the dual-branch network is 5.96% higher than that of the single-branch network; the MIoU value of the three-branch network is further improved by 3.05% compared with the dual-branch network. At the same time, the three-branch network only adds a small number of parameters compared to the dual-branch network, but significantly reduces the FLOPs of the model, which shows that the three-branch network has low computational complexity while maintaining high efficiency and accuracy.

[0052] In summary, the present invention provides an AI recognition method, system and model training method for overlapping targets in millimeter wave images based on a three-branch network model, wherein the parsing branch preserves high resolution and is combined with the rich and accurate semantic information provided by the aggregation branch. The semantic information provided by the aggregation branch can effectively prevent the parsing branch from being suppressed by too much background or global context information, so that the detail branch can selectively learn high-level semantic information. The predicted boundary branch is spliced ​​with the semantic information provided by the aggregation branch, so that the boundary branch directly adds the feature map provided by the aggregation branch, and the boundary branch is sensitive to changes to achieve accurate prediction of changes in the boundary area, thereby compensating for the problem that the mask caused by the averaging of the conventional high-resolution detail features and the low-resolution context semantics when fused is not detailed enough to describe the boundary of the predicted target, so that the edge segmentation of the target can be more refined. The aggregation branch is rich in semantics and can provide more accurate semantics, but it loses too many spatial and geometric details, especially for boundary areas and small objects. The detail branch better retains the spatial details. Finally, when the features provided by the three branches are fused, the model trusts the detail branch more along the boundary area, and uses the context features to fill the area inside the object, so as to achieve more accurate recognition of the image.

[0053] Those skilled in the art should understand that those skilled in the art can implement variations by combining the prior art and the above embodiments, which will not be described in detail here. Such variations do not affect the essential content of the present invention, and will not be described in detail here.

[0054] The above describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and the devices and structures that are not described in detail should be understood to be implemented in a common manner in the art; any technician familiar with the art can use the above-disclosed methods and technical contents to make many possible changes and modifications to the technical solutions of the present invention without departing from the scope of the technical solutions of the present invention, or modify them into equivalent embodiments of equivalent changes, which does not affect the essential content of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention are still within the scope of protection of the technical solutions of the present invention.

[0055] Those skilled in the art will appreciate that the units, i.e., algorithm steps, of the various examples described in conjunction with this embodiment can be implemented in electronic hardware or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

Claims

1. AI recognition system for overlapping targets in millimeter wave images, characterized in that: It includes three branch networks and their branch fusion modules: The parsing branch network is used to parse branches and preserve the details in the high-resolution feature map. It is also fused with the two different-scale feature maps of the aggregation branch network to selectively learn semantic features from the aggregation branch network. An aggregation branch network is used to aggregate context information to resolve long-range dependencies; the aggregation branch includes a recursive fusion module, which realizes recursive alignment fusion of feature layers by learning feature position offsets of a single high-resolution feature map and multiple adjacent levels of low-resolution feature maps; The prediction boundary branch network is merged with the feature maps of different sizes of the aggregation branch network to extract the high-frequency features of the image to predict the boundary area; Branch fusion module, the mathematical expression of the fusion strategy is: In the formula, Represent the vectors mapped by the three branch networks, F out and f represents the collective function of convolution, normalization and ReLU.

2. The AI ​​recognition system for overlapping targets in millimeter wave images according to claim 1 is characterized in that: The recursive fusion module of the aggregation branch network adopts multi-scale feature fusion to perform pooling, normalization and convolution operations on feature maps of different scales.

3. The AI ​​recognition method for overlapping targets in millimeter wave images according to claim 1, characterized in that: The parsing branch network is provided with a first fusion module for fusing with two different scale feature maps of the aggregation branch network; the prediction boundary branch network is provided with a second fusion module for merging with different size feature maps of the aggregation branch network.

4. The AI ​​recognition system for overlapping targets in millimeter wave images according to claim 1 is characterized in that: The branch fusion module further trusts the parsing branch network and the prediction boundary branch network along the boundary area, and uses the context information features of the aggregated branch network to fill the area inside the object.

5. A model training method for a millimeter wave image overlapping target AI recognition system, characterized in that: The following steps are involved: S1. Collect millimeter wave image datasets and divide the images into training sets and test sets; S2. Constructing a system model as claimed in any one of claims 1 to 4; S3. Use the collected training set to train the system model: downsample the training set into a high-resolution feature image and output it to three branch networks; the parsing branch network maintains the scale of the high-resolution feature image, merges it with the two different-scale feature maps of the aggregation branch network, and selectively learns semantic features from the aggregation branch network; the aggregation branch network performs a series of downsampling and upsampling on the input image, merges the information of the three scales, and then realizes the recursive alignment fusion of the feature layer through the aggregation module; the prediction boundary branch network maintains the image scale as the high-resolution feature image scale, and merges it with the different-scale feature maps of the aggregation branch network to extract the high-frequency features of the image to predict the boundary area; after the processing of each branch network is completed, the three-branch fusion processing is performed, and the mathematical expression of the fusion strategy is: In the formula, Represent the vectors mapped by the three branch networks, F out and f represents the collective function of convolution, normalization and ReLU; S4, test the trained model with the test set, and obtain the prediction effect graph through the segmentation head module; The segmentation head module is the key part for extracting the final segmentation result from the feature map. The function uses an upsampling operation to restore the image to the resolution of the input image. It contains several convolutional layers for further enhancing features and generating category predictions.

6. The model training method of the millimeter wave image overlapping target AI recognition system according to claim 5 is characterized in that: The aggregation branch network performs a series of downsampling and upsampling on the input image, merges the information of three scales, and then performs pooling, normalization and convolution operations through the aggregation module to achieve better aggregation of contextual information.

7. An AI recognition method for overlapping targets in millimeter wave images, characterized in that: The millimeter wave image is subjected to AI recognition of overlapping targets using the AI ​​recognition system as described in any one of claims 1 to 4 or the AI ​​recognition system trained by the model training method as described in any one of claims 5 or 6.

Images