Millimeter wave image overlapping target ai recognition method, system and model training method

Abstract

The application discloses an AI identification method, system and model training method for millimeter wave image overlapping targets based on a three-branch network, and solves the security target detection problem in millimeter wave images with low resolution, mutual interference or mutual overlap through collaborative work of three independent branch networks, namely, an analysis branch network, an aggregation branch network and a prediction boundary branch network. The analysis branch network saves the detailed information in a high-resolution feature map; the aggregation branch network realizes the feature position offset alignment of high and low resolution feature maps through a recursive fusion module; and the prediction boundary branch network extracts high-frequency features and enhances the perception and prediction of target edges. The multi-branch network structure effectively realizes the segmentation and identification of images through the fusion between branches. While improving the segmentation performance of the model, the model significantly reduces the calculation complexity and the detection error and missed detection probability in millimeter wave images, and finally realizes the accurate detection of overlapping objects.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically, to an AI recognition method, system, and model training method for overlapping targets in millimeter-wave images. Background Technology

[0002] Electromagnetic waves with wavelengths of 1–10 millimeters are called millimeter waves, lying between microwaves and far-infrared waves, possessing characteristics of both. The theory and technology of millimeter waves represent the extension of microwaves to higher frequencies and the development of light waves to lower frequencies. As a type of radio wave, millimeter waves do not have ionizing radiation issues compared to X-rays, and low-power millimeter wave signals are harmless to the human body. In addition to sharing the characteristics of infrared and microwaves, millimeter waves also possess unique properties. Compared to microwaves, millimeter waves have better directivity, stronger anti-interference capabilities, excellent detection performance, and better plasma penetration. In human security checks, millimeter wave imaging technology can not only penetrate ordinary clothing to detect concealed metallic contraband but also detect non-metallic contraband. Because the wavelength of millimeter waves is harmless to the human body, millimeter waves have shown good application results in human detection. However, due to the wavelength diffraction limit, the resolution of millimeter wave images is generally low. Moreover, due to limitations in the performance of millimeter wave detectors and imaging algorithms, the signal-to-noise ratio of millimeter wave images is much lower than that of optical images, resulting in larger pixel sizes and relatively blurry images, making it difficult for the human eye to identify information in the images.

[0003] Therefore, if existing technologies are implemented in human security checks, there will be shortcomings. At this point, it is necessary to use appropriate algorithm models to further improve the accuracy and efficiency of the detection. Summary of the Invention

[0004] To address the aforementioned shortcomings of existing technologies, this invention provides an AI-based method, system, and model training method for recognizing overlapping targets in millimeter-wave images. Based on a multi-branch fusion network, it simultaneously performs multiple tasks, including preserving detailed information in high-resolution feature maps, effectively aggregating contextual information, and extracting high-frequency features to predict boundary regions, thereby enhancing the perception of target edges.

[0005] To achieve the above objectives, in a first aspect, the present invention provides an AI recognition system for overlapping targets in millimeter-wave images, characterized in that it comprises three branch networks and their branch fusion modules:

[0006] The parsing branch network is used to parse and preserve detailed information in the high-resolution feature map, while fusing it with the two feature maps of the aggregation branch network at different scales, selectively learning semantic features from the aggregation branch network;

[0007] An aggregation branch network is used to aggregate contextual information to resolve long-range dependencies; the aggregation branch network includes a recursive fusion module, which achieves recursive alignment fusion of feature layers by learning the feature position offsets of a single high-resolution feature map and multiple adjacent low-resolution feature maps;

[0008] The prediction boundary branch network is merged with feature maps of different sizes from the aggregation branch network to extract high-frequency features of the image to predict boundary regions.

[0009] The mathematical expression for the fusion strategy in the branch fusion module is:

[0010]

[0011] In the formula, F represents the vectors mapped by the three branch networks, respectively. out f represents the set functions of convolution, normalization, and ReLU.

[0012] The parsing branch network preserves high-resolution semantic information and combines it with the rich and accurate semantic information provided by the aggregation branch network. The semantic information provided by the aggregation branch network effectively prevents the parsing branch network from being suppressed by excessive background or global context information, allowing the detail branch to selectively learn high-level semantic information. The prediction boundary branch network is concatenated with the semantic information provided by the aggregation branch network, allowing the boundary branch network to directly add the feature map provided by the aggregation branch network. The boundary branch network's sensitivity to changes enables accurate prediction of boundary region changes, thus compensating for the problem of insufficient detail in the predicted target boundary depiction caused by averaging when fusing conventional high-resolution detail features with low-resolution contextual semantics. This results in more refined edge segmentation of the target. While the aggregation branch network is semantically rich and provides more accurate semantics, it loses too much spatial and geometric detail, especially for boundary regions and small objects. The detail branch better preserves spatial details. Finally, when fusing the features provided by the three branches, the model trusts the detail branch more along the boundary region and uses contextual features to fill in the internal regions of the object, achieving more accurate image recognition.

[0013] Furthermore, the recursive fusion module of the aggregation branch network adopts multi-scale feature fusion, which performs pooling, normalization and convolution fusion operations on feature maps of different scales.

[0014] Furthermore, the parsing branch network includes a first fusion module for fusing with two feature maps of different scales from the aggregation branch network; the prediction boundary branch network includes a second fusion module for merging with feature maps of different sizes from the aggregation branch network.

[0015] Furthermore, the branch fusion module further trusts the parsed branch network and the predicted boundary branch network along the boundary region, and uses the contextual information features of the aggregated branch network to fill the region inside the object.

[0016] Secondly, the present invention provides a model training method for an AI recognition system for overlapping targets in millimeter-wave images, characterized by comprising the following steps:

[0017] S1. Collect millimeter-wave image datasets and divide the images into training and test sets;

[0018] S2. Construct the AI ​​recognition system model as described above;

[0019] S3. Train the system model using the collected training set: Downsample the training set into high-resolution feature images and output them to three branch networks; the parsing branch network maintains the high-resolution feature image scale and fuses with the two different scale feature maps of the aggregation branch network, selectively learning semantic features from the aggregation branch network; the aggregation branch network performs a series of downsampling and upsampling on the input image, merging the information at the three scales, and then achieving recursive alignment and fusion of the feature layers through the aggregation module; the boundary prediction branch network maintains the image scale at the high-resolution feature image scale and merges with the different scale feature maps of the aggregation branch network, extracting high-frequency features of the image to predict the boundary region; after each branch network is processed, a three-branch fusion process is performed, and the mathematical expression of the fusion strategy is:

[0020]

[0021] In the formula, F represents the vectors mapped by the three branch networks, respectively. out f represents the set function of convolution, normalization, and ReLU;

[0022] S4. Test the trained model using the test set and obtain the prediction result map through the segmentation head module. The segmentation head module extracts the key parts of the final segmentation result from the feature map. It uses upsampling operation to restore the image to the resolution of the input image. It contains several convolutional layers to further enhance features and generate category predictions.

[0023] Furthermore, the aggregation branch network performs a series of downsampling and upsampling on the input image, merging information from the three scales. Then, the aggregation module performs pooling, normalization, and convolution operations to achieve better aggregation of contextual information.

[0024] Finally, the present invention provides an AI recognition method for overlapping targets in millimeter-wave images, characterized in that the millimeter-wave images are used to perform AI recognition of overlapping targets using the AI ​​recognition system described above or an AI recognition system trained using the model training method described above.

[0025] In practical applications of contraband detection on the human body, there are often multiple targets, with several objects overlapping in the same location, increasing the difficulty of image recognition. Compared with existing technologies, this invention has the following technical advantages:

[0026] The recognition system and method of this invention can accurately detect and segment overlapping targets in low-resolution millimeter-wave images. The multi-branch network structure can simultaneously perform multiple tasks: preserving detailed information in high-resolution feature maps; effectively aggregating contextual information; and extracting high-frequency features to predict boundary regions, thereby enhancing the perception of target edges. The branch fusion module utilizes boundary features to guide the fusion of detailed and contextual representations, achieving better segmentation results. This invention achieves fast and accurate target recognition in overlapping millimeter-wave images.

[0027] More specifically, this invention solves the problem of security target detection in low-resolution, interfering, or overlapping millimeter-wave images by employing three independent branches—a parsing branch network, an aggregation branch network, and a prediction boundary branch network—working collaboratively. The parsing branch network preserves detailed information from the high-resolution feature maps; the aggregation branch network aligns the feature positions of high- and low-resolution feature maps through a recursive fusion module; and the prediction boundary branch network extracts high-frequency features to enhance the perception and prediction of target edges. The multi-branch network structure effectively achieves image segmentation and recognition through branch fusion. While improving the model's segmentation performance, this model significantly reduces computational complexity and decreases the probability of detection errors and missed detections in millimeter-wave images, ultimately achieving accurate detection of overlapping objects. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the architecture of an AI recognition system in one embodiment of the present invention.

[0029] Figure 2 This is a schematic diagram of the architecture of the first fusion module in one embodiment of the present invention.

[0030] Figure 3 This is a schematic diagram of the architecture of the second fusion module in one embodiment of the present invention.

[0031] Figure 4 This is a schematic diagram of the branch fusion module architecture in one embodiment of the present invention.

[0032] Figure 5 This is a flowchart of a model training method for an AI recognition system according to an embodiment of the present invention. Detailed Implementation

[0033] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the invention.

[0034] In the following detailed description, numerous specific details are set forth to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that well-known algorithms and models (such as pooling algorithms, convolutional networks, and normalization algorithms) are not shown in detail to avoid obscuring the spirit of the invention.

[0035] Furthermore, the execution order of actions, steps, etc. in the apparatus and methods shown in the claims, specification, and drawings can be implemented in any order, unless a specific order is explicitly specified, and as long as the output of the preceding processing is not used in the subsequent processing.

[0036] Example

[0037] See Figure 1 This embodiment provides an AI recognition system for overlapping targets in millimeter-wave images, including three branch networks and their branch fusion modules. First, the input image is downsampled to 1 / 8 of its original size. Then, this feature map is used in the three branch networks respectively. The upper branch is a detail parsing branch network, used to parse and save detailed information in the high-resolution feature map, and simultaneously fuses it with two feature maps of different scales from the aggregation branch network. Semantic features are selectively learned from the aggregation branch network, thereby enhancing the detail parsing capability. More specifically, the parsing branch network includes a first fusion module for fusing with the two feature maps of different scales from the aggregation branch network. See also... Figure 2 First, the two branches are residually connected. Then, the aggregation branch undergoes convolution, batch normalization, and upsampling, while the parsing branch undergoes convolution and batch normalization. Next, the outputs of the two branches are multiplied and added to fuse the feature maps. This effectively fuses details and contextual information, enhancing feature representation capabilities and improving model performance. The parsing branch network ensures that important semantic information is not lost when processing image details, while avoiding interference from background information.

[0038] The aggregation branch network is the main branch, used to aggregate contextual information to resolve long-range dependencies. This aggregation branch network includes a recursive fusion module, which learns the feature position offsets of a single high-resolution feature map and multiple adjacent low-resolution feature maps to achieve recursive alignment and fusion of feature layers. More specifically, in the aggregation branch network, it first undergoes two downsampling operations to obtain scales of 1 / 16 and 1 / 32, respectively, serving as backup support for the other two branch networks, providing them with the necessary information. Then, a series of upsampling and downsampling operations are performed to merge the information from the three scales, followed by pooling, normalization, and convolution operations in the aggregation module to achieve better aggregation of contextual information.

[0039] The next branch is the boundary prediction branch network, which maintains the image size at 1 / 8 of the original size. It is merged with feature maps of different sizes from the aggregation branch network to extract high-frequency features of the image to predict the boundary region. More specifically, the boundary prediction branch network includes a second fusion module for merging feature maps of different sizes from the aggregation branch network. See also Figure 3 First, the two branches are residually connected. Then, the boundary branch outputs directly, while the aggregation branch is processed through convolution, batch normalization, and upsampling. Finally, the results of the two branches are summed and output. This module preserves the original information of the input features, enhances feature representation, and improves the model's expressiveness and training efficiency. The design of the boundary branch network enables the model to more accurately identify boundary regions in images, thus better handling problems of blurred or discontinuous edges during segmentation.

[0040] See Figure 4 The three-branch fusion process is performed after each branch network in the identification system has completed its processing. The mathematical expression for the branch fusion module is:

[0041]

[0042] Through this fusion strategy, the model can comprehensively utilize the advantages of each branch to achieve more accurate image segmentation and target recognition. In the formula, These represent the vectors mapped by the three branch networks, and F... out`f` represents the set function of convolution, normalization, and ReLU. T1 and T3 are the first and third branches, respectively, which first process themselves. Then, the activation function of the third branch assigns values ​​between 0 and 1, with the sum of the two terms being 1. These two terms are then tensor-producted with the other two branches, and the results are summed to obtain the final vector value. Finally, the set function is used to perform a series of convolution and normalization operations to obtain the output of the final fusion module. The third branch is the boundary branch, which uses the weights of boundary features to guide the allocation of the fusion ratio, thereby achieving dynamic feature fusion. This effectively enhances feature representation capabilities and improves the model's performance and robustness.

[0043] See Figure 5 This embodiment also provides a model training method for the above AI recognition system, including the following steps:

[0044] S1. Collect millimeter-wave image datasets and divide the images into training and test sets;

[0045] S2. Construct the AI ​​recognition system model as described above;

[0046] S3. Train the system model using the collected training set: Downsample the training set into high-resolution feature images and output them to three branch networks; the parsing branch network maintains the high-resolution feature image scale and fuses with the two different scale feature maps of the aggregation branch network, selectively learning semantic features from the aggregation branch network; the aggregation branch network performs a series of downsampling and upsampling on the input image, merging the information at the three scales, and then achieving recursive alignment and fusion of the feature layers through the aggregation module; the boundary prediction branch network maintains the image scale at the high-resolution feature image scale and merges with the different scale feature maps of the aggregation branch network, extracting high-frequency features of the image to predict the boundary region; after each branch network is processed, a three-branch fusion process is performed, and the mathematical expression of the fusion strategy is:

[0047]

[0048] In the formula, F represents the vectors mapped by the three branch networks, respectively. out f represents the set function of convolution, normalization, and ReLU;

[0049] S4. Test the trained model using the test set and obtain the prediction result map through the segmentation head module. The segmentation head module extracts the key parts of the final segmentation result from the feature map. It uses upsampling operation to restore the image to the resolution of the input image. It contains several convolutional layers to further enhance features and generate category predictions.

[0050] To facilitate understanding of the implementation effect of the three-branch network model in this invention, we conducted experiments on a dataset of millimeter-wave images for human detection. Using two-branch and single-branch network models as baselines, all models were evaluated through single-scale inference with an input image resolution of 640×640. Parameters used to test network performance included Mean Intersection over Union (MIoU), number of parameters, and FLOPs (Floating-Point Operations). MIoU measures the similarity between the predicted result and the true label in a semantic segmentation task; the number of parameters represents the number of parameters that need to be learned during training; and FLOPs represents the number of floating-point operations performed during inference, typically used to measure algorithm complexity.

[0051] Experimental results show that the MIoU value of the two-branch network is 5.96% higher than that of the single-branch network; the MIoU value of the three-branch network is further improved by 3.05% compared to the two-branch network. At the same time, the three-branch network only increases the number of parameters by a small amount compared to the two-branch network, but significantly reduces the model's FLOPs, indicating that the three-branch network has lower computational complexity while maintaining high efficiency and accuracy.

[0052] In summary, this invention provides an AI recognition method, system, and model training method for overlapping targets in millimeter-wave images based on a three-branch network model. The parsing branch preserves high resolution and combines it with the rich and accurate semantic information provided by the aggregation branch. The semantic information provided by the aggregation branch effectively prevents the parsing branch from being suppressed by excessive background or global context information, allowing the detail branch to selectively learn high-level semantic information. The prediction boundary branch is concatenated with the semantic information provided by the aggregation branch, allowing the boundary branch to directly add the feature map provided by the aggregation branch. The boundary branch's sensitivity to changes enables accurate prediction of boundary region changes, thus compensating for the problem of insufficient detail in the predicted target boundary depiction caused by averaging when fusing conventional high-resolution detail features with low-resolution contextual semantics. This results in more refined edge segmentation of the target. While the aggregation branch is semantically rich and provides more accurate semantics, it loses too much spatial and geometric detail, especially for boundary regions and small objects. The detail branch better preserves spatial details. Finally, when fusing the features provided by the three branches, the model trusts the detail branch more along the boundary region and uses contextual features to fill in the internal regions of the object, achieving more accurate image recognition.

[0053] Those skilled in the art should understand that variations can be implemented by combining existing technology with the above embodiments, which will not be elaborated here. Such variations do not affect the essence of the present invention, and will not be elaborated here either.

[0054] The preferred embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and the devices and structures not described in detail should be understood as being implemented in a conventional manner in the art. Any person skilled in the art can make many possible variations and modifications to the technical solutions of the present invention using the methods and techniques disclosed above, or modify them into equivalent embodiments with equivalent changes, without departing from the scope of the present invention. This does not affect the essential content of the present invention. Therefore, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the present invention's technical solutions still fall within the protection scope of the present invention.

[0055] Those skilled in the art will recognize that the units, i.e., algorithm steps, of the various examples described in connection with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

Claims

1. An AI recognition system for overlapping targets in millimeter-wave images, characterized in that, Includes three branch networks and their branch fusion modules: The parsing branch network is used to parse and preserve detailed information in the high-resolution feature map, while fusing it with the two feature maps of the aggregation branch network at different scales, selectively learning semantic features from the aggregation branch network; The parsing branch network includes a first fusion module for fusing with two feature maps of different scales from the aggregation branch network; the prediction boundary branch network includes a second fusion module for merging with feature maps of different sizes from the aggregation branch network. Aggregate branch networks to aggregate contextual information in order to resolve remote dependencies; The aggregation branch includes a recursive fusion module, which learns the feature position offsets of a single high-resolution feature map and multiple adjacent low-resolution feature maps to achieve recursive alignment fusion of the feature layers; the recursive fusion module of the aggregation branch network adopts multi-scale feature fusion, which performs pooling, normalization and convolution fusion operations on feature maps of different scales. The prediction boundary branch network is merged with feature maps of different sizes from the aggregation branch network to extract high-frequency features of the image to predict boundary regions. The mathematical expression for the fusion strategy in the branch fusion module is: In the formula, These represent the vectors mapped by the three branch networks, respectively. and The set of functions representing convolution, normalization, and ReLU; the branch fusion module trusts the parsing branch network and the predicted boundary branch network more along the boundary region, and uses the contextual information features of the aggregated branch network to fill the region inside the object.

2. A model training method for an AI recognition system of overlapping targets in millimeter-wave images, characterized in that, Includes the following steps: S1. Collect millimeter-wave image datasets and divide the images into training and test sets; S2. Construct the system model as described in claim 1; S3. Train the system model using the collected training set: Downsample the training set into high-resolution feature images and output them to three branch networks; the parsing branch network maintains the high-resolution feature image scale and fuses with the two different scale feature maps of the aggregation branch network, selectively learning semantic features from the aggregation branch network; the aggregation branch network performs a series of downsampling and upsampling on the input image, merging the information at the three scales, and then achieving recursive alignment and fusion of the feature layers through the aggregation module; the boundary prediction branch network maintains the image scale at the high-resolution feature image scale and merges with the different scale feature maps of the aggregation branch network, extracting high-frequency features of the image to predict the boundary region; after each branch network is processed, a three-branch fusion process is performed, and the mathematical expression of the fusion strategy is: In the formula, These represent the vectors mapped by the three branch networks, respectively. and Set functions representing convolution, normalization, and ReLU; S4. Test the trained model using the test set and obtain the prediction results through the segmentation head module; The segmentation head module extracts the key parts of the final segmentation result from the feature map. It uses upsampling operations to restore the image to the resolution of the input image and contains several convolutional layers to further enhance features and generate category predictions.

3. The model training method for the millimeter-wave image overlapping target AI recognition system according to claim 2, characterized in that, The aggregation branch network performs a series of downsampling and upsampling on the input image, merging information from the three scales. Then, the aggregation module performs pooling, normalization, and convolution operations to achieve better aggregation of contextual information.

4. An AI-based method for recognizing overlapping targets in millimeter-wave images, characterized in that, AI recognition of overlapping targets is performed on millimeter-wave images using the AI ​​recognition system as described in claim 1 or the model training method as described in claim 2 or 3.

Images