Visual detection method for weak and small target driven by radar detection information

Abstract

The invention relates to the technical field of visual detection, in particular to a weak and small target visual detection method driven by radar detection information, which comprises the following steps of: giving an azimuth signal of a radar detection target, and extracting azimuth characteristics of the radar detection target; performing affine transformation on the orientation features to generate a space weight, a channel weight and an offset of the weak and small target visual image; extracting visual features of the visual image; performing enhancement processing on the visual features through the spatial weight, the channel weight and the bias of the weak and small target visual image to obtain enhanced visual features; and mapping the enhanced visual features and the orientation features to the same dimension through linear projection. According to the weak and small target visual detection method, the azimuth features are brought into the weak and small target visual detection method, extraction and fusion of multi-source heterogeneous data features are completed, information complementation is carried out by using abstract and condensed physical meanings contained behind radar detection target azimuth information, the weak and small target visual detection efficiency is enhanced, and the weak and small target detection precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of visual detection technology, and in particular to a small target visual detection method driven by radar detection information. Background Art

[0002] Visual inspection is to use machines instead of human eyes to make measurements and judgments. Visual inspection refers to converting the captured target into image signals through machine vision products, transmitting them to a dedicated image processing system, and converting them into digital signals based on pixel distribution, brightness, color and other information. Small targets are not easy to visually inspect because of their low flying altitude, slow speed and small size.

[0003] Existing small target detection methods need to solve the following problems: pure visual detection methods have a single data source and cannot fully utilize auxiliary information to complete small target detection; visual feature extraction networks cannot adapt to the needs of multi-source heterogeneous data fusion; the physical meanings behind multi-source data are different, so how to perform small target visual detection within a unified framework. Summary of the invention

[0004] To this end, the present invention provides a small target visual detection method driven by radar detection information to solve the above-mentioned problems.

[0005] The present invention provides the following technical solution: a radar detection information driven dim small target visual detection method, comprising the following steps:

[0006] Given the radar detection target azimuth signal, extract the radar detection target azimuth feature;

[0007] Perform affine transformation on orientation features to generate spatial weights, channel weights and biases of the visual image of weak targets;

[0008] Extract visual features from visual images;

[0009] The visual features are enhanced by using the spatial weight, channel weight and bias of the weak target visual image to obtain enhanced visual features;

[0010] The enhanced visual features and orientation features are mapped to the same dimension through linear projection, and position embedding is added to obtain multimodal features;

[0011] Localizing small objects through multimodal features.

[0012] As a preferred solution of the present invention, the visual feature extraction network extracts the features of the visual image information. In the middle layer of the pre-trained visual feature extraction network, the extracted radar detection target orientation features are: l and d represent the token number and feature dimension of the orientation feature.

[0013] As a preferred solution of the present invention, according to the visual image I, the visual features are extracted The information interaction attention module then maps the feature of the i-th stage of the visual feature Modulate and generate adaptive weights and biases related to the visual image of weak targets, namely channel weights Spatial weight and bias B i .

[0014] As a preferred solution of the present invention, for the channel weight and bias B i , aggregate the information from the orientation features through average pooling, map this information to the same dimension as the visual feature channel using a fully connected layer, and apply the tanh activation function to obtain the channel weight and bias B i ∈[-1, 1], and use the fully connected layer to further refine the features, and get Formula 1;

[0015]

[0016] Spatial weight The same process as in formula 1 is used to learn the visual features related to the radar detection direction, and the deconvolution layer is used to generate the visual features. The spatial dimension attention weight is used to learn the parameters to adapt to the feature extraction process, retain more spatial dimension information, reduce the number of parameters, and obtain Formula 2;

[0017]

[0018] In formula 2, using spatial weight Adjust the extraction process of visual features to obtain Formula 3;

[0019]

[0020] Given visual features and orientation features Use a fully connected layer to perform linear projection to the same dimension and add position embedding to get formula 4;

[0021]

[0022] In formula 4, pos(·) is the position embedding function. Where a and b represent the number of symbols and dimensions, and the visual features F are fused through self-attention and cross-attention modules. v and orientation feature D l Obtain the visual orientation feature F after orientation feature enhancement + , we get formula 5;

[0023]

[0024] The visual features are enhanced by using the spatial weight, channel weight and bias of the weak target visual image to obtain enhanced visual features;

[0025] In Formula 5, MHSA(·) and MHCA(·) are the self-attention and cross-attention modules respectively. The cross-attention module is used to transform D′ l and F + Combined to obtain the characteristic attributes of the described target, we get Formula 6;

[0026] F m =MHCA(F + , D′ l ) Sixth, finally obtain the multimodal feature F m .

[0027] As a preferred solution of the present invention, multi-layer cross-modal interaction is used to aggregate and enhance the visual features of small targets and adjust the extraction process of visual features, which is used to infer the bounding box coordinates of small targets and align the visual features and orientation features to accurately locate small targets.

[0028] As a preferred solution of the present invention, the multimodal feature F m It is ultimately used for target localization, integrating orientation features with visual features for cross-modal interaction.

[0029] As a preferred solution of the present invention, the orientation features and enhanced visual features are decoded and then interactively fused to reduce the ambiguity in understanding the target description during the reasoning process, and multi-layer cross-modal interactions are used to aggregate and enhance the visual features of weak targets.

[0030] As a preferred solution of the present invention, the visual feature The h, w, and c in the figure represent the height, width, and number of channels of the visual feature, respectively.

[0031] A radar detection information-driven dim target visual detection system comprises a detection point, wherein the output end of the detection point is electrically connected to a visual feature extraction network, the output end of the visual feature extraction network is electrically connected to an information interaction attention module, the output end of the information interaction attention module is electrically connected to an orientation feature extraction network, the receiving end of the orientation feature extraction network is electrically connected to a radar detection orientation model, the receiving end of the radar detection orientation model is electrically connected to a radar detection target orientation signal, the output end of the orientation feature extraction network is electrically connected to a multi-level feature interaction enhancement module, the output end of the orientation feature extraction network is electrically connected to a cross-modal information fusion decoder, the output end of the cross-modal information fusion decoder is electrically connected to a positioning module, the output end of the visual feature extraction network is electrically connected to the receiving end of the multi-level feature interaction enhancement module, and the output end of the multi-level feature interaction enhancement module is electrically connected to the receiving end of the cross-modal information fusion decoder.

[0032] As a preferred solution of the present invention, the orientation feature extraction network adjusts the visual feature extraction process by adding radar detection target orientation signals into the visual feature extraction network, and is used to align visual features and orientation features to accurately locate small targets.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] In the present invention, by incorporating the orientation features into the visual detection method for small targets, the extraction and fusion of multi-source heterogeneous data features are completed, and the abstract and condensed physical meaning behind the radar detection target orientation information is used to complement information, thereby enhancing the visual detection efficiency of small targets and improving the detection accuracy of small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of the overall composition system of the present invention;

[0036] Figure 2 This is a flow chart of the information interaction attention module system in the present invention;

[0037] Figure 3 This is a flow chart of the multi-level feature interactive fusion decoder system in the present invention.

[0038] In the figure: 1. Detection point; 2. Radar detection target direction signal; 3. Visual feature extraction network; 4. Information interaction attention module; 5. Radar detection direction model; 6. Direction feature extraction network; 7. Cross-modal information fusion decoder; 8. Positioning module. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] See also Figure 1-3 The technical solution provided by the present invention specifically includes the following embodiments:

[0041] Embodiment 1: A method for visually detecting small targets driven by radar detection information, comprising the following steps:

[0042] Given the radar detection target azimuth signal, extract the radar detection target azimuth feature;

[0043] Perform affine transformation on orientation features to generate spatial weights, channel weights and biases of the visual image of weak targets;

[0044] Extract visual features from visual images;

[0045] The visual features are enhanced by using the spatial weight, channel weight and bias of the weak target visual image to obtain enhanced visual features;

[0046] The enhanced visual features and orientation features are mapped to the same dimension through linear projection, and position embedding is added to obtain multimodal features;

[0047] Localizing small objects through multimodal features.

[0048] The visual feature extraction network extracts the features of visual image information. In the middle layer of the pre-trained visual feature extraction network, the extracted radar detection target orientation features are l and d represent the token number and feature dimension of the orientation feature.

[0049] Extract visual features based on visual image I The information interaction attention module then maps the feature of the i-th stage of the visual feature Modulate and generate adaptive weights and biases related to the visual image of weak targets, namely channel weights Spatial weight and bias B i .

[0050] For channel weight and bias B i , aggregate the information from the orientation features through average pooling, map this information to the same dimension as the visual feature channel using a fully connected layer, and apply the tanh activation function to obtain the channel weight and bias B i ∈[-1, 1], and use the fully connected layer to further refine the features, and get Formula 1;

[0051]

[0052] Spatial weight The same process as in formula 1 is used to learn the visual features related to the radar detection direction, and the deconvolution layer is used to generate the visual features. The spatial dimension attention weight is used to learn the parameters to adapt to the feature extraction process, retain more spatial dimension information, reduce the number of parameters, and obtain Formula 2;

[0053]

[0054] In formula 2, using spatial weight Adjust the extraction process of visual features to obtain Formula 3;

[0055]

[0056] Given visual features and orientation features Use a fully connected layer to perform linear projection to the same dimension and add position embedding to get formula 4;

[0057]

[0058] In formula 4, pos(·) is the position embedding function. Where a and b represent the number of symbols and dimensions, and the visual features F are fused through self-attention and cross-attention modules. v and orientation feature D l Obtain the visual orientation feature F after orientation feature enhancement + , we get formula 5;

[0059]

[0060] The visual features are enhanced by using the spatial weight, channel weight and bias of the weak target visual image to obtain enhanced visual features;

[0061] In Formula 5, MHSA(·) and MHCA(·) are the self-attention and cross-attention modules respectively. The cross-attention module is used to transform D′ l and F + Combined to obtain the characteristic attributes of the described target, we get Formula 6;

[0062] F m =MHCA(F + , D′ l) Sixth, finally obtain the multimodal feature F m .

[0063] Multi-layer cross-modal interactions are used to aggregate and enhance the visual features of small targets and adjust the visual feature extraction process to infer the bounding box coordinates of small targets and align the visual features and orientation features to accurately locate small targets.

[0064] Multimodal feature F m It is ultimately used for target localization, integrating orientation features with visual features for cross-modal interaction.

[0065] The orientation features and enhanced visual features are decoded and then interactively fused to reduce the ambiguity in the understanding of the target description during the reasoning process, and multi-layer cross-modal interactions are used to aggregate and enhance the visual features of weak targets.

[0066] Visual Features The h, w, and c in it represent the height, width, and number of channels of the visual feature, respectively;

[0067] The present invention incorporates the radar detection target azimuth information into the small target visual detection system and the multimodal information fusion design in the small target visual detection scenario. The present invention incorporates the azimuth feature into the small target visual detection method, that is, completes the extraction and fusion of multi-source heterogeneous data features, and uses the abstract and condensed physical meaning behind the radar detection target azimuth information to complement information, enhance the visual detection efficiency of small targets, and improve the small target detection accuracy.

[0068] In this scheme, a radar detection information-driven small target visual detection method is used. When the radar detection target azimuth signal is given, the extracted radar detection target azimuth feature is Where l and d represent the number of tokens and feature dimensions of the orientation feature. l Feed it to the information interaction attention module to generate weights and biases for the visual feature extraction network. Given a visual image I, extract visual features Where h, w, and c represent the height, width, and number of channels of the visual feature, respectively. v and orientation feature F l Through linear projection mapping to the same dimension, the orientation features and enhanced visual features are decoded and then interactively fused. Multi-layer cross-modal interaction is used to aggregate and enhance the visual features of small targets and adjust the extraction process of visual features to infer the bounding box coordinates of small targets and align the visual features and orientation features to accurately locate small targets.

[0069] Embodiment 2: A small target visual detection system driven by radar detection information, comprising a detection point 1, the output end of the detection point 1 is electrically connected to a visual feature extraction network 3, the output end of the visual feature extraction network 3 is electrically connected to an information interaction attention module 4, the output end of the information interaction attention module 4 is electrically connected to an orientation feature extraction network 6, the receiving end of the orientation feature extraction network 6 is electrically connected to a radar detection orientation model 5, the receiving end of the radar detection orientation model 5 is electrically connected to a radar detection target orientation signal 2, the output end of the orientation feature extraction network 6 is electrically connected to a multi-level feature interaction enhancement module 6, the output end of the orientation feature extraction network 6 is electrically connected to a cross-modal information fusion decoder 7, the output end of the cross-modal information fusion decoder 7 is electrically connected to a positioning module 8, the output end of the visual feature extraction network 3 is electrically connected to the receiving end of the multi-level feature interaction enhancement module 6, and the output end of the multi-level feature interaction enhancement module 6 is electrically connected to the receiving end of the cross-modal information fusion decoder 7.

[0070] The orientation feature extraction network 6 adjusts the visual feature extraction process by adding the radar detection target orientation signal 2 to the visual feature extraction network 3, so as to align the visual features and the orientation features to accurately locate small targets.

[0071] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the invention.

Claims

1. A small target visual detection method driven by radar detection information, characterized in that: The steps include: Given the radar detection target azimuth signal, extract the radar detection target azimuth feature; Perform affine transformation on orientation features to generate spatial weights, channel weights and biases of the visual image of weak targets; Extract visual features from visual images; The visual features are enhanced by using the spatial weight, channel weight and bias of the weak target visual image to obtain enhanced visual features; The enhanced visual features and orientation features are mapped to the same dimension through linear projection, and position embedding is added to obtain multimodal features; Localizing small objects through multimodal features.

2. The method for visually detecting small targets driven by radar detection information according to claim 1, characterized in that: The visual feature extraction network extracts the features of visual image information. In the middle layer of the pre-trained visual feature extraction network, the extracted radar detection target orientation features are l and d represent the token number and feature dimension of the orientation feature.

3. The method for visually detecting small targets driven by radar detection information according to claim 1, characterized in that: Extract visual features based on visual image I Feature mapping of the i-th stage of visual features Modulate and generate channel weights related to the visual image of weak targets Spatial weight and bias B i .

4. The method for visually detecting small targets driven by radar detection information according to claim 1, characterized in that: For channel weight and bias B i , aggregate the information from the orientation features through average pooling, use the fully connected layer to map the information to the same dimension as the visual feature channel, and apply the tanh activation function to obtain the channel weight and bias B i ∈[-1, 1], and use the fully connected layer to further refine the features, and get Formula 1; Spatial weight The same process as in formula 1 is used to learn the visual features related to the radar detection direction, and the deconvolution layer is used to generate the visual features. The spatial dimension attention weight is used to learn the parameters to adapt to the feature extraction process, retain more spatial dimension information, reduce the number of parameters, and obtain Formula 2; In formula 2, using spatial weight Adjust the extraction process of visual features to obtain Formula 3; Given visual features and orientation features Use a fully connected layer to perform linear projection to the same dimension and add position embedding to get formula 4; In formula 4, pos(·) is the position embedding function. Where a and b represent the number of symbols and dimensions, and the visual features F are fused through self-attention and cross-attention modules. v and orientation feature D l Obtain the visual orientation feature F after orientation feature enhancement + , we get formula 5; The visual features are enhanced by using the spatial weight, channel weight and bias of the weak target visual image to obtain enhanced visual features; In Formula 5, MHSA(·) and MHCA(·) are the self-attention and cross-attention modules respectively. The cross-attention module is used to transform D′ l and F + Combined to obtain the characteristic attributes of the described target, we get Formula 6; F m =MHCA(F + , D′ l ) six Finally, the multimodal feature F is obtained m .

5. The method for visually detecting small targets driven by radar detection information according to claim 1, characterized in that: Multi-layer cross-modal interactions are used to aggregate and enhance the visual features of small targets and adjust the visual feature extraction process to infer the bounding box coordinates of small targets and align the visual features and orientation features to accurately locate small targets.

6. The method for visually detecting small targets driven by radar detection information according to claim 4, characterized in that: The multimodal feature F m Used for target localization, cross-modal interaction between orientation features and visual features.

7. The radar detection information driven small target visual detection method according to claim 4, characterized in that: The orientation features and enhanced visual features are decoded and then interactively fused, and the visual features of weak targets are aggregated and enhanced using multi-layer cross-modal interactions.

8. The radar detection information driven small target visual detection method according to claim 3, characterized in that: Visual features The h, w, and c in the figure represent the height, width, and number of channels of the visual feature, respectively.

9. A small target visual detection system driven by radar detection information, characterized in that: The invention comprises a detection point (1), wherein the output end of the detection point (1) is electrically connected to a visual feature extraction network (3), the output end of the visual feature extraction network (3) is electrically connected to an information interaction attention module (4), the output end of the information interaction attention module (4) is electrically connected to an orientation feature extraction network (6), the receiving end of the orientation feature extraction network (6) is electrically connected to a radar detection orientation model (5), the receiving end of the radar detection orientation model (5) is electrically connected to a radar detection target orientation signal (2), and the orientation The output end of the feature extraction network (6) is electrically connected to the multi-level feature interaction enhancement module (6), the output end of the orientation feature extraction network (6) is electrically connected to the cross-modal information fusion decoder (7), the output end of the cross-modal information fusion decoder (7) is electrically connected to the positioning module (8), the output end of the visual feature extraction network (3) is electrically connected to the receiving end of the multi-level feature interaction enhancement module (6), and the output end of the multi-level feature interaction enhancement module (6) is electrically connected to the receiving end of the cross-modal information fusion decoder (7).

10. The radar detection information driven small target visual detection system according to claim 9, characterized in that: The orientation feature extraction network (6) adjusts the visual feature extraction process by adding the radar detection target orientation signal (2) to the visual feature extraction network (3), and is used to align the visual features and the orientation features to accurately locate the weak target.

Images