Radar target identification method based on radar and photoelectric cross-modal knowledge distillation

CN119024327AInactive Publication Date: 2024-11-26INST OF SOFTWARE - CHINESE ACAD OF SCI

Patent Information

Application Number
CN202411071590.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure CN119024327A_ABST
    Figure CN119024327A_ABST
Patent Text Reader

Abstract

The invention discloses a radar target identification method based on radar and photoelectric cross-modal knowledge distillation. The system comprises a radar image acquisition device, a holder photoelectric device and a data processor. The data processor comprises a radar photoelectric linkage module, a photoelectric target identification module, a radar photoelectric cross-modal knowledge distillation module and a radar target identification module. The radar image acquisition device and the holder photoelectric device are connected with the radar photoelectric linkage module in the data processor. The radar photoelectric linkage module in the data processor is connected with the photoelectric target identification module in the data processor, and the photoelectric target identification module in the data processor is connected with the radar photoelectric cross-modal knowledge distillation module in the data processor. The radar photoelectric cross-modal knowledge distillation module in the data processor is connected with the radar target identification module in the data processor. According to the invention, the target can be quickly and accurately identified, and the intelligent monitoring and response capability is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the cross-technical field of intelligent monitoring technology and computer multimedia, and relates to a multi-modal sensor-oriented identification method for monitoring marine targets. Background Art

[0002] Radar is widely used in civil and military monitoring systems such as navigation, aviation, vehicle-mounted, and security due to its all-day and all-weather observation capabilities. Radar target recognition is one of the core requirements of radar monitoring. However, compared with other widely used monitoring systems, the data quality of radar images is relatively low, and the image resolution and signal-to-noise ratio are slightly insufficient, which leads to limited accuracy in target recognition.

[0003] A direct solution to improve radar target recognition accuracy is to use high-quality data in optoelectronic images to introduce key information required for accurate recognition, which falls into the category of computer multimedia and cross-modal learning. Cross-modal learning aims to model the relationship between different data modalities to utilize multimodal information to improve performance.

[0004] Knowledge transfer plays a vital role in cross-modal learning and is usually accomplished through knowledge distillation. Traditional knowledge distillation methods adopt the idea of ​​imitating output values ​​and develop improved methods based on this to achieve better transfer of knowledge with large amounts of information and unexplainable information. However, there is a lack of analysis on the effectiveness and interpretability of the transferred knowledge, which limits the further development of the performance of such methods. Summary of the invention

[0005] In view of the technical problems existing in the prior art, the purpose of the present invention is to provide a radar target recognition method based on radar and optoelectronic cross-modal knowledge distillation. The present invention adopts radar and optoelectronic target recognition based on deep learning, and a cross-modal knowledge distillation method. Through the knowledge distillation of radar and optoelectronic data, the target data detected by the two sensors are associated and fused, which can greatly increase the credibility of radar target automatic recognition.

[0006] The present invention studies and discovers the correlation between the discriminability of features and their dimensional structure, thereby providing a method for expressing the feature dimensional structure using deep feature channel correlation and intermediate feature spatial distribution, and on this basis proposes a cross-modal knowledge distillation method with outstanding effectiveness and interpretability, which can learn semantically unrelated features from radar modal data to improve its target recognition accuracy.

[0007] In order to solve the problem that the method of radar and optoelectronic cross-modal knowledge distillation lacks effectiveness and interpretability, the radar target recognition system based on radar and optoelectronic cross-modal knowledge distillation of the present invention includes a radar image acquisition device, a pan-tilt optoelectronic device and a data processor; the data processor includes a radar optoelectronic linkage module, an optoelectronic target recognition module, a radar optoelectronic cross-modal knowledge distillation module and a radar target recognition module, and the radar image acquisition device and the pan-tilt optoelectronic device are connected to the radar optoelectronic linkage module in the data processor;

[0008] The radar optoelectronic linkage module in the data processor is connected to the optoelectronic target recognition module in the data processor, the optoelectronic target recognition module in the data processor is connected to the radar optoelectronic cross-modal knowledge distillation module in the data processor, and the radar optoelectronic cross-modal knowledge distillation module in the data processor is connected to the radar target recognition module in the data processor.

[0009] Preferably, the radar image acquisition device is a radar signal acquisition card.

[0010] Preferably, the pan-tilt optoelectronic device is a network camera.

[0011] Preferably, the data processor is a computer or an embedded motherboard.

[0012] Preferably, the radar optoelectronic linkage module pairs the radar and optoelectronic image data of the same target according to the relative position relationship between the radar target position, the radar image acquisition device and the pan-tilt optoelectronic device.

[0013] Preferably, the optoelectronic target recognition module adopts an optoelectronic target recognition method based on deep learning to identify targets in optoelectronic images.

[0014] Preferably, the radar optoelectronic cross-modal knowledge distillation module adopts a cross-modal knowledge distillation method based on deep learning to fuse and distill the knowledge contained in the target data under radar and optoelectronic modes.

[0015] Preferably, the radar target recognition module adopts a radar target recognition method based on deep learning to identify targets in radar images.

[0016] The technical solution of the present invention is:

[0017] A radar target recognition method based on radar and optoelectronic cross-modal knowledge distillation, the steps of which include:

[0018] 1) The radar image acquisition device 1 detects the target area, obtains the radar image of the target object and sends it to the data processor 3, and at the same time, the PTZ optoelectronic device 2 detects the target area, obtains the optoelectronic image of the target object and sends it to the data processor 3; the data processor 3 includes a radar optoelectronic linkage module 4, an optoelectronic target recognition module 5, a radar optoelectronic cross-modal knowledge distillation module 6 and a radar target recognition module 7;

[0019] 2) The radar optoelectronic linkage module 4 obtains the radar target position from the radar image, and then pairs the radar image and optoelectronic image data of the same target object according to the relative position relationship between the radar target position, the radar image acquisition device 1 and the PTZ optoelectronic device, to obtain the image data of the same target object in both radar and optoelectronic modes;

[0020] 3) The optoelectronic target recognition module 5 uses the target recognition model to identify the type of target object in the optoelectronic image, annotates the optoelectronic image and the paired radar image, and obtains a data set in, is the i-th training sample in the data set X, is the radar image containing the i-th target object, is the photoelectric image containing the i-th target object, y i is the label; i = 1~N, N is the total number of samples;

[0021] 4) The radar optoelectronic cross-modal knowledge distillation module 6 adopts a cross-modal knowledge distillation algorithm to fuse and distill the knowledge contained in the target data under the radar and optoelectronic modes based on the data set X, transfers the target knowledge in the optoelectronic mode to the target recognition model of the radar mode and saves it to the radar target recognition module 7; the cross-modal knowledge distillation algorithm is: first define the dimensional structure of the feature, and then optimize the dimensional structure of the feature The target knowledge in the optoelectronic mode is transferred to the target recognition model in the radar mode; b is the number of samples in the learning batch, σ is the parameter of the radial basis kernel function, for The features of the network map head After mapping, we get the i-th dimension of the embedded representation. for The features of the network map head After mapping, we get the i-th dimension of the embedded representation. for The characteristics of After mapping, we get the jth dimension of the embedded representation. λ is a parameter that weighs the importance of the two sub-items in the loss function. The target data x i go through The intermediate features obtained by mapping, The target data x j After feature extraction network The intermediate features obtained by mapping, The target data x j After feature extraction network The intermediate features obtained by mapping; x i is the radar data or photoelectric image data of the ith target, x j is the radar data or photoelectric image data of the jth target, and The structure is the same but the learnable parameters are different; and The structure is the same but the learnable parameters are different; and Feature extraction networks with the same structure but different learnable parameters;

[0022] 5) For a radar image P to be identified, the radar target identification module 7 identifies the type of target in the radar image P using a target identification model of the radar modality.

[0023] Furthermore, the cross-modal knowledge distillation algorithm is a cross-modal knowledge distillation method based on deep learning.

[0024] Furthermore, the radar image acquisition device 1 is a radar signal acquisition card.

[0025] Furthermore, the pan / tilt optoelectronic device 2 is a network camera.

[0026] Furthermore, the data processor 3 may be a computer or an embedded motherboard.

[0027] Furthermore, the target recognition model is a YOLOv7 pre-trained model.

[0028] A radar target recognition system based on radar and optoelectronic cross-modal knowledge distillation, characterized in that it includes a radar image acquisition device 1, a pan-tilt optoelectronic device 2 and a data processor 3; the data processor 3 includes a radar optoelectronic linkage module 4, an optoelectronic target recognition module 5, a radar optoelectronic cross-modal knowledge distillation module 6 and a radar target recognition module 7;

[0029] The radar image acquisition device 1 is used to detect the target area, obtain the radar image of the target object and send it to the data processor 3;

[0030] The PTZ optoelectronic device 2 is used to detect the target area, obtain the optoelectronic image of the target object and send it to the data processor 3;

[0031] The radar optoelectronic linkage module 4 is used to obtain the radar target position from the radar image, and then pair the radar image and optoelectronic image data of the same target object according to the radar target position and the position of the radar image acquisition device 1 to obtain the image data of the same target object in both radar and optoelectronic modes;

[0032] The optoelectronic target recognition module 5 is used to use the target recognition model to identify the type of the target object in the optoelectronic image, annotate the optoelectronic image and the radar image paired with it, and obtain a data set. in, is the i-th training sample in the data set X, is the radar image containing the i-th target object, is the photoelectric image containing the i-th target object, y i is the label; i = 1~N, N is the total number of samples;

[0033] The radar optoelectronic cross-modal knowledge distillation module 6 is used to use a cross-modal knowledge distillation algorithm to fuse and distill the knowledge contained in the target data under the radar and optoelectronic modes based on the data set X, transfer the target knowledge in the optoelectronic mode to the target recognition model of the radar mode and save it to the radar target recognition module 7; the cross-modal knowledge distillation algorithm is: first define the dimensional structure of the feature, and then optimize the dimensional structure of the feature

[0034] The target knowledge in the optoelectronic mode is transferred to the target recognition model in the radar mode; b is the number of samples in the learning batch, σ is the parameter of the radial basis kernel function, for The features of the network map head After mapping, we get the i-th dimension of the embedded representation. for The features of the network map head After mapping, we get the i-th dimension of the embedded representation. for The characteristics of After mapping, we get the jth dimension of the embedded representation. λ is a parameter that weighs the importance of the two sub-items in the loss function. The target data x i go through The intermediate features obtained by mapping, The target data xj After feature extraction network The intermediate features obtained by mapping, The target data x j After feature extraction network The intermediate features obtained by mapping; x i is the radar data or photoelectric image data of the ith target, x j is the radar data or photoelectric image data of the jth target, and The structure is the same but the learnable parameters are different; and The structure is the same but the learnable parameters are different;

[0035] The radar target recognition module 7 is used to recognize the type of target in the radar image P to be recognized by using the target recognition model of the radar modality.

[0036] The beneficial effects of the above technical solution of the present invention are as follows:

[0037] In the above scheme, the radar image acquisition device can process the radar signal in real time to generate radar images, and the radar target recognition module adopts a target recognition method based on deep learning, which can quickly and accurately identify the target. Compared with the radar target recognition method, it introduces effective information in the photoelectric image data, and uses the channel correlation of deep features and the spatial distribution of intermediate features to express the dimensional structure of the features. By optimizing the dimensional structure to obtain higher accuracy, it has high robustness in different scenarios. Cross-modal knowledge distillation of target data detected by radar and photoelectric sensors can greatly increase the accuracy of radar target recognition and improve the intelligent monitoring and response capabilities of the system. At the same time, the data processor can be an embedded motherboard that can be built on a variety of intelligent monitoring devices, improving the portability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a block diagram of the radar target recognition method based on radar and optoelectronic cross-modal knowledge distillation of the present invention.

[0039] Figure numerals: 1-radar image acquisition equipment, 2-pan-tilt optoelectronic equipment, 3-data processor, 4-radar optoelectronic linkage module, 5-optoelectronic target recognition module, 6-radar optoelectronic cross-modal knowledge distillation module, 7-radar target recognition module. DETAILED DESCRIPTION

[0040] See also Figure 1, an embodiment of the present invention provides a radar target recognition method based on radar and optoelectronic cross-modal knowledge distillation, the radar target recognition method based on radar and optoelectronic cross-modal knowledge distillation: a radar image acquisition device 1, a pan-tilt optoelectronic device 2 and a data processor 3; the data processor 3 includes: a radar optoelectronic linkage module 4, an optoelectronic target recognition module 5, a radar optoelectronic cross-modal knowledge distillation module 6 and a radar target recognition module 7, the radar optoelectronic linkage module 4 is connected to the optoelectronic target recognition module 5, the optoelectronic target recognition module 5 is connected to the radar optoelectronic cross-modal knowledge distillation module 6, and the radar optoelectronic cross-modal knowledge distillation module 6 is connected to the radar target recognition module 7; the radar image acquisition device 1 is connected to the radar optoelectronic linkage module 4 in the data processor 3, and the pan-tilt optoelectronic device 2 is connected to the radar optoelectronic linkage module 4 in the data processor 3.

[0041] The radar image acquisition device 1 may be a radar or other image acquisition devices.

[0042] The pan / tilt optoelectronic device 2 may be a camera or other image acquisition devices.

[0043] In the above embodiment, the data processor 3 may be a computer or an embedded motherboard.

[0044] In the above embodiment, the radar optoelectronic linkage module 4 can pair the radar and optoelectronic image data of the same target according to the radar target position and the radar optoelectronic device position.

[0045] In the above embodiment, the optoelectronic target recognition module 5 may use a deep learning method to identify the types of marine targets.

[0046] In the above embodiment, the radar optoelectronic cross-modal knowledge distillation module 6 may use a deep learning method to fuse and distill the knowledge contained in the target data under the radar and optoelectronic modes.

[0047] In the above embodiment, the radar target recognition module 7 can use a deep learning method to identify the types of marine targets.

[0048] The specific working process of the embodiment of the present invention is as follows:

[0049] First, the radar image acquisition device detects the target area to obtain the original radar echo, obtains the target radar image through imaging and inputs it into the data processor 3. At the same time, the pan-tilt optoelectronic device 2 detects the target area to obtain the optoelectronic image of the target and inputs it into the data processor 3.

[0050] Secondly, the radar optoelectronic linkage module 4 in the data processor 3 obtains the radar target position from the target radar image, and then pairs the radar image and optoelectronic image data of the same target according to the radar target position and the position of the radar image acquisition device 1, and obtains the image data of the same target in both radar and optoelectronic modes. For example, firstly, the specific angle and distance of the target relative to the radar image acquisition device are calculated according to the position of the target in the radar image, and then based on this, combined with the relative position relationship between the radar image acquisition device and the optoelectronic image acquisition device, the specific angle and distance of the target relative to the optoelectronic image acquisition device can be calculated, so that the optoelectronic image acquisition device can be adjusted to acquire the optoelectronic image of the target, which also realizes the pairing of the radar image and optoelectronic image data of the same target.

[0051] Then, the optoelectronic target recognition module 5 uses the trained target recognition model (such as the YOLOv7 pre-trained model) to identify the target types in the optoelectronic image, including cargo ships, fishing boats, sailboats, etc.; the optoelectronic image and the paired radar image are annotated according to the target recognition results to obtain a data set

[0052] Next, the radar optoelectronic cross-modal knowledge distillation module 6 uses a cross-modal knowledge distillation algorithm to define the dimensional structure of the features starting from the channel correlation of the deep features and the spatial distribution of the intermediate features.

[0053] The data obtained is in and is the radar and optoelectronic image data of the i-th target, y i Label it. and It is used to extract features from radar and electro-optical images. and They were used to and The output feature map is mapped to a high-dimensional latent space, and the feature is and Get the embedded image after mapping and The cross-correlation matrix between radar and optoelectronic embedding is C, which is used to define the optimization target L1 of the feature dimension structure at the deep feature level, which is used to enhance the discriminability of radar features to targets. and They are two independent network mapping heads with the same structure but different learnable parameters.

[0054] b is the number of samples in the learning batch; for The characteristics of After mapping, we get the i-th dimension of the embedded representation. for The characteristics of After mapping, we get the i-th dimension of the embedded representation. for The characteristics of After mapping, the jth dimension of the embedded representation is obtained;

[0055] λ is a parameter that weighs the importance of the two sub-items in the loss function;

[0056] Target data x i and x j go through and The intermediate features are obtained by mapping and Then the autocorrelation and cross-correlation distances between the intermediate features are obtained, which are used to define the spatial distribution of the intermediate features, thereby obtaining the optimization target L2, x of the feature dimension structure at the intermediate feature level. i is the radar data or optoelectronic image data of the i-th target. and They are two independent feature extraction networks with the same structure but different learnable parameters.

[0057] σ is the parameter of the radial basis kernel function; The target data x j go through The intermediate features obtained by mapping.

[0058]

[0059] The corresponding model optimization algorithm is used to optimize L1+L2 and transfer the target knowledge in the optoelectronic modality to the target recognition model of the radar modality, prompting it to learn semantically irrelevant features from the radar modality.

[0060] Finally, the radar target recognition module 7 in the data processor 3 uses the target recognition model that has been trained and optimized after knowledge distillation to identify the type of target in the radar image.

[0061] A radar target recognition method based on radar and optoelectronic cross-modal knowledge distillation in an embodiment of the present invention can process images acquired by radar and optoelectronic image acquisition equipment in real time. The target recognition module uses a deep learning method to quickly and accurately identify targets at sea. The radar optoelectronic cross-modal knowledge distillation module 6 introduces effective information in optoelectronic image data, and uses the channel correlation of deep features and the spatial distribution of intermediate features to express the dimensional structure of features. By optimizing the dimensional structure to obtain a higher accuracy rate, it has high robustness in different scenarios. Cross-modal knowledge distillation of target data detected by radar and optoelectronic sensors can greatly increase the accuracy of radar target recognition and improve the intelligent monitoring and response capabilities of the system.

[0062] In order to illustrate the purpose of the present invention, the above embodiments are only for description and not for limiting the scope of the present invention. The scope of the present invention is determined by the claims, and any equivalent substitutions and modifications that do not deviate from the principles and core features of the embodiments of the present invention should be included in the scope of the present invention.

[0063] It is obvious to those skilled in the art that the specific forms of the embodiments of the present invention are not limited to the details in the above exemplary embodiments. Therefore, these embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is limited by the claims rather than the above description. Any figure mark should not be regarded as limiting the claims. In addition, the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units, modules or devices can be implemented by the same unit, module or device through software or hardware.

[0064] Finally, although the above preferred embodiments are described in detail, they are only for describing the technical solutions of the embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art should understand that the technical solutions may be modified or replaced by equivalents, but should not depart from the principles and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A radar target recognition method based on radar and optoelectronic cross-modal knowledge distillation, the steps of which include: 1) A radar image acquisition device (1) detects a target area, obtains a radar image of the target object and sends it to a data processor (3); at the same time, a pan / tilt optoelectronic device (2) detects the target area, obtains an optoelectronic image of the target object and sends it to the data processor (3); the data processor (3) includes a radar optoelectronic linkage module (4), an optoelectronic target recognition module (5), a radar optoelectronic cross-modal knowledge distillation module (6) and a radar target recognition module (7); 2) the radar optoelectronic linkage module (4) obtains the radar target position from the radar image, and then pairs the radar image and optoelectronic image data of the same target object according to the relative position relationship between the radar target position, the radar image acquisition device (1) and the pan-tilt optoelectronic device, to obtain image data of the same target object in both radar and optoelectronic modes; 3) The optoelectronic target recognition module (5) uses the target recognition model to identify the type of the target object in the optoelectronic image, and annotates the optoelectronic image and the radar image paired with it to obtain a data set. in, is the i-th training sample in the data set X, is the radar image containing the i-th target object, is the photoelectric image containing the i-th target object, y i is the label; i = 1~N, N is the total number of samples; 4) The radar optoelectronic cross-modal knowledge distillation module (6) adopts a cross-modal knowledge distillation algorithm to fuse and distill the knowledge contained in the target data under the radar and optoelectronic modes based on the data set X, transfers the target knowledge in the optoelectronic mode to the target recognition model of the radar mode and saves it to the radar target recognition module (7); the cross-modal knowledge distillation algorithm is: first define the dimensional structure of the feature, and then optimize the dimensional structure of the feature The target knowledge in the optoelectronic mode is transferred to the target recognition model in the radar mode; b is the number of samples in the learning batch, σ is the parameter of the radial basis kernel function, for The features of the network map head After mapping, we get the i-th dimension of the embedded representation. for The characteristics of After mapping, we get the i-th dimension of the embedded representation. for The features of the network map head After mapping, we get the jth dimension of the embedded representation. λ is a parameter that weighs the importance of the two sub-items in the loss function. The target data x i go through The intermediate features obtained by mapping, The target data x j After feature extraction network The intermediate features obtained by mapping, The target data x j After feature extraction network The intermediate features obtained by mapping; x i is the radar data or photoelectric image data of the ith target, x j is the radar data or photoelectric image data of the jth target, and The structure is the same but the learnable parameters are different; and The structure is the same but the learnable parameters are different; 5) For a radar image P to be identified, the radar target identification module (7) uses a target identification model of the radar modality to identify the type of target in the radar image P.

2. The method according to claim 1, characterized in that The cross-modal knowledge distillation algorithm is a cross-modal knowledge distillation method based on deep learning.

3. The method according to claim 1, characterized in that The radar image acquisition device (1) is a radar signal acquisition card.

4. The method according to claim 1, characterized in that: The pan / tilt optoelectronic device (2) is a network camera.

5. The method according to claim 1, characterized in that The data processor (3) may be a computer or an embedded motherboard.

6. The method according to claim 1, characterized in that The target recognition model is a YOLOv7 pre-trained model.

7. A radar target recognition system based on radar and optoelectronic cross-modal knowledge distillation, characterized in that: It comprises a radar image acquisition device (1), a pan-tilt optoelectronic device (2) and a data processor (3); the data processor (3) comprises a radar optoelectronic linkage module (4), an optoelectronic target recognition module (5), a radar optoelectronic cross-modal knowledge distillation module (6) and a radar target recognition module (7); The radar image acquisition device (1) is used to detect the target area, obtain the radar image of the target object and send it to the data processor (3); The pan / tilt optoelectronic device (2) is used to detect the target area, obtain an optoelectronic image of the target object and send it to the data processor (3); The radar optoelectronic linkage module (4) is used to obtain the radar target position from the radar image, and then pair the radar image and optoelectronic image data of the same target object according to the radar target position and the position of the radar image acquisition device (1), so as to obtain image data of the same target object in both radar and optoelectronic modes; The optoelectronic target recognition module (5) is used to use the target recognition model to identify the type of the target object in the optoelectronic image, annotate the optoelectronic image and the radar image paired with it, and obtain a data set in, is the i-th training sample in the data set X, is the radar image containing the i-th target object, is the photoelectric image containing the i-th target object, y i is the label; i = 1~N, N is the total number of samples; The radar optoelectronic cross-modal knowledge distillation module (6) is used to use a cross-modal knowledge distillation algorithm to fuse and distill the knowledge contained in the target data under the radar and optoelectronic modes based on the data set X, transfer the target knowledge in the optoelectronic mode to the target recognition model of the radar mode and save it to the radar target recognition module (7); the cross-modal knowledge distillation algorithm is: first define the dimensional structure of the feature, and then optimize the dimensional structure of the feature. The target knowledge in the optoelectronic mode is transferred to the target recognition model in the radar mode; b is the number of samples in the learning batch, σ is the parameter of the radial basis kernel function, for The features of the network map head After mapping, we get the i-th dimension of the embedded representation. for The features of the network map head After mapping, we get the i-th dimension of the embedded representation. for The characteristics of After mapping, we get the jth dimension of the embedded representation. λ is a parameter that weighs the importance of the two sub-items in the loss function. The target data x i go through The intermediate features obtained by mapping, The target data x j After feature extraction network The intermediate features obtained by mapping, The target data x j After feature extraction network The intermediate features obtained by mapping; x i is the radar data or photoelectric image data of the ith target, x j is the radar data or photoelectric image data of the jth target, and The structure is the same but the learnable parameters are different; and The structure is the same but the learnable parameters are different; and Feature extraction networks with the same structure but different learnable parameters; The radar target recognition module (7) is used to recognize the type of target in the radar image P to be recognized using a target recognition model of the radar modality.

8. The system according to claim 7, characterized in that The cross-modal knowledge distillation algorithm is a cross-modal knowledge distillation method based on deep learning.

9. The system according to claim 7, characterized in that The radar image acquisition device (1) is a radar signal acquisition card, the pan / tilt optoelectronic device (2) is a network camera, and the data processor (3) can be a computer or an embedded motherboard.

10. The system according to claim 7, characterized in that The target recognition model is a YOLOv7 pre-trained model.

Citation Information

Patent Citations

  • 3D target detection system and method based on 4D millimeter wave radar and camera fusion

    CN117452396A

  • Cross-domain human behavior recognition method based on knowledge distillation

    CN117711064A

Cited By

  • Image semantic segmentation method and system based on cross-modal hierarchical knowledge distillation

    CN121564354A

  • An image semantic segmentation method and system based on cross-modal hierarchical knowledge distillation

    CN121564354B