Long-distance weak and small target detection and identification method for visible light and infrared dual-light images
By using visible and infrared dual-light images, combined with dual-light image registration, semantic alignment training and feature fusion, the Yolov5m network is constructed, which solves the problems of false alarm and missed detection in the single-light image detection and recognition method, and achieves high-precision detection and recognition of long-distance weak targets.
Patent Information
- Application Number
- CN202411810449.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-10
- Publication Date
- 2025-03-28
AI Technical Summary
The existing weak object detection and recognition methods are mainly based on single-light images, which are prone to false alarms and missed detection, and are poorly robust.
Using visible and infrared dual-light images, through dual-light image registration, semantic alignment training and feature fusion, the Yolov5m network is built to improve the detection and recognition ability of long-distance weak targets.
Through the complementary dual-light features, the detection and recognition accuracy of long-distance weak targets is improved, the false alarm and missed detection are reduced, and the robustness is improved.
Smart Images
Figure CN119723051A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of aircraft mission system design, and relates to a long-distance small target recognition method, and in particular to a long-distance small target detection and recognition method of visible light and infrared dual-light images. Background Art
[0002] The number of pixels of the target imaged in the optical system is inversely proportional to the distance between the two. When the internal parameters of the optical system are fixed, such as focal length, target size, etc., the farther the target is, the fewer pixels it has, making the distant target appear "small". At the same time, the light signal reflected by the target will be attenuated by the atmosphere during the propagation process, making the distant target appear "weak". Therefore, the task of long-distance target detection and recognition is essentially the task of weak target detection and recognition.
[0003] At present, the detection and recognition methods of small targets are mainly based on single-light images of visible light or infrared, and the detection and recognition of small targets are realized by constructing convolutional neural networks such as Yolo and Faster-RCNN. However, small targets usually lack features such as shape, color, contour, and texture. Using only single-light images to complete small target detection tasks is prone to false alarms, missed detections, etc., and has poor robustness. Summary of the invention
[0004] In order to solve the above problems, the present invention provides a long-distance dim small target detection and recognition method of visible light and infrared dual-light images. Considering that most complex optical systems are usually equipped with visible light and infrared sensors at the same time, compared with the single-light image detection and recognition method, the use of dual-light images can obtain richer feature information of the target in different bands, making the target area more distinguishable than the background area, thereby improving the overall accuracy of dim target detection and recognition.
[0005] The technical solution of the present invention is as follows:
[0006] A method for detecting and identifying small and dim targets at a long distance using visible light and infrared dual-light images comprises the following steps:
[0007] S1, collecting a number of pairs of bi-optical images of dim targets and registering the pairs of bi-optical images; the bi-optical images refer to visible light images and infrared images;
[0008] S2, construct the Yolov5m network, use the labeled bi-optical images to train the Yolov5m network, and obtain the initial Yolov5m network weights;
[0009] S3, extracting bi-optical image features using the constructed Yolov5m network;
[0010] S4, after extracting the features of the bi-optical image, the features of the bi-optical image are fused.
[0011] Furthermore, in S1, the bi-optical image registration method is:
[0012] During the registration process, the pixel position of the visible light image is used as the reference, and the homography matrix from the infrared image to the visible light image is obtained by matching SIFT feature points. The calculation method of the homography matrix is as follows:
[0013]
[0014] Where (x′, y′) and (x, y) are the pixel positions of the visible light feature point and the infrared feature point in the paired feature point, and h ij are the elements in the homography matrix:
[0015] Transform the above homogeneous equation into:
[0016] (h 31 x+h 32 y+h 33 )x′=h 11 x+h 12 y+h 13
[0017] (h 31 x+h 32 y+h 33 )y′=h 21 x+h 22 y+h 23
[0018] By bringing in 4 sets of feature point pairs, we can solve the element h in the homography matrix. ij , and finally obtain the mapping relationship between the infrared image pixel position and the visible light image pixel position, and realize the registration of the dual light image.
[0019] Furthermore, in S2, semantic alignment constraints are added during the training of the Yolov5m network. The semantic alignment loss function is designed using a self-supervised learning method. The input is the registered visible light image and infrared image. By minimizing the L2 distance between the feature layers, the network achieves semantic alignment of the features extracted from the bi-optical image.
[0020] Furthermore, the Yolov5m network training process is:
[0021] T r =f(θ,I r )
[0022] T e =f(θ,I e )
[0023]
[0024] f∈R H*W*C →R h*w*c
[0025] p∈R h*w*c →R N
[0026]
[0027] Where T r 、T e are infrared features and visible light features, f(*) is the inference calculation function of the backbone part of Yolov5m, θ is the backbone network parameter of Yolov5m, I r ,I e is the infrared image and the visible light image, Y r and Y e is the detection result output by the infrared and visible light models, p(*) is the inference calculation function of the Yolov5m network feature pyramid and detection head, are the network parameters of Yolov5m feature pyramid and detection head, R is the real number field, H, W, C are the width, height and number of channels of Yolov5m network input layer, h, w, c are the width, height and number of channels of Yolov5m network detection head output result;
[0028] N = h'*w'*c', h', w', c' represent the width, height and number of channels of the output result of the Yolov5m network detection head, argmin(*) is the minimum optimization problem, L s is the original loss function of Yolov5m, y r and e is the detection result output by the infrared and visible light models, λ is the regularization factor, ‖*‖ 2 Calculates the function for the second norm.
[0029] Furthermore, the feature fusion in S4 adopts the channel splicing method. By introducing the concat layer in the Yolov5m network, the splicing fusion of the deep features of the bi-optical image is realized. After the features are fused, the features will be sent to the feature pyramid layer for calculation, and the bi-optical image features of different scales will be output. Finally, the target category information and position information are obtained through the head layer regression of Yolov5m.
[0030] Furthermore, after the concat layer is introduced, the Yolov5m network is fine-tuned. During the fine-tuning, the network layer parameters before the concat layer are frozen, and a smaller learning rate is set to fine-tune the network layer parameters after the concat layer.
[0031] Furthermore, the input data used for fine-tuning comes from paired bi-optical image data, that is, the registered visible light image and infrared image collected by the same optical system at the same time, and the data has target annotation information; the fine-tuning process is:
[0032] T r =f(θ,I r )
[0033] T e =f(θ,I e )
[0034]
[0035] f∈R H*W*C →R h*w*c
[0036] p∈R h*w*c →R N
[0037]
[0038] Where T r 、T e are infrared features and visible light features, f(*) is the inference calculation function of the backbone part of Yolov5m, θ is the backbone network parameter of Yolov5m, I r ,I e is the infrared image and the visible light image, Y r and Y e is the detection result output by the infrared and visible light models, p(*) is the inference calculation function of the Yolov5m network feature pyramid and detection head, are the network parameters of Yolov5m feature pyramid and detection head, R is the real number field, H, W, C are the width, height and number of channels of Yolov5m network input layer, h, w, c are the width, height and number of channels of Yolov5m network detection head output, N = h'*w'*c', h', w', c' represent the width, height and number of channels of Yolov5m network detection head output, argmin(*) is the minimum optimization problem, L s is the original loss function of Yolov5m, y r and e is the detection result output by the infrared and visible light models, λ is the regularization factor, ‖*‖2 Calculates the function for the second norm.
[0039] Furthermore, the fine-tuned Yolov5m network model is deployed at the edge of the Yolov5m network model.
[0040] The beneficial effects of the present invention are as follows:
[0041] 1. The detection and recognition method based on dual-light images utilizes the rich color, texture features and high resolution of visible light images on the one hand, and the thermal sensitivity and strong penetration of infrared images on the other hand, so it has good robustness.
[0042] 2. The present invention detects and identifies small and dim targets at a distance based on visible light and infrared dual-light images. Through dual-light image registration, semantic alignment training, etc., the deep learning network can simultaneously use visible light and infrared images for target reasoning and identification. Compared with the small and dim target detection and identification method based on single-light images, this method overcomes the problems of missing and single features of small and dim targets in single-light images by complementing dual-light features. It can effectively improve the optical system's ability to detect and identify small and dim targets at a distance and reduce false alarms and missed detections.
[0043] 3. This method can not only be used for visible light-infrared bi-optical systems, but can also be extended to visible light-low light, medium-wave infrared-long-wave infrared and other bi-optical systems, and has good application prospects. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 This is a schematic diagram of dual-light detection and recognition network training of the present invention;
[0045] Figure 2 Schematic diagram for fine-tuning the dual-light detection and recognition network;
[0046] Figure 3 The figure is a flow chart of the algorithm implementation of the present invention. DETAILED DESCRIPTION
[0047] This section is an embodiment of the present invention, which is used to explain and illustrate the technical solution of the present invention. In the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.
[0048] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate directions or positional relationships based on the orientations or positional relationships in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or case referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implying the number of technical features indicated. Thus, features defined as "first", "second", etc. may explicitly or implicitly include more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.
[0049] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be interpreted in a broad sense, for example, it can be a fixed connection, a detachable connection or an integrated connection; it can be a mechanical connection or a point connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood by specific circumstances.
[0050] Embodiment 1:
[0051] A method for detecting and identifying small and dim targets at a long distance using visible light and infrared dual-light images comprises the following steps:
[0052] S1, collecting a number of pairs of bi-optical images of dim targets and registering the pairs of bi-optical images; the bi-optical images refer to visible light images and infrared images;
[0053] S2, construct the Yolov5m network, use the labeled bi-optical images to train the Yolov5m network, and obtain the initial Yolov5m network weights;
[0054] S3, extracting bi-optical image features using the constructed Yolov5m network;
[0055] S4, after extracting the features of the bi-optical image, the features of the bi-optical image are fused.
[0056] In S1, the bi-optical image registration method is:
[0057] During the registration process, the pixel position of the visible light image is used as the reference, and the homography matrix from the infrared image to the visible light image is obtained by matching SIFT feature points. The calculation method of the homography matrix is as follows:
[0058]
[0059] Where (x′, y′) and (x, y) are the pixel positions of the visible light feature point and the infrared feature point in the paired feature point, and h ij are the elements in the homography matrix:
[0060] Transform the above homogeneous equation into:
[0061] (h 31 x+h 32 y+h 33 )x′=h 11 x+h 12 y+h 13
[0062] (h 31 x+h 32 y+h 33 )y′=h 21 x+h 22 y+h 23
[0063] By bringing in 4 sets of feature point pairs, we can solve the element h in the homography matrix. ij , and finally obtain the mapping relationship between the infrared image pixel position and the visible light image pixel position, and realize the registration of the dual light image.
[0064] In S2, semantic alignment constraints are added during the training of the Yolov5m network. The semantic alignment loss function is designed using a self-supervised learning method. The input is the registered visible light image and infrared image. By minimizing the L2 distance between feature layers, the network achieves semantic alignment of features extracted from the bi-optical image.
[0065] The Yolov5m network training process is:
[0066] T r =f(θ,I r )
[0067] T e =f(θ,I e )
[0068]
[0069] f∈R H*W*C →R h*w*c
[0070] p∈R h*w*c →R N
[0071]
[0072] Where T r 、T e are infrared features and visible light features, f(*) is the inference calculation function of the backbone part of Yolov5m, θ is the backbone network parameter of Yolov5m, I r ,I e is the infrared image and the visible light image, Y r and Y e is the detection result output by the infrared and visible light models, p(*) is the inference calculation function of the Yolov5m network feature pyramid and detection head, are the network parameters of Yolov5m feature pyramid and detection head, R is the real number field, H, W, C are the width, height and number of channels of Yolov5m network input layer, h, w, c are the width, height and number of channels of Yolov5m network detection head output result;
[0073] N = h'*w'*c', h', w', c' represent the width, height and number of channels of the output result of the Yolov5m network detection head, argmin(*) is the minimum optimization problem, L s is the original loss function of Yolov5m, y r and e is the detection result output by the infrared and visible light models, λ is the regularization factor, ‖*‖ 2 Calculates the function for the second norm.
[0074] The feature fusion in S4 adopts the channel splicing method. By introducing the concat layer in the Yolov5m network, the splicing fusion of the deep features of the bi-optical image is realized. After the features are fused, the features will be sent to the feature pyramid layer for calculation, and the bi-optical image features of different scales will be output. Finally, the target category information and position information are obtained through Yolov5m head layer regression.
[0075] After the concat layer is introduced into the Yolov5m network, the Yolov5m network is fine-tuned. During fine-tuning, the network layer parameters before the concat layer are frozen, and a smaller learning rate is set to fine-tune the network layer parameters after the concat layer.
[0076] The input data used for fine-tuning comes from paired bi-optical image data, that is, the registered visible light image and infrared image collected by the same optical system at the same time, and the data contains target annotation information; the fine-tuning process is:
[0077] T r =f(θ,I r )
[0078] T e=f(θ,I e )
[0079]
[0080] f∈R H*W*C →R h*w*c
[0081] p∈R h*w*c →R N
[0082]
[0083] Where T r 、T e are infrared features and visible light features, f(*) is the inference calculation function of the backbone part of Yolov5m, θ is the backbone network parameter of Yolov5m, I r ,I e is the infrared image and the visible light image, Y r and Y e is the detection result output by the infrared and visible light models, p(*) is the inference calculation function of the Yolov5m network feature pyramid and detection head, are the network parameters of Yolov5m feature pyramid and detection head, R is the real number field, H, W, C are the width, height and number of channels of Yolov5m network input layer, h, w, c are the width, height and number of channels of Yolov5m network detection head output, N = h'*w'*c', h', w', c' represent the width, height and number of channels of Yolov5m network detection head output, argmin(*) is the minimum optimization problem, L s is the original loss function of Yolov5m, y r and e is the detection result output by the infrared and visible light models, λ is the regularization factor, ‖*‖ 2 Calculates the function for the second norm.
[0084] The fine-tuned Yolov5m network model is deployed at the edge of the Yolov5m network model.
[0085] Embodiment 2:
[0086] The specific implementation of the long-distance dim target detection and recognition method based on visible light and infrared dual-light images is divided into the following steps:
[0087] Step 1: Collect several sets of bi-optical images (paired visible light-infrared images) for typical distances;
[0088] Step 2: Extract SIFT features of the bi-optical image and calculate matching feature points based on the Ransic algorithm;
[0089] Step 3: Calculate the homography matrix from the infrared image to the visible light image, and calculate the optimal homography matrix by the least squares method;
[0090] Step 4: Build the Yolov5m network and train it using labeled bi-optical images to obtain the initial Yolov5m network weights;
[0091] Step 5: Insert a concat layer (before the FPN layer) into the network, freeze the parameters of the network layer before the concat layer, and fine-tune the parameters of the network layer after the concat layer based on paired bi-optical images using a small learning rate to obtain the final Yolov5m network weights;
[0092] Step 6: Deploy the network to the computing deployment platform of the optical system, collect bi-optical images in real time, infer the bi-optical images through the trained Yolov5m network, and output the location information and category information of distant dim targets.
[0093] In the above process method, the method mainly includes three parts: bi-optical image registration, bi-optical image feature extraction and target position and category regression.
[0094] (1) Bifocal Image Registration
[0095] In the structural design of the optical system, there is usually a certain installation distance (baseline distance) between the visible light sensor and the infrared sensor. At the same time, due to the differences in the target surface parameters (target surface size and pixel size) of the two sensors, the positions of each pixel in the visible light image and the infrared image are not one-to-one corresponding. If the original dual-light image is used directly for target detection and recognition, the target position will be inconsistent. Therefore, before using the dual-light image for target detection and recognition tasks, the dual-light image needs to be registered first to keep the pixel positions of the visible light and infrared images consistent.
[0096] During the registration process, the pixel position of the visible light image is used as the reference, and the homography matrix from the infrared image to the visible light image is obtained by matching SIFT feature points. The calculation method of the homography matrix is as follows:
[0097]
[0098] Where (x′, y′) and (x, y) are the pixel positions of the visible light feature point and the infrared feature point in the paired feature point, and h ij are the elements in the homography matrix.
[0099] The above homogeneous equation can be transformed into:
[0100] (h 31 x+h 32 y+h 33 )x′=h 11 x+h 12 y+h 13
[0101] (h 31 x+h 32 y+h 33 )y′=h 21 x+h 22 y+h 23
[0102] By bringing in 4 sets of feature point pairs, we can solve the element h in the homography matrix. ij , and finally obtain the mapping relationship between the infrared image pixel position and the visible light image pixel position, and realize the registration of the dual light image.
[0103] (2) Bi-optical image feature extraction
[0104] The feature extraction of bi-optical images mainly relies on deep convolutional neural networks to extract deep features of images. The basic framework of feature extraction uses the classic Yolov5m network. Considering that the bi-optical image features extracted by directly training the Yolov5m network in a supervised learning manner will have large differences and cannot be directly fused, it is necessary to add semantic alignment constraints during the training process to reduce the differences in the feature layer of bi-optical images. The specific training method is as shown in the attached Figure 1 .
[0105] The semantic alignment loss function is designed using a self-supervised learning method. The input is the registered visible light image and infrared image. By minimizing the L2 distance between feature layers, the network achieves semantic alignment of features extracted from the bi-optical image. The specific training process is as follows:
[0106] T r =f(θ,I r )
[0107] T e =f(θ,I e )
[0108]
[0109] f∈R H*W*C →R h*w*c
[0110] p∈R h*w*c →R N
[0111]
[0112] Where T r 、T e are infrared features and visible light features, f(*) is the inference calculation function of the backbone part of Yolov5m, θ is the backbone network parameter of Yolov5m, I r ,I e is the infrared image and the visible light image, Y r and Y e is the detection result output by the infrared and visible light models, p(*) is the inference calculation function of the Yolov5m network feature pyramid and detection head, are the network parameters of Yolov5m feature pyramid and detection head, R is the real number field, H, W, C are the width, height and number of channels of Yolov5m network input layer, h, w, c are the width, height and number of channels of Yolov5m network detection head output, N = h'*w'*c', h', w', c' represent the width, height and number of channels of Yolov5m network detection head output, argmin(*) is the minimum optimization problem, L s is the original loss function of Yolov5m, y r and e is the detection result output by the infrared and visible light models, λ is the regularization factor, ‖*‖ 2 Calculates the function for the second norm.
[0113] (3) Target location and category regression
[0114] After obtaining the deep features of the bi-optical image, the features of the visible light and infrared images need to be fused. The feature fusion adopts the channel splicing method. By introducing the concat layer in the network, the splicing fusion of the deep features of the bi-optical image is realized. After the features are fused, the features will be sent to the feature pyramid layer for calculation, and the bi-optical image features of different scales will be output. Finally, the target category information and location information are obtained through Yolov5m head layer regression. The fine-tuning process is as follows:
[0115] T r =f(θ,I r )
[0116] T e =f(θ,I e )
[0117]
[0118] f∈R H*W*C →R h*w*c
[0119] p∈R h*w*c →R N
[0120]
[0121] Where T r 、T e are infrared features and visible light features, f(*) is the inference calculation function of the backbone part of Yolov5m, θ is the backbone network parameter of Yolov5m, I r ,I e is the infrared image and the visible light image, Y r and Y e is the detection result output by the infrared and visible light models, p(*) is the inference calculation function of the Yolov5m network feature pyramid and detection head, are the network parameters of Yolov5m feature pyramid and detection head, R is the real number field, H, W, C are the width, height and number of channels of Yolov5m network input layer, h, w, c are the width, height and number of channels of Yolov5m network detection head output, N = h'*w'*c', h', w', c' represent the width, height and number of channels of Yolov5m network detection head output, argmin(*) is the minimum optimization problem, L s is the original loss function of Yolov5m, y r and e is the detection result output by the infrared and visible light models, λ is the regularization factor, ‖*‖ 2 Calculates the function for the second norm.
[0122] It should be noted that after the concat layer is introduced, the network needs to be fine-tuned considering the change in the number of feature channels. Considering that the network layer before concat has been trained for semantic alignment, the parameters of the network layer before the concat layer will be frozen during fine-tuning, and a smaller learning rate (due to the small number of samples) will be set to fine-tune the parameters of the network layer after the concat layer. The input data used for fine-tuning comes from paired visible light-infrared data (registered visible light and infrared images collected by the same optical system at the same time), and the data has target annotation information (position information and category information).
[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the field can easily think of various equivalent modifications or substitutions within the technical scope disclosed by the present invention, and these modifications or substitutions should all be covered within the protection scope of the present invention.
Claims
1. A method for detecting and identifying small and dim targets at long distances using visible light and infrared dual-light images, characterized in that: The following steps are involved: S1, collecting a number of pairs of bi-optical images of dim targets and registering the pairs of bi-optical images; the bi-optical images refer to visible light images and infrared images; S2, construct the Yolov5m network, use the labeled bi-optical images to train the Yolov5m network, and obtain the initial Yolov5m network weights; S3, extracting bi-optical image features using the constructed Yolov5m network; S4, after extracting the features of the bi-optical image, the features of the bi-optical image are fused.
2. The method for detecting and identifying small and dim targets at a distance using visible light and infrared dual light images according to claim 1, characterized in that: In S1, the bi-optical image registration method is: During the registration process, the pixel position of the visible light image is used as the reference, and the homography matrix from the infrared image to the visible light image is obtained by matching SIFT feature points. The calculation method of the homography matrix is as follows: Where (x′y′) and (x, y) are the pixel positions of the visible light feature point and the infrared feature point in the paired feature point, h ij are the elements in the homography matrix: Transform the above homogeneous equation into: (h 31 x+h 32 y+h 33 )x′=h 11 x+h 12 y+h 13 (h 31 x+h 32 y+h 33 )y′=h 21 x+h 22 y+h 23 By bringing in 4 sets of feature point pairs, we can solve the element h in the homography matrix. ij , and finally obtain the mapping relationship between the infrared image pixel position and the visible light image pixel position, and realize the registration of the dual light image.
3. The method for detecting and identifying small and dim targets at a distance using visible light and infrared dual light images according to claim 1, characterized in that: In S2, semantic alignment constraints are added during the training of the Yolov5m network. The semantic alignment loss function is designed using a self-supervised learning method. The input is the registered visible light image and infrared image. By minimizing the L2 distance between feature layers, the network achieves semantic alignment of features extracted from the bi-optical image.
4. The method for detecting and identifying small and dim targets at a distance using visible light and infrared dual light images according to claim 3, characterized in that: The Yolov5m network training process is: T r =f(θ,I r ) T e =f(θ,I e ) Where T r , T e are infrared features and visible light features, f(*) is the inference calculation function of the backbone part of Yolov5m, θ is the backbone network parameter of Yolov5m, I r ,I e is the infrared image and the visible light image, Y r and Y e is the detection result output by the infrared and visible light models, p(*) is the inference calculation function of the Yolov5m network feature pyramid and detection head, are the network parameters of Yolov5m feature pyramid and detection head, R is the real number field, H, W, C are the width, height and number of channels of Yolov5m network input layer, h, w, c are the width, height and number of channels of Yolov5m network detection head output result; N = h'*w'*c', h', w', c' represent the width, height and number of channels of the output result of the Yolov5m network detection head, argmin(*) is the minimum optimization problem, L s is the original loss function of Yolov5m, y r and e is the detection result output by the infrared and visible light models, λ is the regularization factor, and ||*||2 is the two-norm calculation function.
5. The method for detecting and identifying small and dim targets at a distance using visible light and infrared dual light images according to claim 1, characterized in that: The feature fusion in S4 adopts the channel splicing method. By introducing the concat layer in the Yolov5m network, the splicing fusion of the deep features of the bi-optical image is realized. After the features are fused, the features will be sent to the feature pyramid layer for calculation, and the bi-optical image features of different scales will be output. Finally, the target category information and position information are obtained through Yolov5m head layer regression.
6. The method for detecting and identifying small and dim targets at a distance using visible light and infrared dual light images according to claim 5, characterized in that: After the concat layer is introduced into the Yolov5m network, the Yolov5m network is fine-tuned. During fine-tuning, the network layer parameters before the concat layer are frozen, and a smaller learning rate is set to fine-tune the network layer parameters after the concat layer.
7. The method for detecting and identifying small and dim targets at a distance using visible light and infrared dual light images according to claim 6, characterized in that: The input data used for fine-tuning comes from paired bi-optical image data, that is, the registered visible light image and infrared image collected by the same optical system at the same time, and the data contains target annotation information; The fine-tuning process is: T r =f(θ,I r ) T e =f(θ,I e ) Where T r , T e are infrared features and visible light features, f(*) is the inference calculation function of the backbone part of Yolov5m, θ is the backbone network parameter of Yolov5m, I r ,I e is the infrared image and the visible light image, Y r and Y e is the detection result output by the infrared and visible light models, p(*) is the inference calculation function of the Yolov5m network feature pyramid and detection head, are the network parameters of Yolov5m feature pyramid and detection head, R is the real number field, H, W, C are the width, height and number of channels of Yolov5m network input layer, h, w, c are the width, height and number of channels of Yolov5m network detection head output, N = h'*w'*c', h', w', c' represent the width, height and number of channels of Yolov5m network detection head output, argmin(*) is the minimum optimization problem, L s is the original loss function of Yolov5m, y r and e is the detection result output by the infrared and visible light models, λ is the regularization factor, and ||*||2 is the two-norm calculation function.
8. The method for detecting and identifying small and dim targets at long distances using visible light and infrared dual light images according to claim 7, characterized in that: The fine-tuned Yolov5m network model is deployed at the edge of the Yolov5m network model.