Personnel target detection method in infrared environment based on improved yov3-tinier network

Abstract

The invention discloses a personnel target detection method in an infrared environment based on an improved yolov3-tinier network. The method comprises the following steps: 1, preprocessing a data set; 2, an improved yolov3-tinier network is established; 3, constructing a loss function Loss, training and improving a yolov3-tinier network to obtain a personnel target detection model in the optimal infrared environment, and deploying the personnel target detection model on a ZYNQ hardware system for detecting personnel targets in the infrared environment; the invention aims to deploy an improved model which is smaller in size and higher in recognition speed on a ZYNQ hardware system so as to achieve the purpose of quickly and effectively detecting a personnel target in an infrared environment under the condition that the precision is not lost.

Description

Technical Field

[0001] The invention belongs to the field of deep learning and target detection, and more specifically to a method for detecting human targets in an infrared environment based on an improved YOLOv3-Tinier network. Background Art

[0002] In infrared environments, due to the loss of effective target areas, it is very challenging to perform personnel target detection on visible light images with limited quality. This makes personnel target detection in infrared environments more difficult, resulting in false detections, missed detections, and even misjudgments of targets of rescued personnel. Therefore, there is an urgent need for a system that can quickly detect personnel targets in infrared environments.

[0003] Personnel target detection methods are widely used under visible light conditions, but the performance of existing personnel target detection methods in infrared environments is very limited. In addition, the existing personnel target detection network is very large, with too many calculation parameters and the required calculation carrier is also very large, which cannot be transplanted to hardware for processing. As a result, the accuracy of personnel target detection is reduced and the detection speed is too slow in infrared environments, which limits the real-time and portability of personnel target detection in infrared environments. Summary of the invention

[0004] The present invention aims to solve the shortcomings of the above-mentioned prior art and proposes a method for detecting human targets in an infrared environment based on an improved yolov3-tinier network, in order to quickly and effectively detect human targets in an infrared environment by deploying an optimized model with faster recognition speed and better recognition performance on the ZYNQ side, thereby improving the speed of human target detection in an infrared environment without losing accuracy.

[0005] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme:

[0006] The present invention is characterized in that a method for detecting human targets in an infrared environment based on an improved yolov3-tinier network comprises the following steps:

[0007] Step 1: Get the infrared environment personnel detection dataset Image, and let any infrared personnel image in Image be recorded as ,Will The positions and categories of the K infrared personnel in the image are marked to obtain the position labels and category labels. The location tag of the kth infrared person in is recorded as ,and ,in, Indicates Infrared personnel in The coordinates of the center point of the rectangular box at the position in , and =( ), represents the horizontal coordinate of the center point of the rectangular frame, represents the ordinate of the center point of the rectangular frame, Indicates Infrared personnel in The size of the rectangular box at the location in , and = , Indicates Infrared personnel in The width of the rectangular box at the position in Indicates Personnel in The height of the rectangular box at the position in The category label of the kth infrared person in is recorded as ;

[0008] Step 2: Establish an improved YOLOv3-Tinier network, which includes: backbone module, feature extraction module, and detection module;

[0009] Step 2.1, the backbone modules are alternately indivual Unit and The maximum pooling layer is composed of indivual Unit by convolutional layers and The activation function layers are alternately composed and Processing to obtain infrared activation features ;

[0010] Step 2.2, the feature extraction module is composed of a first infrared feature extraction layer and a second infrared feature extraction layer connected in parallel, and Processing is performed to obtain the final infrared activation features of the first layer and the final infrared activated features of the second layer ;

[0011] Step 2.3, the detection module consists of two CR units, two decoding layers and a non-maximum suppression layer, and and Process and obtain Infrared suppression characteristics ;

[0012] Step 3: Based on , construct the loss function ;

[0013] Step 4: Based on Image, use the gradient descent method to train the improved yolov3-tinier network and calculate the loss function To adjust the network parameters, when the number of training iterations reaches the set number or the loss function When convergence occurs, the training stops, and the optimal human target detection model in the infrared environment is obtained. The model is deployed on the ZYNQ development board for detecting human targets in the infrared environment.

[0014] The method for detecting human targets in an infrared environment based on an improved yolov3-tinier network described in the present invention is also characterized in that step 2.1 includes the following steps:

[0015] When m=1,c=1, Input into the backbone module and pass through the mth Unit The mth convolutional layer is processed to obtain The cth infrared convolution feature output by the unit ;Will Enter the mth Unit activation function layer, and use formula (1) to get the mth The cth infrared activation feature output by the unit ;

[0016] (1)

[0017] When m=1,c=2,3,…,C, the c-1th infrared activation feature After the mth Unit The convolutional layers and The activation function layer is processed, so that the mth Unit The activation function layer outputs infrared activation features ;

[0018] When m=1, c=C, Input into the mth maximum pooling layer for processing to obtain the mth maximum pooling infrared feature ;

[0019] When m=2,3,…,M, the m-1th maximum pooled infrared feature Enter the mth The Mth The unit outputs the final infrared activation signature .

[0020] Further, step 2.2 includes the following steps:

[0021] Step 2.2.1, the first infrared feature extraction layer consists of a maximum pooling layer, a feature fusion layer and F CR units, and Processing is performed to obtain the final infrared activation features of the first layer ;

[0022] Step 2.2.1.1: The maximum pooling layer in the first infrared feature extraction layer Processing is performed to obtain the maximum pooled infrared eigenvalue ;

[0023] Step 2.2.1.2: After input into the first CR unit for convolution and activation operations, the first infrared activation feature of the first layer is obtained. ;Will After input into the second CR unit for convolution and activation operations, the second infrared activation feature of the first layer is obtained. ;

[0024] Step 2.2.1.3: The feature fusion layer in the first infrared feature extraction layer will and After splicing in the channel number dimension, the first infrared fusion feature is obtained ;

[0025] Step 2.2.1.4: The convolution and activation operations are performed in the remaining CR units in turn to obtain the final infrared activation features of the first layer. ;

[0026] Step 2.2.2: The second infrared feature extraction layer consists of a feature fusion layer and D CR units. Processing is performed to obtain the final infrared activation features of the second layer ;

[0027] Step 2.2.2.1: The first CR unit pair in the second infrared characteristic extraction layer After convolution and activation operations, the first infrared activation feature of the second layer is obtained ;

[0028] Step 2.2.2.2. After inputting the second CR unit in the second infrared feature extraction layer for convolution and activation operations, the second infrared activation feature of the second layer is obtained. ;

[0029] Step 2.2.2.3: The feature fusion layer in the second infrared feature extraction layer will and Splicing is performed in the dimension of channel number to obtain the second infrared fusion feature ;

[0030] Step 2.2.2.4, After the convolution and activation operations of the remaining CR units in the second infrared feature extraction layer, the final infrared activation feature of the second layer is obtained. .

[0031] Further, step 2.3 includes the following steps:

[0032] Step 2.3.1. and Input into two CR units for processing respectively, and the first infrared activation feature is obtained accordingly and a second infrared activated feature ;

[0033] Step 2.3.2: and Input into two decoding layers for processing respectively, and the first infrared decoding feature is obtained accordingly and the second infrared decode feature ;

[0034] Step 2.3.3: and Input into the non-maximum suppression layer for processing, and get Infrared suppression characteristics ,in, represent The predicted rectangular box of the kth infrared person's location in ; where The horizontal coordinate of the center point of the predicted rectangular box representing the location of the kth infrared person in I, The vertical coordinate of the center point of the predicted rectangular box representing the location of the kth infrared person in I, Represents the width of the predicted rectangular box of the kth infrared person's location in I, Represents the height of the predicted rectangular box of the kth infrared person's location in I; represent The prediction confidence of the kth infrared personnel in ; where, represent The predicted category of the kth infrared person in .

[0035] Furthermore, in step 3, the loss function is constructed using formula (2): :

[0036] (2)

[0037] In formula (2), for The center point loss of the rectangular box where the kth infrared person is located, for The width and height loss of the rectangular box where the kth infrared person is located, for The confidence loss of the kth infrared person in, for The class loss of the kth infrared person in .

[0038] An electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the person target detection method, and the processor is configured to execute the program stored in the memory.

[0039] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the person target detection method when the computer program is executed by a processor.

[0040] Compared with the prior art, the beneficial effects of the present invention are:

[0041] 1. The present invention uses human target detection images in infrared environments as network data sets. Infrared imaging highlights human targets through temperature field imaging on the surface of objects. It is not limited by lighting conditions and is less affected by the environment. It has a strong ability to penetrate fog, rain, snow and other weather conditions. It can overcome the phenomenon that it is difficult to distinguish objects under low light conditions in visible light images, resulting in missed detection of human targets, and plays a role in supplementing information.

[0042] 2. The improved YOLOv3-Tinier network of the present invention includes a backbone module, a feature extraction module, and a detection module. The three modules include convolution, pooling, etc., which overcome the defects of the original YOLOv3 network, such as large size, large number of parameters, slow calculation speed, and large calculation carrier. While maintaining the detection accuracy, the number of parameters can be reduced, the calculation speed can be accelerated, and the calculation process can be performed on a smaller carrier ZYNQ end, thereby improving the real-time and portability of personnel target detection in infrared environments, and personnel target detection can be performed in a wider range of infrared environments, thereby improving recognition efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of the method for detecting human targets in an infrared environment of an improved yolov3-tinier network in the present invention;

[0044] Figure 2This is the improved yolov3-tinier network structure diagram in the present invention.

[0045] Figure 3 This is the architecture diagram of the personnel target detection system in the infrared environment of the improved yolov3-tinier network in the present invention. DETAILED DESCRIPTION

[0046] In this embodiment, a method for detecting human targets in an infrared environment based on an improved yolov3-tinier network is described. Figure 1 , including the following steps:

[0047] Step 1: Get the infrared environment personnel detection dataset Image, and let any infrared personnel image in Image be recorded as ,Will The positions and categories of the K infrared personnel in the image are marked to obtain the position labels and category labels. The location tag of the kth infrared person in is recorded as ,and ,in, Indicates Infrared personnel in The coordinates of the center point of the rectangular box at the position in , and =( ), Represents the horizontal coordinate of the center point of the rectangular box. Represents the vertical coordinate of the center point of the rectangular box. Indicates Infrared personnel in The size of the rectangular box at the location in , and = , Indicates Infrared personnel in The width of the rectangular box at the position in Indicates Personnel in The height of the rectangular box at the position in The category label of the kth infrared person in is recorded as In this embodiment, the processing is performed on the LLVIP low-light vision infrared dataset. The input images of the model will be uniformly resized by bilinear interpolation, and the final resized value is set to 416×416.

[0048] Step 2: Establish an improved yolov3-tinier network, such as Figure 2 As shown, it includes: a backbone module, a feature extraction module, and a detection module;

[0049] Step 2.1: The backbone modules are replaced by indivual Unit and The maximum pooling layer is composed of indivual Unit by convolutional layers and activation function layers are alternately composed; in this embodiment, Take 5, Taking 1, the backbone module can ensure a certain feature acquisition capability while making the inference speed faster, the model size smaller, the memory usage smaller, and the risk of overfitting lower.

[0050] When m=1,c=1, Input into the backbone module and pass through the mth Unit The mth convolutional layer is processed to obtain The cth infrared convolution feature of the unit ; Enter the mth Unit activation function layer, and use formula (1) to get the mth The cth infrared activation feature of the cell ;

[0051]

[0052] When m=1,c=2,3,…,C, the c-1th infrared activation feature After the mth Unit The convolutional layers and The activation function layer is processed, so that the mth Unit The activation function layer outputs infrared activation features In this embodiment, the first indivual The convolutional layer of the unit is composed of filters, each filter size is 3×3, each filter has a total of 2× convolution kernels, and the stride and padding value of each convolution kernel is 1.

[0053] When m=1, c=C, Input into the mth maximum pooling layer for processing to obtain the mth maximum pooling infrared feature ; In this embodiment, the size of any pooling layer in the backbone module is 2×2, and the stride is 2;

[0054] When m=2,3,…,M, the m-1th maximum pooled infrared feature Enter the mth The Mth The unit outputs the final infrared activation signature .

[0055] Step 2.2, the feature extraction module is composed of a first infrared feature extraction layer and a second infrared feature extraction layer connected in parallel;

[0056] Step 2.2.1, the first infrared feature extraction layer consists of a maximum pooling layer, a feature fusion layer and F CR units, and Processing is performed to obtain the final infrared activation features of the first layer ; In this embodiment, the value of F is 3;

[0057] Step 2.2.1.1: The maximum pooling layer in the first infrared feature extraction layer Processing is performed to obtain the maximum pooled infrared eigenvalue In this embodiment, infrared features are extracted through maximum pooling operation, thereby enhancing the generalization ability of the model and retaining the most significant infrared maximum pooling features.

[0058] Step 2.2.1.2: After input into the first CR unit for convolution and activation operations, the first infrared activation feature of the first layer is obtained. ;Will After input into the second CR unit for convolution and activation operations, the second infrared activation feature of the first layer is obtained. ; In this embodiment, while performing infrared feature extraction, dimension reduction and information concentration can be performed to construct a hierarchical infrared feature representation;

[0059] Step 2.2.1.3: The feature fusion layer in the first infrared feature extraction layer will and After splicing in the channel number dimension, the first infrared fusion feature is obtained ; In this embodiment, feature fusion aims to capture richer contextual infrared information and multi-scale infrared details.

[0060] Step 2.2.1.4: The convolution and activation operations are performed in the remaining CR units in turn to obtain the final infrared activation features of the first layer. In this embodiment, the feature fusion operation is followed by convolution and activation operations to further integrate the infrared fusion feature Key infrared features are extracted while retaining important structural information.

[0061] Step 2.2.2: The second infrared feature extraction layer consists of a feature fusion layer and D CR units. Processing is performed to obtain the final infrared activation features of the second layer ; In this embodiment, the value of D is 3;

[0062] Step 2.2.2.1: The first CR unit pair in the second infrared characteristic extraction layer After convolution and activation operations, the first infrared activation feature of the second layer is obtained ;

[0063] Step 2.2.2.2. After inputting the second CR unit in the second infrared feature extraction layer for convolution and activation operations, the second infrared activation feature of the second layer is obtained. .

[0064] Step 2.2.2.3: The feature fusion layer in the second infrared feature extraction layer will and Splicing is performed in the dimension of channel number to obtain the second infrared fusion feature ;

[0065] Step 2.2.2.4, After the convolution and activation operations of the remaining CR units in the second infrared feature extraction layer, the final infrared activation feature of the second layer is obtained. .

[0066] Step 2.3, the detection module consists of two CR units, two decoding layers and a non-maximum suppression layer; in this embodiment, each filter of the CR unit of the detection module contains 255×256 convolution kernels, the size of each convolution kernel is 1×1, the stride value is 1, and the padding value is 0;

[0067] Step 2.3.1. and Input into two CR units for processing respectively, and the first infrared activation feature is obtained accordingly and a second infrared activated feature ;

[0068] Step 2.3.2: and Input into two decoding layers for processing respectively, and the first infrared decoding feature is obtained accordingly and the second infrared decode feature .

[0069] Step 2.3.3: and Input into the non-maximum suppression layer for processing, and get Infrared suppression characteristics ,in, represent The predicted rectangular box of the kth infrared person's location in ; where The horizontal coordinate of the center point of the predicted rectangular box representing the location of the kth infrared person in I, The vertical coordinate of the center point of the predicted rectangular box representing the location of the kth infrared person in I, Represents the width of the predicted rectangular box of the kth infrared person's location in I, Represents the height of the predicted rectangular box of the kth infrared person's location in I; represent The prediction confidence of the kth infrared personnel in ; where, represent The predicted category of the kth infrared person in .

[0070] Step 3: Use formula (2) to construct the loss function :

[0071]

[0072] In formula (2) for The center point loss of the rectangular box where the kth infrared person is located, for The width and height loss of the rectangular box where the kth infrared person is located, for The confidence loss of the kth infrared person in, for The class loss of the kth infrared person in .

[0073] In this embodiment, equations (3), (4), (5) and (6) are used to construct , , and :

[0074] (3)

[0075] (4)

[0076] (5)

[0077] (6)

[0078] In formula (3), It is a coordination function used to coordinate the inconsistent contributions of rectangular boxes of different sizes to the error function; Indicates whether the predicted rectangular box of the k-th infrared personnel position in I successfully predicts the k-th infrared personnel target. If the k-th infrared personnel target is successfully predicted, let =1, otherwise, let =0;

[0079] In formula (5), Indicates that the predicted rectangular box at the location of the kth infrared person in I contains the true value of the infrared person; is a weight value, which indicates the weight of the confidence error in the loss function when the predicted rectangle does not predict the infrared personnel target; If the predicted rectangle of the kth infrared person in I fails to predict the kth infrared person, then =1, otherwise, let =0;

[0080] Step 4: Based on Image, use the gradient descent method to train the improved yolov3-tinier network and calculate the loss function To adjust the network parameters, when the number of training iterations reaches the set number or the loss function When convergence occurs, the training stops, thereby obtaining the optimal human target detection model in the infrared environment, and deploying it on the ZYNQ development board for detecting human targets in the infrared environment. In this embodiment, the number of training rounds is set to 1000, and the trained model is obtained after 300 epochs of training iterations.

[0081] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0082] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium, and the computer program executes the steps of the above method when executed by a processor.

[0083] In this embodiment, a method for detecting human targets in an infrared environment based on an improved yolov3-tinier network is based on a hardware system as shown in FIG. Figure 3, including: infrared camera, ZYNQ development board, HDMI data stream transmission cable, HDMI display. Among them, the infrared camera is mounted on the ZYNQ development board, takes images of human targets in infrared environment, and stores them in the DDR on the PS side for detection. The trained improved yolov3-tinier network is loaded in the ZYNQ development board to detect the infrared image data in the DDR. The HDMI display is connected to the ZYNQ development board through the HDMI data stream transmission cable to display the human target detection results.

Claims

1. A method for detecting human targets in infrared environment based on improved yolov3-tinier network, characterized in that: The steps include: Step 1: Get the infrared environment personnel detection dataset Image, and let any infrared personnel image in Image be recorded as I, mark the positions and categories of K infrared personnel in I, obtain the position label and category label, and record the position label of the kth infrared personnel in I as ,and ,in, represents the coordinates of the center point of the rectangular box where the kth infrared person is located in I, and , represents the horizontal coordinate of the center point of the rectangular frame, represents the ordinate of the center point of the rectangular frame, represents the size of the rectangular box where the kth infrared person is located in I, and , The width of the rectangular box representing the position of the kth infrared person in I, The height of the rectangular box representing the position of the kth person in I; the category label of the kth infrared person in I is recorded as ; Step 2: Establish an improved YOLOv3-Tinier network, which includes: backbone module, feature extraction module, and detection module; Step 2.1, the backbone module is composed of alternating M CR units and M-1 maximum pooling layers, wherein any mth CR unit is composed of C convolutional layers and C activation function layers alternately, and I is processed to obtain the infrared activation feature ; Step 2.2, the feature extraction module is composed of a first infrared feature extraction layer and a second infrared feature extraction layer connected in parallel, and Processing is performed to obtain the final infrared activation features of the first layer and the final infrared activated features of the second layer ; Step 2.3, the detection module consists of two CR units, two decoding layers and a non-maximum suppression layer, and and After processing, the infrared suppression characteristics of I are obtained ; Step 3: Based on , construct the loss function Loss; Step 4: Based on Image, the improved YOLOv3-tinier network is trained using the gradient descent method, and the loss function Loss is calculated to adjust the network parameters. When the number of training iterations reaches the set number or the loss function Loss converges, the training stops, thereby obtaining the optimal personnel target detection model in the infrared environment, and deploying it on the ZYNQ development board for detecting personnel targets in the infrared environment.

2. According to claim 1, a method for detecting human targets in an infrared environment based on an improved yolov3-tinier network is characterized in that: Step 2.1 includes the following steps: When m=1, c=1, I is input into the backbone module and processed by the cth convolution layer of the mth CR unit to obtain the cth infrared convolution feature output by the mth CR unit. ;Will Then input it into the cth activation function layer of the mth CR unit, and use formula (1) to obtain the cth infrared activation feature output by the mth CR unit: ; (1) When m=1, c=2,3,…,C, the c-1th infrared activation feature After being processed by the cth convolution layer and the cth activation function layer of the mth CR unit, the Cth activation function layer of the mth CR unit outputs the infrared activation feature ; When m=1, c=C, Input into the mth maximum pooling layer for processing to obtain the mth maximum pooling infrared feature ; When m=2,3,…,M, the m-1th maximum pooled infrared feature The input is processed in the mth CR unit, and the final infrared activation feature is output by the Mth CR unit. .

3. A method for detecting human targets in an infrared environment based on an improved yolov3-tinier network according to claim 2, characterized in that: Step 2.2 includes the following steps: Step 2.2.1, the first infrared feature extraction layer consists of a maximum pooling layer, a feature fusion layer and F CR units, and Processing is performed to obtain the final infrared activation features of the first layer ; Step 2.2.1.1: The maximum pooling layer in the first infrared feature extraction layer Processing is performed to obtain the maximum pooled infrared eigenvalue ; Step 2.2.1.2: After input into the first CR unit for convolution and activation operations, the first infrared activation feature of the first layer is obtained. ;Will After input into the second CR unit for convolution and activation operations, the second infrared activation feature of the first layer is obtained. ; Step 2.2.1.3: The feature fusion layer in the first infrared feature extraction layer will and After splicing in the channel number dimension, the first infrared fusion feature is obtained ; Step 2.2.1.4: The convolution and activation operations are performed in the remaining CR units in turn to obtain the final infrared activation features of the first layer. ; Step 2.2.2: The second infrared feature extraction layer consists of a feature fusion layer and D CR units. Processing is performed to obtain the final infrared activation features of the second layer ; Step 2.2.2.1: The first CR unit pair in the second infrared characteristic extraction layer After convolution and activation operations, the first infrared activation feature of the second layer is obtained ; Step 2.2.2.

2. After inputting the second CR unit in the second infrared feature extraction layer for convolution and activation operations, the second infrared activation feature of the second layer is obtained. ; Step 2.2.2.3: The feature fusion layer in the second infrared feature extraction layer will and Splicing is performed in the dimension of channel number to obtain the second infrared fusion feature ; Step 2.2.2.4, After the convolution and activation operations of the remaining CR units in the second infrared feature extraction layer, the final infrared activation feature of the second layer is obtained. .

4. The method for detecting human targets in an infrared environment based on an improved yolov3-tinier network according to claim 3 is characterized in that: Step 2.3 includes the following steps: Step 2.3.

1. and Input into two CR units for processing respectively, and the first infrared activation feature is obtained accordingly and a second infrared activated feature ; Step 2.3.2: and Input into two decoding layers for processing respectively, and the first infrared decoding feature is obtained accordingly and the second infrared decode feature ; Step 2.3.3: and Input into the non-maximum suppression layer for processing to obtain the infrared suppression feature of I ,in, Represents the predicted rectangular box of the kth infrared person's location in I; where, The horizontal coordinate of the center point of the predicted rectangular box representing the location of the kth infrared person in I, The vertical coordinate of the center point of the predicted rectangular box representing the location of the kth infrared person in I, Represents the width of the predicted rectangular box of the kth infrared person's location in I, Represents the height of the predicted rectangular box of the kth infrared person's location in I; represents the prediction confidence of the kth infrared personnel in I; where, Represents the predicted category of the kth infrared person in I.

5. The method for detecting human targets in an infrared environment based on an improved yolov3-tinier network according to claim 4 is characterized in that: In step 3, the loss function Loss is constructed using formula (2): (2) In formula (2), is the center point loss of the rectangular box where the kth infrared person is located in I, is the width and height loss of the rectangular box where the kth infrared person is located in I, is the confidence loss of the kth infrared person in I, is the category loss of the kth infrared person in I.

6. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the person target detection method described in any one of claims 1 to 5, and the processor is configured to execute the program stored in the memory.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for detecting a person target in any one of claims 1 to 5 are executed.

Images