Target detection and identification method in low-illumination environment

The pseudo-good illumination image is generated through histogram equalization and Wiener filtering processing, and the information is extracted using the style transformation network, which solves the problem of object detection in low-illumination environments, improves the recognition accuracy and real-timeness, and achieves a wider application range .

CN118941779AInactive Publication Date: 2024-11-12CHANGCHUN UNIV OF SCI & TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411176414.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-11-12
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the case of poor lighting conditions in the prior art, it is difficult to identify the target area in the image, resulting in insufficient recognition accuracy and real-time performance.

Method used

Histogram equalization and Wiener filtering are used to preprocess the dark light images to generate pseudo-good illumination images, and texture and semantic information are extracted through the style transformation network, and input to the target detection network for detection and identification.

Benefits of technology

It improves the accuracy and real-timeness of object detection and recognition in low-illumination environments, reduces redundant areas of interest, reduces the positive and negative sample gap, stabilizes the network convergence process, fast calculation speed, and does not affect hyperparameters.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941779A_ABST
    Figure CN118941779A_ABST
Patent Text Reader

Abstract

The invention provides a target detection and identification method in a low-illumination environment, and the method comprises the following steps: S1, carrying out the histogram equalization processing of a dark light image, and obtaining a first image; s2, performing Wiener filtering on the first image to obtain a second image; s3, constructing a style transformation network; inputting the second image into a style transformation network to generate a pseudo good illumination image; s4, inputting the pseudo good illumination image into a target detection network for detection and identification; and S5, the classification layer and the regression layer in the target detection network output final detection and identification results. The problem that in the prior art, a target area in an image cannot be recognized under the condition that the illumination condition is not good enough is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a method for detecting and identifying a target in a low-light environment. Background Art

[0002] Target detection and recognition have always been one of the research focuses and hot topics in the field of computer vision. It has increasingly demonstrated its important application value in the fields of civil transportation, security, medicine, and military monitoring, navigation, etc. The accuracy of target detection and recognition technology directly determines the reliability of applications such as automatic monitoring and traffic identification, and its real-time nature determines the scope of application of downstream tasks.

[0003] At present, the target detection and recognition technology based on deep learning methods has achieved or even exceeded the human level in automatic recognition and monitoring performance when the lighting and shooting environment are good. However, under poor lighting conditions such as cloudy days and nights, the exposure rate of ordinary cameras is insufficient, and the obtained images have low signal-to-noise ratio, low contrast, and strong noise, making it difficult to identify areas of interest, which poses new challenges to current target detection and recognition methods.

[0004] Therefore, there is an urgent need for a simple and effective target detection and recognition method to improve the recognition accuracy under poor lighting conditions. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a method for target detection and recognition in a low-light environment to solve the problem that the prior art cannot recognize the target area in the image under poor lighting conditions, and can improve the recognition accuracy and real-time performance, while having a wider range of applications.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is a method for detecting and identifying a target in a low-light environment, comprising the following steps:

[0007] S1: performing histogram equalization processing on the dark light image to obtain a first image;

[0008] S2: Perform Wiener filtering on the first image to obtain a second image;

[0009] S3: construct a style transformation network; input the second image into the style transformation network to generate a pseudo well-illuminated image;

[0010] S4: Input the pseudo good illumination image into the target detection network for detection and recognition;

[0011] S5: The classification layer and regression layer in the target detection network output the final detection and recognition results.

[0012] Furthermore, the S1 includes:

[0013] The calculation formula for histogram equalization is as follows:

[0014]

[0015] Among them, s k is the pixel value obtained after histogram equalization processing, T(r k ) represents the gray level r in the dark light image k The pixels are transformed by histogram, L represents the total number of gray levels in the image, L-1 is the maximum gray level; j represents the gray level, k represents the gray level index, n j p represents the total number of pixels with gray level j, where gray level j is an integer value between 0 and 254, 0≤k≤L-1; r (r j ) represents the continuous probability density of the dark-light image in the grayscale continuous form, M and N represent the height and width of the dark-light image respectively, and MN represents the total number of pixels in the dark-light image.

[0016] Further, the S2 includes:

[0017] S21: Use Gaussian noise as the noise signal n(t) to simulate the noise conditions encountered in actual shooting;

[0018] S22: transform the first image obtained in S1 from the spatial domain to the frequency domain using a two-dimensional discrete Fourier transform DFT;

[0019] S23: Determine the degradation function. The calculation formula of the degradation function is as follows:

[0020] H(u,v)=exp(-0.0025×((uM / 2) 2 +(vN / 2) 2 ) 56 )

[0021] Wherein, u and v represent the horizontal and vertical coordinate values ​​of the first image in the frequency domain, respectively, and M and N represent the height and width of the first image, respectively;

[0022] S24: Perform a Wiener filter operation on the image converted in S22; the calculation formula of the Wiener filter operation is as follows:

[0023]

[0024] Where, f is the input, i.e., the horizontal and vertical coordinate values ​​u and v of the first image in the frequency domain; G(f) represents the image obtained by performing Wiener filtering on the first image, i.e., the second image; H(f) represents the degradation function, i.e., H(u,v); |H(f)| refers to the modulus of the degradation function; S(f) is the power spectrum of the first image x(t); N(f) is the power spectrum of the noise signal n(t); SNR(f) is the signal-to-noise ratio;

[0025] S25: converting the second image back to the spatial domain through two-dimensional inverse discrete Fourier transform IDFT and then saving it.

[0026] Further, the S3 includes:

[0027] S31: Construct a style transformation network; use a simplified VGG19 network as the style transformation network. The style transformation network consists of five convolution blocks. The first four convolution blocks are composed of two convolution layers and a maximum pooling layer connected in sequence. The fifth convolution block is composed of two convolution layers connected, where the output of the second convolution layer is used as the final feature output of the style transformation network. The activation function of each convolution block is relu, the convolution kernel size of each convolution layer is 3×3, the convolution step size is 1, and the number of channels of the five convolution blocks is 64, 128, 256, 512, and 512 respectively.

[0028] S32: Input the well-illuminated image into the style transformation network, and use the output of the fourth convolutional block as the style image feature map S;

[0029] S33: The second image obtained in S25 is input into the style transformation network of S31, and the output of the second convolution layer of the fifth convolution block is to generate the image feature map G.

[0030] Furthermore, the S3 also includes:

[0031] S34: Use the Gram matrix to calculate the correlation between different feature channels in the generated image feature map and the style image feature map, and define the style loss function;

[0032] The Gram matrix calculation formula is:

[0033]

[0034] Where, for a given convolutional layer l, l is the second convolutional layer of the fifth convolutional block; It represents the inner product of the c-th dimension channel feature and the d-th dimension channel feature in the second convolution layer of the fifth convolution block. I and J represent the height and width of the generated image feature map, respectively. i and j represent the position coordinates of the pixels in the generated image feature map, respectively. represents the feature activation value at position ij in the c-th dimension channel in the second convolution layer of the fifth convolution block, Represents the feature activation value at position ij in the d-th dimension channel in the second convolution layer of the fifth convolution block;

[0035] Use the Gram matrix to calculate the Gram matrix of the generated image feature map and style image feature map, respectively, G l (G), G l (S);

[0036] The calculation formula of style loss function L(S,G) is as follows:

[0037]

[0038] Among them, M l is the total number of elements in the generated image feature map, and is the height H of the generated image feature map. l , Width W l The product of H l ×W l ;

[0039] N l is the number of channels used to generate the image feature map; l = L = 1, ||·|| F represents the Frobenius norm, that is, the matrix G l (S)-G l The square root of the sum of the squares of the elements of (G);

[0040] S35: Iterative process; using the gradient descent algorithm to update the pixel values ​​of the generated image feature map G to minimize the style loss function, iterating continuously until a predetermined number of iterations is reached; obtaining a pseudo well-illuminated image.

[0041] Further, the S4 includes:

[0042] The target detection network includes a backbone network, a multi-scale network, and prediction and classification modules;

[0043] S41: The backbone network includes convolutional layers B3, B4, and B5, and the multi-scale network includes low-level layers P3, P4, base layer P5, and high-level layers P6 and P7;

[0044] In the backbone network, the pseudo good illumination image obtained by S35 is input into the B3 layer to obtain a first feature map with 8 channels and a length × width of 100 × 128. The first feature map is input into the B4 layer to obtain a second feature map with 16 channels and a length × width of 50 × 64. The second feature map is input into the B5 layer to obtain a third feature map with 32 channels and a length × width of 25 × 32.

[0045] S42: In the multi-scale network, copy the third feature map to obtain a P5 layer feature map, input the P5 layer feature map to the P6 layer to obtain a fourth feature map with 64 channels and a length and width of 13×16, input the fourth feature map to the P7 layer to obtain a fifth feature map with 128 channels and a length and width of 7×8; upsample the P5 layer feature map to obtain a sixth feature map with 16 channels and a length and width of 50×64, add the sixth feature map to the second feature map of the B4 layer to obtain the P4 layer feature map, upsample the P4 layer feature map to obtain a seventh feature map with 8 channels and a length and width of 100×128, and add the seventh feature map to the first feature map of the B3 layer to obtain the P3 layer feature map;

[0046] S43: The multi-level features extracted by the multi-scale network are, from top to bottom: the fifth feature map, the fourth feature map, the P5 layer feature map, the P4 layer feature map, and the P3 layer feature map. The five feature maps are respectively input into five prediction and classification modules. Each prediction and classification module consists of four convolutional layers and two parallel output layers. The convolution kernels of the four convolutional layers are all 3×3, the step size is 1, the number of channels is 256, and the two parallel output layers are the classification layer and the regression layer.

[0047] Furthermore, the output process of the classification layer and the regression layer in S43 is:

[0048] S431: The classification layer outputs each target category in the pseudo good illumination image in the form of a mask;

[0049] S432: The regression layer uses each pixel point in the input feature map as the center point of the region of interest, predicts the length and width of each region of interest, and uses it as a candidate bounding box. The output of the regression layer is a four-dimensional vector, which represents the two-dimensional coordinates of the length, width, and upper left corner of the candidate bounding box;

[0050] S433: All candidate bounding boxes whose center points are within the true bounding box are taken as positive samples.

[0051] Furthermore, the S4 further includes:

[0052] S44: Add the center point detection loss in the regression layer output process. The center point detection loss calculation formula is as follows:

[0053]

[0054] Among them, centerness represents the center point detection loss, l, r, t, and b are the distances from the center point of each candidate bounding box output by the regression layer to the upper edge, lower edge, left edge, and right edge respectively;

[0055] The overall loss of the target detection network consists of three parts: classification loss, regression loss, and center point detection loss. The specific calculation formula is as follows:

[0056]

[0057] Among them, L cls is the classification loss, L reg is the regression loss, N pos is the total number of regions of interest, p x,y , are the true category and predicted category output by the classification layer, respectively, x,y , are the true position and predicted position output by the regression layer, respectively, L(p x,y ,t x,y ) represents the overall loss of the target detection network, x and y represent the horizontal and vertical coordinates of each center point, respectively. represents the classification loss of the true category and the predicted category, λ represents the balance coefficient, Regression loss representing the true position and predicted position;

[0058] S45: training the prediction and classification module; first collect and label the data of the center point, redundant candidate bounding box, and true bounding box, then use the training set to train the prediction and classification module, and update the model parameters of the prediction and classification module to minimize the overall loss function L(p x,y ,t x,y ), and iterates continuously until the predetermined number of iterations is reached.

[0059] Further, the S5 includes:

[0060] Calculate the confidence of all positive samples, obtain the classification score of each positive sample from the classification layer, set the confidence threshold, sort all positive samples that pass the threshold screening by their confidence from high to low, select the positive sample with the highest confidence, verify and confirm the screened positive sample with the highest confidence, and use non-maximum suppression to remove overlapping bounding boxes;

[0061] The final output result is: the category output of the classification layer and the four-dimensional position vector of the positive sample with the highest confidence output by the regression layer.

[0062] The beneficial effects of the present invention are:

[0063] 1. The present invention uses a combination of spatial domain and frequency to perform histogram equalization and Wiener filtering on low-illumination images, which can quickly remove noise in the image and enhance the area of ​​interest in the image with better robustness.

[0064] 2. The present invention uses a style transfer network to effectively extract texture and semantic information from dark-light images, and then generates pseudo dark-light images. The network has low complexity, fast convergence speed and good transferability.

[0065] 3. The present invention regresses the region of interest, extracts image features and performs multi-task learning, effectively reducing redundant regions of interest, reducing the gap between positive and negative samples, improving the regression accuracy and network training speed, making the network convergence process stable and real-time. Compared with traditional detection methods, the performance of the method of the present invention is not affected by hyperparameters and has a faster calculation speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0067] Figure 1 It is a flow chart of the present invention.

[0068] Figure 2 It is the process architecture diagram of the target detection network.

[0069] Figure 3 It is the recognition result of the low-illuminance image using the existing image enhancement method and the method of the present invention, wherein a is the low-illuminance image, b is the recognition result of the low-illuminance image using the existing image enhancement method, and c is the recognition result of the low-illuminance image using the method of the present invention.

[0070] Figure 4 It is the recognition result of the low illumination image using the existing noise reduction method and the method of the present invention, wherein a is the low illumination image, b is the recognition result of the low illumination image using the existing noise reduction method, and c is the recognition result of the low illumination image using the method of the present invention. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] See also Figure 1 The complete steps of the method of the present invention are as follows:

[0073] S1: Perform histogram equalization on the dark image to obtain a first image. Histogram equalization refers to equalizing the grayscale level r in the input image. k The pixels of are mapped to the gray level s in the first image x(t). kThe pixels of the processed image are expanded from a single-peak distribution to a uniform distribution. The calculation formula for histogram equalization is as follows:

[0074]

[0075] Among them, s k is the pixel value obtained after the transformation operation, T(r k ) indicates that the gray level of the input image is r k The pixels are transformed by histogram, L represents the total number of gray levels in the image, L-1 is the maximum gray level, usually 255; j represents the gray level, k represents the gray level index, n j Indicates the total number of pixels with gray level j, where gray level j is an integer value between 0 and 254. For example, when k = 10, Indicates the number of pixels with gray levels ranging from 0 to 10, 0≤k≤L-1; p r (r j ) represents the continuous probability density of the input image in grayscale continuous form, M and N represent the height and width of the input image respectively, and MN represents the total number of pixels in the input image.

[0076] S2: Perform Wiener filtering on the first image to remove noise in the first image and obtain a noise-reduced second image. The specific steps are as follows:

[0077] S21: The input image is the first image. In this embodiment, the dark light image contains Gaussian noise with a noise parameter mean of 0 and a variance of 0.005 as the noise signal n(t) to simulate the noise conditions that may be encountered in the actual shooting or transmission process.

[0078] S22: transform the first image from the spatial domain to the frequency domain using a two-dimensional discrete Fourier transform (DFT).

[0079] S23: Determine a degradation function to reflect the degradation caused by the shooting or transmission process. The calculation formula of the degradation function is as follows:

[0080] H(u,v)=exp(-0.0025×((uM / 2) 2 +(vN / 2) 2 ) 5 / 6 )

[0081] Among them, u and v represent the horizontal and vertical coordinate values ​​of the first image in the frequency domain, respectively, and M and N represent the height and width of the input image, respectively. The captured image always has interference signals, namely noise, due to hardware equipment problems. The degradation function H(u,v) is used to compensate for the signal interference caused by the hardware equipment. If the degradation function is not used, it is believed that the captured image is exactly the same as the real scene, which conflicts with the actual situation.

[0082] S24: Perform a Wiener filter operation on the image in S22. The calculation formula of the Wiener filter operation is as follows:

[0083]

[0084] Among them, f is the input signal, which means that the input is the horizontal and vertical coordinate values ​​u and v of the first image in the frequency domain, G(f) represents the image obtained by Wiener filtering the first image, that is, the second image; H(f) represents the degradation function, that is, H(u,v), |H(f)| refers to the modulus of the degradation function, S(f) is the power spectrum of the first image x(t); N(f) is the power spectrum of the noise signal n(t) with a mean of 0 and a variance of 0.005, and SNR(f) is the given signal-to-noise ratio.

[0085] S25: convert the second image back to the spatial domain through a two-dimensional inverse discrete Fourier transform (IDFT) and save it, thereby realizing preprocessing in the spatial domain and the frequency domain.

[0086] S3: Input the second image into the style transformation network to generate a pseudo well-lit image. The style transformation network refers to a pre-trained VGG19 network. First, the features in the second image are extracted using the pre-trained VGG19 network, and then the features are filtered using the style transformation formula to obtain a converted pseudo well-lit image. The style transformation method of the present invention replaces the last convolution layer in the fifth convolution block with a style transfer layer, and the output is a well-lit image after the style is transformed. The specific steps are described as follows:

[0087] S31: The standard VGG19 consists of five convolutional blocks and a total of 19 parameterized convolutional layers. The present invention simplifies the VGG19 network, which consists of five convolutional blocks. The first four convolutional blocks are composed of two convolutional layers and a maximum pooling layer connected in sequence; the fifth convolutional block is composed of two convolutional layers connected, where the output of the second convolutional layer is used as the final feature output.

[0088] The activation function of each convolution block is relu, the convolution kernel size of each convolution layer is 3×3, the convolution step size is 1, and the number of channels of the five convolution blocks are 64, 128, 256, 512, and 512 respectively.

[0089] Since the grayscale histogram of the dark light image is unimodal, the low-level features and high-level features in the dark light image are blurred. The shallow neural network mainly extracts low-level features of the image, such as edges, textures, etc., and the deep neural network extracts high-level features of the image, such as semantic information. The present invention uses the first four convolutional blocks of the VGG19 network with SOTA performance to extract low-level features in the second image, such as edges, textures, etc.

[0090] S32: The style image is a well-lit image, and the style features are extracted. The first four convolution blocks of VGG19 are used to process the well-lit image, and its features are extracted as the target style image features. The output features of the first four convolution blocks of VGG19 are mainly considered, especially the output of the fourth convolution block. The output features of these convolution blocks are considered to be able to better capture the local texture and style information of the image, which contains high-order image statistical features. The output of the fourth convolution block is a style feature map.

[0091] S33: The second image obtained in S25 is input into the style transformation network of S31, and the output of the last convolution layer of the fifth convolution block is the generated image.

[0092] S34: The present invention uses the last convolution layer of the fifth convolution block of the VGG19 network as a style transfer layer to bring the style statistical features of the dark light image closer to the well-illuminated image. Compared with other generation methods, this method can preserve the shape, texture and other information of the target to be measured, that is, the input dark light image, to the greatest extent. First, the Gram matrix is ​​used to calculate the correlation between different feature channels in the generated image and the style image. The Gram matrix calculation formula is:

[0093]

[0094] Among them, for a given convolutional layer l, which is the last convolutional layer of the fifth convolutional block, It represents the inner product of the c-th dimension channel feature and the d-th dimension channel feature in the l-th layer, reflecting the correlation between the c-th dimension feature and the d-th dimension feature. I and J represent the height and width of the input feature map of the style image and the generated image, respectively. i and j represent the position coordinates of the pixels in the feature map, respectively. represents the feature activation value at position ij in the c-th dimension channel in the l-th layer, Represents the feature activation value at position ij in the d-dimensional channel in the l-th layer.

[0095] Use the Gram matrix to calculate the Gram matrix of the generated image and the style image, respectively, G l (G), G l (S).

[0096] The style loss function is used to measure the similarity between the generated image and the style image in style.

[0097] The style loss function calculation formula is as follows:

[0098]

[0099] Among them, S is the style image, G is the generated image, and for the reference style image S and the generated image G, their feature representations on layer l are extracted respectively.l (S) represents the feature representation of image S at layer l, G l (S) represents the feature representation of image G on layer l. l is the total number of elements in the generated image feature map, and is the height H of the generated image feature map. l , Width W l The product of H l ×W l ;

[0100] For the style loss function, L is the number of selected layers, N l is the number of channels used to generate the image feature map; l = L = 1, ||·|| F represents the Frobenius norm, that is, the matrix G l (S)-G l The square root of the sum of the squares of the elements of (G);

[0101] M l Is the total number of elements in the feature map, equal to the product of the height and width of the feature map H l ×W l .

[0102] S35: Iteration process: Use the gradient descent algorithm to update the pixel values ​​of the generated image to minimize the style loss, and continue to iterate until the predetermined number of iterations is reached. At this point, a pseudo good lighting image is obtained, which maintains the edges and textures of the dark light image while having a style close to the target style image.

[0103] S4: Input the pseudo good-light image into the target detection network to obtain the target categories, positions and sizes in the dark-light image, and complete target detection and recognition.

[0104] See also Figure 2 The target detection network includes a backbone network, a multi-scale network, and a prediction and classification module. The backbone network is used to downsample the pseudo-well-illuminated image to obtain multi-level images of different scales. The multi-scale network is used to extract multi-level features from multi-level images of different scales. The prediction and classification module divides the pseudo-well-illuminated image into multiple target categories based on its multi-level features, and outputs the position and size of each target category, which is mapped to the dark light image to obtain the target detection and recognition results of the dark light image. The specific steps are described as follows:

[0105] S41: The backbone network includes convolutional layers B3, B4, and B5, and the multi-scale network includes two high-level layers and two low-level layers, namely low-level layers P3 and P4, base layer P5, and high-level layers P6 and P7. The number of layers of the backbone network and multi-scale network is determined by experiments.

[0106] First, the pseudo good illumination image obtained by S35 is input into the B3 layer to obtain the first feature map with 8 channels and a length × width of 100 × 128. The first feature map is input into the B4 layer to obtain the second feature map with 16 channels and a length × width of 50 × 64. The second feature map is input into the B5 layer to obtain the third feature map with 32 channels and a length × width of 25 × 32.

[0107] S42: Copy the third feature map to obtain the P5 layer feature map, input the P5 layer feature map into the P6 layer to obtain a fourth feature map with 64 channels and a length and width of 13×16, input the fourth feature map into the P7 layer to obtain a fifth feature map with 128 channels and a length and width of 7×8; upsample the P5 layer feature map to obtain a sixth feature map with 16 channels and a length and width of 50×64, add the sixth feature map to the second feature map of the B4 layer to obtain the P4 layer feature map, upsample the P4 layer feature map to obtain a seventh feature map with 8 channels and a length and width of 100×128, add the seventh feature map to the first feature map of the B3 layer to obtain the P3 layer feature map, and the multi-scale network feature extraction is completed.

[0108] S43: The multi-level features extracted by the multi-scale network are from top to bottom: the fifth feature map, the fourth feature map, the P5 layer feature map, the P4 layer feature map, and the P3 layer feature map, which are input into five prediction and classification modules respectively. Each prediction and classification module consists of four convolutional layers and two parallel output layers. The role of the four convolutional layers is to increase the depth of the network and enhance the generalization performance of the network. The four-layer setting is the appropriate number of layers determined by experiments. The image features directly obtained are not suitable for direct classification and regression tasks. Reprocessing the features through four convolutional layers can get better results; the convolution kernels of the four convolutional layers are all 3×3, the step size is 1, the number of channels is 256, and the two parallel outputs include the classification layer and the regression layer. The specific steps are described as follows:

[0109] S431: The prediction head consists of a classification layer and a regression layer. Five prediction heads can obtain five groups of results. The five prediction heads process different scale features from layers P3 to P7 respectively. This design is because features of different scales are more sensitive to detecting targets of different sizes. There are multiple categories of targets to be detected in an image, that is, targets of multiple scales. For example, larger targets tend to have stronger responses on feature maps of higher layers (such as P5), so large-scale targets will be output on large-scale feature layers (such as P5 in the figure below), while small targets are more easily captured on lower layers (such as P3), so small-scale targets will be output on small feature layers. The classification layer outputs each target category in the pseudo-well-illuminated image. The output of the classification layer is in the form of a mask. For example, there are four categories of objects to be detected, such as people, vehicles, objects, equipment, etc. In the prediction head corresponding to P5, the classification layer outputs (0, 0, 0, 1), which means that the layer detects the fourth category, and P4 outputs (1, 0, 0, 1), which means that the first and fourth categories are output, and so on.

[0110] S432: The regression layer uses each pixel in the feature map as the center point of the region of interest. Under dark conditions, the target object may be difficult to distinguish due to insufficient light or background noise. The region of interest refers to the specific area in the image where target detection and recognition are required to optimize the performance and effect of the algorithm. The length and width of each region of interest are predicted and used as the candidate bounding box. In actual applications, the position of the target object in the image may change due to factors such as camera angle and object movement. Alternative bounding boxes at different positions will be generated to cover possible position changes. The output of the regression layer is a four-dimensional vector, which represents the length, width and two-dimensional coordinates of the upper left corner of the candidate bounding box.

[0111] S433: Take the candidate bounding box whose center point is within the true bounding box as a positive sample.

[0112] S44: Adding center point detection loss in the regression layer output process. Using multi-level prediction results, i.e., feature maps extracted by multi-scale networks, can reduce fuzzy positive samples to a certain extent, but the accuracy is still lower than that of the two-stage target detection method. The two-stage target detection method achieves accurate detection of target objects in the image by first extracting the object area and then classifying and refining the positioning of the area. Therefore, the present invention adds center point detection loss in the regression layer output process, which can further reduce the number of redundant regions of interest, i.e., alternative bounding boxes. The center point detection loss calculation formula is as follows:

[0113]

[0114] Among them, centerness represents the center point detection loss, l, r, t, and b are the distances from the center point of each candidate bounding box output by the regression layer to the upper edge, lower edge, left edge, and right edge respectively.

[0115] The overall loss of the target detection network consists of three parts: classification loss, regression loss, and center point detection loss. This formula is used in classification and regression operations. The specific calculation formula is as follows:

[0116]

[0117] Among them, L cls is the classification loss, L reg is the regression loss, N pos is the total number of regions of interest, p x,y , are the true category and predicted category output by the classification layer, respectively, x,y , are the true position and predicted position output by the regression layer, respectively, L(p x,y ,t x,y ) represents the overall loss of category and position, x and y represent the horizontal and vertical coordinates of each center point respectively. represents the classification loss of the true category and the predicted category, λ represents the balance coefficient, Represents the regression loss between the true position and the predicted position.

[0118] S45: Training prediction and classification module. The prediction and classification module uses four convolutional layers to extract features, connecting two parallel classification layers and regression layers. Compared with other target monitoring methods that train a single detector, this module is mainly characterized by a small network structure and fast speed. At the same time, due to the addition of center point prediction, it can achieve relatively high accuracy in the case of a small model.

[0119] The training data set contains multiple groups of samples, each of which contains an image and an annotation file. The annotation file contains the parameters of the bounding box of the object in the image. It is generally a four-dimensional vector that can describe a square box at any position. The purpose of the entire training process is to use a large amount of labeled training data, that is, the center point and redundant area must be labeled first, so that the prediction and classification modules can accurately understand and identify the objects in the image. The overall loss function L(p x,y ,t x,y ) is used to update the model parameters to the optimal parameters and improve the generalization ability of the model on unknown data, so that it can accurately and quickly complete object detection and classification tasks in practical applications. This step can reduce negative samples, that is, redundant candidate bounding boxes, whose center points will deviate from the center of the candidate bounding box.

[0120] In practical application fields, positive samples generally refer to objects or targets that are successfully detected and identified by the system. These objects usually need to be accurately located and classified under low-light conditions through algorithms and techniques. During the training process, the positive sample is the label bounding box provided in the data set. The real box is obtained through data collection and manual annotation. In the traditional positive sample determination method, a threshold is set, such as 0.5, and each output will calculate the intersection and union ratio with the real label. If it is greater than the threshold, it is set as a positive sample. Therefore, the traditional detection method needs to calculate the intersection and union ratio of the real box and the alternative bounding box, and set the threshold, and take the predicted box greater than the threshold as a positive sample. In actual images and video frames, the target is inside the real bounding box, and the real bounding box is the real box. In the present invention, it is found that if the center point of the output alternative bounding box is inside the real box, it means that the distance from the alternative bounding box to the real box will not be too far, so there is no need to calculate the threshold, and the alternative bounding box is directly output as a positive sample. The simplicity and accuracy of the output results are guaranteed.

[0121] The center point of the positive sample is at the center of the candidate bounding box, and all candidate boxes whose center points are within the real box are set as positive samples. Compared with traditional detection methods, the performance of the target detection and recognition method of the present invention is not affected by hyperparameters and has a faster calculation speed.

[0122] S5: Calculate the confidence of all positive samples and select the four-dimensional position vector of the positive sample with the highest confidence as the final output.

[0123] In order to filter out the positive samples with the highest confidence, we must first obtain the confidence and obtain the classification score of each positive sample from the classification layer. These scores represent the probability that each candidate bounding box belongs to a specific category. We set a confidence threshold, such as 0.5, to remove those positive samples with lower confidence. We sort all positive samples that pass the threshold screening from high to low according to their confidence, and select the positive sample with the highest confidence from the sorted list. Finally, we verify and confirm the screened positive samples with the highest confidence, and use non-maximum suppression (NMS) to remove overlapping bounding boxes.

[0124] Therefore, the final output result is: the category output of the classification layer and the four-dimensional position vector of the positive sample output by the regression layer.

[0125] As shown in the figure below, Figure 3 Middle a, Figure 4 The middle a is a dark light image under low illumination. Figure 3 Middle b, Figure 4 b in the figure is the recognition result of low-light image using image enhancement method and image denoising method. Figure 3 Middle c, Figure 4Figure c is the detection and recognition result of the method of the present invention. It can be clearly seen that the present invention can achieve detection and recognition under poor lighting conditions, and the recognition result of the method of the present invention has a higher accuracy rate and a clearer recognition result.

[0126] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0127] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for target detection and recognition in a low-light environment, characterized in that: The following steps are involved: S1: performing histogram equalization processing on the dark light image to obtain a first image; S2: Perform Wiener filtering on the first image to obtain a second image; S3: construct a style transformation network; input the second image into the style transformation network to generate a pseudo well-illuminated image; S4: Input the pseudo good illumination image into the target detection network for detection and recognition; S5: The classification layer and regression layer in the target detection network output the final detection and recognition results.

2. The method for target detection and recognition in a low-light environment according to claim 1, characterized in that: The S1 includes: The calculation formula for histogram equalization is as follows: Among them, s k is the pixel value obtained after histogram equalization processing, T(r k ) represents the gray level r in the dark light image k The pixels are transformed by histogram, L represents the total number of gray levels in the image, L-1 is the maximum gray level; j represents the gray level, k represents the gray level index, n j p represents the total number of pixels with gray level j, where gray level j is an integer value between 0 and 254, 0≤k≤L-1; r (r j ) represents the continuous probability density of the dark-light image in the grayscale continuous form, M and N represent the height and width of the dark-light image respectively, and MN represents the total number of pixels in the dark-light image.

3. The method for target detection and recognition in a low-light environment according to claim 2, characterized in that: The S2 includes: S21: Use Gaussian noise as the noise signal n(t) to simulate the noise conditions encountered in actual shooting; S22: transform the first image obtained in S1 from the spatial domain to the frequency domain using a two-dimensional discrete Fourier transform DFT; S23: Determine the degradation function. The calculation formula of the degradation function is as follows: H(u,v)=exp(-0.0025×((u-M / 2) 2 +(v-N / 2) 2 ) 56 ) Wherein, u and v represent the horizontal and vertical coordinate values ​​of the first image in the frequency domain, respectively, and M and N represent the height and width of the first image, respectively; S24: Perform a Wiener filter operation on the image converted in S22; the calculation formula of the Wiener filter operation is as follows: Where, f is the input, i.e., the horizontal and vertical coordinate values ​​u and v of the first image in the frequency domain; G(f) represents the image obtained by performing Wiener filtering on the first image, i.e., the second image; H(f) represents the degradation function, i.e., H(u,v); |H(f)| refers to the modulus of the degradation function; S(f) is the power spectrum of the first image x(t); N(f) is the power spectrum of the noise signal n(t); SNR(f) is the signal-to-noise ratio; S25: converting the second image back to the spatial domain through two-dimensional inverse discrete Fourier transform IDFT and then saving it.

4. The method for target detection and recognition in a low-light environment according to claim 3, characterized in that: The S3 includes: S31: Construct a style transformation network; use a simplified VGG19 network as the style transformation network. The style transformation network consists of five convolution blocks. The first four convolution blocks are composed of two convolution layers and a maximum pooling layer connected in sequence. The fifth convolution block is composed of two convolution layers connected, where the output of the second convolution layer is used as the final feature output of the style transformation network. The activation function of each convolution block is relu, the convolution kernel size of each convolution layer is 3×3, the convolution step size is 1, and the number of channels of the five convolution blocks is 64, 128, 256, 512, and 512 respectively. S32: Input the well-illuminated image into the style transformation network, and use the output of the fourth convolutional block as the style image feature map S; S33: The second image obtained in S25 is input into the style transformation network of S31, and the output of the second convolution layer of the fifth convolution block is to generate the image feature map G.

5. The method for target detection and recognition in a low-light environment according to claim 4, characterized in that: The S3 further includes: S34: Use the Gram matrix to calculate the correlation between different feature channels in the generated image feature map and the style image feature map, and define the style loss function; The Gram matrix calculation formula is: Where, for a given convolutional layer l, l is the second convolutional layer of the fifth convolutional block; It represents the inner product of the c-th dimension channel feature and the d-th dimension channel feature in the second convolution layer of the fifth convolution block. I and J represent the height and width of the generated image feature map, respectively. i and j represent the position coordinates of the pixels in the generated image feature map, respectively. represents the feature activation value at position ij in the c-th dimension channel in the second convolution layer of the fifth convolution block, Represents the feature activation value at position ij in the d-th dimension channel in the second convolution layer of the fifth convolution block; Use the Gram matrix to calculate the Gram matrix of the generated image feature map and style image feature map, respectively, G l (G), G l (S); The calculation formula of style loss function L(S,G) is as follows: Among them, M l is the total number of elements in the generated image feature map, and is the height H of the generated image feature map. l , Width W l The product of H l ×W l ; N l is the number of channels used to generate the image feature map; l = L = 1, ||·|| F represents the Frobenius norm, that is, the matrix G l (S)-G l The square root of the sum of the squares of the elements of (G); S35: Iterative process; using the gradient descent algorithm to update the pixel values ​​of the generated image feature map G to minimize the style loss function, iterating continuously until a predetermined number of iterations is reached; obtaining a pseudo well-illuminated image.

6. The method for target detection and recognition in a low-light environment according to claim 5, characterized in that: The S4 includes: The target detection network includes a backbone network, a multi-scale network, and prediction and classification modules; S41: The backbone network includes convolutional layers B3, B4, and B5, and the multi-scale network includes low-level layers P3, P4, base layer P5, and high-level layers P6 and P7; In the backbone network, the pseudo good illumination image obtained by S35 is input into the B3 layer to obtain a first feature map with 8 channels and a length × width of 100 × 128. The first feature map is input into the B4 layer to obtain a second feature map with 16 channels and a length × width of 50 × 64. The second feature map is input into the B5 layer to obtain a third feature map with 32 channels and a length × width of 25 × 32. S42: In the multi-scale network, copy the third feature map to obtain a P5 layer feature map, input the P5 layer feature map to the P6 layer to obtain a fourth feature map with 64 channels and a length and width of 13×16, input the fourth feature map to the P7 layer to obtain a fifth feature map with 128 channels and a length and width of 7×8; upsample the P5 layer feature map to obtain a sixth feature map with 16 channels and a length and width of 50×64, add the sixth feature map to the second feature map of the B4 layer to obtain the P4 layer feature map, upsample the P4 layer feature map to obtain a seventh feature map with 8 channels and a length and width of 100×128, and add the seventh feature map to the first feature map of the B3 layer to obtain the P3 layer feature map; S43: The multi-level features extracted by the multi-scale network are, from top to bottom: the fifth feature map, the fourth feature map, the P5 layer feature map, the P4 layer feature map, and the P3 layer feature map. The five feature maps are respectively input into five prediction and classification modules. Each prediction and classification module consists of four convolutional layers and two parallel output layers. The convolution kernels of the four convolutional layers are all 3×3, the step size is 1, the number of channels is 256, and the two parallel output layers are the classification layer and the regression layer.

7. The method for target detection and recognition in a low-light environment according to claim 6, characterized in that: The output process of the classification layer and regression layer in S43 is: S431: The classification layer outputs each target category in the pseudo good illumination image in the form of a mask; S432: The regression layer uses each pixel point in the input feature map as the center point of the region of interest, predicts the length and width of each region of interest, and uses it as a candidate bounding box. The output of the regression layer is a four-dimensional vector, which represents the two-dimensional coordinates of the length, width, and upper left corner of the candidate bounding box; S433: All candidate bounding boxes whose center points are within the true bounding box are taken as positive samples.

8. The method for target detection and recognition in a low-light environment according to claim 7, characterized in that: The S4 further comprises: S44: Add the center point detection loss in the regression layer output process. The center point detection loss calculation formula is as follows: Among them, centerness represents the center point detection loss, l, r, t, and b are the distances from the center point of each candidate bounding box output by the regression layer to the upper edge, lower edge, left edge, and right edge respectively; The overall loss of the target detection network consists of three parts: classification loss, regression loss, and center point detection loss. The specific calculation formula is as follows: Among them, L cls is the classification loss, L reg is the regression loss, N pos is the total number of regions of interest, p x,y , are the true category and predicted category output by the classification layer, respectively, x,y , are the true position and predicted position output by the regression layer, respectively, L(p x,y ,t x,y ) represents the overall loss of the target detection network, x and y represent the horizontal and vertical coordinates of each center point, respectively. represents the classification loss of the true category and the predicted category, λ represents the balance coefficient, Regression loss representing the true position and predicted position; S45: training the prediction and classification module; first collect and label the data of the center point, redundant candidate bounding box, and true bounding box, then use the training set to train the prediction and classification module, and update the model parameters of the prediction and classification module to minimize the overall loss function L(p x,y ,t x,y ), and iterates continuously until the predetermined number of iterations is reached.

9. The method for target detection and recognition in a low-light environment according to claim 7, characterized in that: The S5 includes: Calculate the confidence of all positive samples, obtain the classification score of each positive sample from the classification layer, set the confidence threshold, sort all positive samples that pass the threshold screening by their confidence from high to low, select the positive sample with the highest confidence, verify and confirm the screened positive sample with the highest confidence, and use non-maximum suppression to remove overlapping bounding boxes; The final output result is: the category output of the classification layer and the four-dimensional position vector of the positive sample with the highest confidence output by the regression layer.