Intelligent detection method for locomotive sanding device in low-light environment

Abstract

The invention discloses an intelligent detection method for a locomotive sanding device in a low-light environment, and the method comprises the steps: carrying out the image enhancement of a locomotive sanding device image through the fusion of a geometric sparse representation branch and a low-light enhancement branch of a low-light enhancement network, and obtaining an enhanced locomotive sanding device image; and inputting the enhanced locomotive sanding device image into the trained target detection model, and outputting a prediction result of the sanding pipe area. The method comprises the following steps: generating an enhanced locomotive sanding device image by utilizing a geometric sparse representation branch and a low light enhancement branch through a low light enhancement network; and meanwhile, the sanding pipe area is predicted based on the enhanced locomotive sanding device image through the backbone network, the neck network and the head network of the target detection model, so that the stability and the accuracy of sanding detection of the locomotive sanding device are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning and target detection, and in particular to an intelligent detection method for a locomotive sand spreading device in a low-light environment. Background Art

[0002] In railway operations, sand spreading is a key means of improving wheel-rail adhesion coefficient. It plays an irreplaceable role in ensuring locomotive traction efficiency, preventing idling, and ensuring that the train braking distance meets safety standards. This technical detail is directly related to the stability and safety of train operation. Therefore, real-time monitoring of the locomotive's sand spreading status and ensuring that the sand spreading device works properly are an indispensable part of railway maintenance operations. However, in actual operation, how to accurately determine whether the locomotive is spreading sand is a complex and difficult problem.

[0003] With the rapid development of computer vision technology, it has become possible to use image recognition technology to monitor the sand spreading status of locomotives. Through the camera installed on the locomotive, the image information of the sand spreading device is collected in real time, and then it can be analyzed and judged whether the sand spreading behavior occurs. However, this solution faces severe challenges in practical application.

[0004] Under low-light conditions, images captured by cameras often show low contrast, high noise, and blurred target details. These unfavorable factors greatly affect the performance of traditional target detection algorithms, making it extremely difficult to accurately identify sand-spreading areas. Although low-light image enhancement technology has made significant progress in recent years, most of these technologies focus on improving image quality while ignoring the close integration with target detection tasks. As a result, in actual detection, even the enhanced images are still unsatisfactory in detection accuracy. Therefore, how to design a method that can effectively solve the low detection accuracy of locomotive sand-spreading devices in low-light environments is a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is: how to provide an intelligent detection method for a locomotive sand-spreading device in a low-light environment, generate an enhanced locomotive sand-spreading device image through a low-light enhancement network using a geometric sparse representation branch and a low-light enhancement branch; at the same time, the sand-spreading pipe area is predicted based on the enhanced locomotive sand-spreading device image through the backbone network, neck network and head network of the target detection model, thereby improving the stability and accuracy of sand-spreading detection of the locomotive sand-spreading device.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] The intelligent detection method of a locomotive sand spreading device in a low-light environment includes:

[0008] S1: Acquire an image of a locomotive sand spreading device to be detected;

[0009] S2: Perform image enhancement on the locomotive sand spreading device image through a low-light enhancement network to obtain an enhanced locomotive sand spreading device image;

[0010] The processing steps of the low-light enhancement network are as follows:

[0011] S201: Calculating a geometric sparse coefficient map of a sand spreading device image of a vehicle;

[0012] S202: Input the geometric sparse coefficient graph into the geometric sparse representation branch, and output the corresponding enhanced geometric sparse coefficient graph;

[0013] S203: inputting the locomotive sand spreading device image into the low-light enhancement branch, and outputting a corresponding low-light enhancement feature map;

[0014] S204: Fusing the outputs of the penultimate layer of the low-light enhancement branch and the geometric sparse representation branch to obtain a corresponding enhanced locomotive sand spreading device image;

[0015] S3: Input the enhanced locomotive sand spreading device image into the trained object detection model and output the prediction result of the sand spreading pipe area;

[0016] S4: Outputting the prediction result of the sand spreading pipe area as the detection result of the locomotive sand spreading device image.

[0017] Preferably, in step S201, the geometric sparse coefficient graph is calculated by the following formula:

[0018]

[0019] Where: J(a k ,∈ k ) represents the geometric sparse coefficient of the locomotive sand spreading device image I; ω k Represents an image block; a k represents the geometric sparse coefficient of pixel k; k represents pixel; D represents dictionary image; λ represents a k The regularization parameter on ∈ k Represents coefficient; μ k Represents the image block ω k The mean value of D in the Represents the image block ω k The mean value of I in Represents the image block ω k The variance of D in .

[0020] Preferably, in step S202, the geometric sparse representation branch includes a first convolution module, a combination of four modules, a second convolution module, a combination of convolution modules, and a third convolution module connected end to end in sequence;

[0021] The module combination 1 includes a series module, three RRDB modules and a convolution module connected end to end in sequence;

[0022] The convolutional module combination includes two convolutional modules connected end to end in sequence;

[0023] The feature map output by the first convolution module and the feature map output by the second convolution module are concatenated and used as the input of the convolution module combination; the output of the convolution module combination is used as the enhanced geometric sparse coefficient map output by the geometric sparse representation branch.

[0024] Preferably, in step S203, the low-light enhancement branch includes a first convolution module, a second combination of five modules, a second convolution module, and a convolution module combination connected end to end in sequence;

[0025] Module combination 2 includes two channel attention dense blocks and three residual dense modules connected end to end in sequence;

[0026] The convolutional module combination includes two convolutional modules connected end to end in sequence;

[0027] The feature map output by the first convolution module and the feature map output by the combined convolution module are concatenated as the low-light enhancement feature map output by the low-light enhancement branch;

[0028] The feature maps output by the first to fourth module combination 2 are used as prior features and correspond one-to-one as the input of the series modules in the first to fourth module combination 1 in the geometric sparse representation branch.

[0029] Preferably, in step S204, the outputs of the penultimate layer of the low-light enhancement branch and the geometric sparse representation branch are fused through a fusion network;

[0030] The fusion network includes a module combination three connected end to end in sequence and a convolution module combination;

[0031] Module combination three includes a series module and a fusion module connected end to end in sequence;

[0032] The convolutional module combination includes two convolutional modules connected end to end in sequence;

[0033] The low-light enhanced feature map and the enhanced geometric sparse coefficient map are used as the input of the series module in module combination three; the feature map output by the convolution module combination is used as the enhanced locomotive sand spreading device image output by the fusion network.

[0034] Preferably, the channel attention dense block includes two dense modules and a channel attention module connected end to end in sequence;

[0035] The dense module includes four module combinations connected end to end in sequence and a convolution module; the module combination four includes a convolution module connected end to end in sequence and a ReLU activation function layer;

[0036] The channel attention module includes a global pooling layer, a fully connected layer, a ReLU activation function layer, a fully connected layer, and a Sigmoid activation function layer connected end to end in sequence;

[0037] The original feature map input to the channel attention dense block is concatenated with the feature map output by the first dense module and used as the input to the second dense module; the feature maps input and output by the second dense module are concatenated and used as the input to the channel attention module; the feature maps input and output by the channel attention module are concatenated and then concatenated with the original feature map as the feature map output by the channel attention dense block.

[0038] Preferably, in step S202, the geometric sparse representation branch is trained based on the generative adversarial network, and the specific training steps are as follows:

[0039] S2021: Acquiring low-light images I Low and real images and geometric sparse coefficient maps for low-light and real images;

[0040] S2022: Train the generator: fix the parameters of the discriminator; use the geometric sparse coefficient map of the low-light image as the input of the generator, and combine the prior features of the low-light enhancement branch to generate an enhanced geometric sparse coefficient map Based on enhanced geometric sparse coefficient graph The loss function of the generator is calculated by using the geometric sparse coefficient graph of the real image. And the pixel loss function And further calculate the total loss function Update the generator parameters through the back-propagation algorithm to minimize its total loss function

[0041] The formula is:

[0042]

[0043] Where: β GSR and γ GSR They represent the enhanced geometric sparse coefficient graphs Pixel loss weight and adversarial loss weight; GSR (I Low ) and A Low Both represent the geometric sparse coefficient maps of low-light images; represents the computational expectation; G GSR , D GSR Represent the generator and discriminator respectively; G GSR (GSR(ILow )) and G(A Low ) both represent enhanced geometric sparse coefficient maps generated by the generator according to the geometric sparse coefficient map of the low-light image; A geometric sparse coefficient map representing a real image;

[0044] S2023: Training the discriminator: fix the parameters of the generator; use the geometric sparse coefficient map of the low-light image as the input of the generator, and combine the prior features of the low-light enhancement branch to generate an enhanced geometric sparse coefficient map Enhanced Geometric Sparse Coefficient Graph Perform image reconstruction to obtain the restored image Will restore the image and real images Input the discriminator and calculate the discriminator loss function Update the discriminator parameters through the back-propagation algorithm to minimize the discriminator loss function

[0045] The formula of the discriminator loss function is expressed as:

[0046]

[0047] Where: All represent computational expectations; Represents the discriminator's restoration of the image The output probability of is the probability of a real image; Represents the discriminator for the real image The output probability of the discriminator is that the real image is the probability of a real image;

[0048] Restore the image The calculation formula is expressed as:

[0049]

[0050] Where: Represents the value of the enhanced geometric sparse coefficient map at pixel k; Represents the value of the restored image at pixel k; Represents the value of the real image at pixel k; represents the representation error at pixel k, which is used to represent the residual or uncertainty in the image reconstruction process;

[0051] S2023: Repeat steps S2021 to S2022 to alternately train the generator and the discriminator;

[0052] S2024: Use the completed generator as the trained geometric sparse representation branch.

[0053] Preferably, in step S203, the low-light enhancement branch is trained based on a generative adversarial network, and the specific training steps are as follows:

[0054] S2031: Acquiring low-light image I Low and real images And the geometric sparse representation branch is based on the low-light image I Low Enhanced geometric sparse coefficient map generated by the geometric sparse coefficient map

[0055] S2032: Training the discriminator: fix the parameters of the generator; combine the low-light image and the enhanced geometric sparse coefficient map generated by the geometric sparse representation branch As the input of the generator, generate low-light enhancement feature map The low light enhancement feature map and real images Input the discriminator, which distinguishes whether the input image is a low-light enhanced feature map generated by the generator or a real image, and calculates the discriminator loss function Update the discriminator parameters through the back-propagation algorithm to minimize the discriminator loss function

[0056] The formula of the discriminator loss function is expressed as:

[0057]

[0058] Where: All represent computational expectations; Represents the probability that the discriminator believes that the low-light enhanced feature map generated by the generator is a real image; Represents the probability that the discriminator considers the real image to be a real image;

[0059] S2033: Training the generator: fix the parameters of the discriminator; combine the low-light image and the enhanced geometric sparse coefficient map generated by the geometric sparse representation branch As the input of the generator, generate low-light enhancement feature map The generated low-light enhancement feature map With real image Input into the discriminator to obtain the discriminator's judgment result; according to the discriminator's judgment result and the low-light enhancement feature map With the real feature map The difference between them is used to calculate the generator loss function Perceptual Loss Function Pixel loss function And the pixel loss function after introducing the geometric sparse coefficient map And further calculate the total loss function Update the parameters of the generator through the back-propagation algorithm to minimize its total loss function

[0060] The formula is:

[0061]

[0062] Where: γ I Represent the low light enhancement feature map The pixel loss weight, the pixel loss weight after introducing the geometric sparse coefficient graph, and the adversarial loss weight; represents the computational expectation; G(I Low ) indicates that the generator is based on the low-light image I LLow Generated low-light enhancement feature map; GSR(G(I Low )) represents an enhanced geometric sparse coefficient map generated by the root geometric sparse representation branch according to the geometric sparse coefficient map of the low-light image; C is the geometric sparse coefficient graph representing the real image; j W represents the number of feature maps in the jth layer of the generator; j H represents the width of the feature map of the jth layer of the generator; j represents the height of the feature map of the jth layer of the generator; φ J represents the j-th layer output of the generator; G represents the generator; D I represents the discriminator;

[0063] S2034: Repeat steps S2021 to S2022 to alternately train the generator and the discriminator;

[0064] S2035: Use the completed generator as the trained low-light enhancement branch.

[0065] Preferably, in step S301, the backbone network includes a combination of three feature extraction modules, a pyramid pooling module, and an attention module connected end to end in sequence; the feature extraction module combination includes a convolution module and a C2F module connected end to end in sequence; the attention module includes two submodules, channel attention and spatial attention; in the channel attention module: the input features are subjected to maximum pooling and average pooling to obtain two different features, which are then sent to a multi-layer perceptron, and then the two feature maps are merged, and the merged feature maps are sent to the spatial attention module, in which the input feature map is subjected to maximum pooling and average pooling along the channel axis to obtain two feature maps, the two feature maps are spliced, and then a convolution layer with a kernel size of 7 is applied to generate a spatial attention map;

[0066] In step S302, the neck module includes an upsampling module, a first cross-layer connection, a first C2F module, a convolution module, a second cross-layer connection, and a second C2F module connected end to end in sequence; the cross-layer connection can splice feature information between different layers and merge features of different layers;

[0067] In step S303, the head network generates a feature map for predicting categories and bounding boxes, and produces the final detection result.

[0068] Preferably, the classification loss function when training the target detection model is as follows:

[0069]

[0070] Where: L i represents the loss of the i-th sample; N represents the total number of samples; y i represents the true label of the i-th sample; p i Indicates the probability that the i-th sample predicted by the model belongs to the positive class;

[0071] Bounding box regression loss function when training object detection models as follows:

[0072]

[0073] Where: represents the weighting coefficient; represents IOU loss; W g Indicates the actual frame width; H g Indicates the real frame height; W i Indicates the width of the overlapping area; H i Indicates the height of the overlapping area; S u Represents the area of ​​the predicted box and the true box; * indicates that W g , H g Separate from the computational graph; x represents the horizontal coordinate of the center point of the prediction box; y represents the vertical coordinate of the center point of the prediction box; x gt Indicates the horizontal coordinate of the center point of the real frame; y gt Indicates the vertical coordinate of the center point of the real box.

[0074] Compared with the prior art, the intelligent detection method of the locomotive sand spreading device in a low-light environment of the present invention has the following beneficial effects:

[0075] In the present invention, the low-light enhancement network generates an enhanced locomotive sand spreading device image through a geometric sparse representation branch and a low-light enhancement branch. First, the geometric sparse representation branch extracts and enhances the geometric structure information in the image by analyzing the geometric sparse coefficient map of the image. These structural information are crucial for identifying objects with specific shapes and positions such as sand spreading pipes, which helps to stably locate the sand spreading pipe area under low-light conditions and improve the stability of sand spreading detection of locomotive sand spreading devices. Secondly, the low-light enhancement branch focuses on improving the brightness, contrast and clarity of the image, effectively suppressing noise, so that the image details that were originally blurred in low-light environments can be restored, which helps subsequent target detection tasks to more accurately identify the sand spreading pipe area; at the same time, the quality and features of the image processed by the low-light enhancement network have been significantly improved, thereby improving the accuracy and robustness of locomotive sand spreading device detection in low-light environments. Finally, the output before the fusion layer of the low-light enhancement branch and the output of the penultimate layer of the geometric sparse representation branch are fused, which can make full use of the complementary features extracted by the two branches, enhance the overall quality of the image, and retain the key geometric structure information. The image quality, including brightness, contrast and detail information, is significantly improved, making the target that was originally difficult to identify under low-light conditions become clearly visible, thereby greatly improving the accuracy of sand spreading detection of the locomotive sand spreading device.

[0076] Based on the generation of enhanced locomotive sand-spreading device images through a low-light enhancement network, the present invention further predicts the sand-spreading pipe area through a target detection model. First, the backbone network of the target detection model can efficiently extract multi-level features in the image through a combination of convolution modules, C2F modules, spatial pyramid pooling modules and attention modules. These features not only contain rich detail information, but also cover global context information, providing a solid foundation for subsequent feature fusion and target detection; at the same time, the attention module is introduced into the backbone network, so that the model can pay more attention to multi-scale information, which is particularly important for detecting targets of different sizes and shapes, because in practical applications, the size and shape of targets are often diverse. By enhancing the processing ability of multi-scale information, the target in the image can be more accurately identified. Secondly, the neck network realizes the effective fusion and splicing of features of different scales through the cooperation of convolution modules, cross-layer connection modules, upsampling modules and C2F modules, which not only improves the robustness and expression ability of the features, but also helps the model to better adapt to sand-spreading pipe areas of different scales and shapes. Finally, the head network uses advanced detection algorithms (such as YOLO and Faster R-CNN) based on the fusion features to accurately locate the sand-spreading pipe area. Through regression prediction and classification judgment, the model can accurately identify the position and shape of the sand-spreading pipe, providing a reliable basis for subsequent decision-making and control. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to make the purpose, technical solution and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0078] Figure 1 It is a logic block diagram of an intelligent detection method for a locomotive sand spreading device in a low-light environment;

[0079] Figure 2 This is the network structure diagram of the low-light enhancement network;

[0080] Figure 3 Network structure diagram for channel attention dense block;

[0081] Figure 4 This is the network structure diagram of the attention module in the backbone network;

[0082] Figure 5 This is the network structure diagram of the C2F module;

[0083] Figure 6 This is the network structure diagram of the pyramid pooling module. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0085] The following is a further detailed description through specific implementation methods:

[0086] Example:

[0087] This embodiment discloses an intelligent detection method for a locomotive sand spreading device in a low-light environment.

[0088] like Figure 1 As shown, the intelligent detection method of the locomotive sand spreading device in a low-light environment includes:

[0089] S1: Acquire an image of a locomotive sand spreading device to be detected;

[0090] S2: Perform image enhancement on the locomotive sand spreading device image through a low-light enhancement network to obtain an enhanced locomotive sand spreading device image;

[0091] The processing steps of the low-light enhancement network are as follows:

[0092] S201: Calculating a geometric sparse coefficient map of a sand spreading device image of a vehicle;

[0093] S202: Input the geometric sparse coefficient graph into the geometric sparse representation branch, and output the corresponding enhanced geometric sparse coefficient graph;

[0094] S203: inputting the locomotive sand spreading device image into the low-light enhancement branch, and outputting a corresponding low-light enhancement feature map;

[0095] S204: Fusing the outputs of the penultimate layer of the low-light enhancement branch and the geometric sparse representation branch to obtain a corresponding enhanced locomotive sand spreading device image;

[0096] S3: Input the enhanced locomotive sand spreading device image into the trained object detection model and output the prediction result of the sand spreading pipe area;

[0097] The processing steps of the object detection model are as follows:

[0098] S301: The backbone network extracts features of the enhanced locomotive sand spreading device image through a convolution module, a C2F module, a spatial pyramid pooling module, and an attention module to obtain image features;

[0099] S302: The neck network performs feature fusion and splicing on the image features through the convolution module, the cross-layer connection module, the upsampling module, and the C2F module to obtain fusion features;

[0100] S303: The head network outputs the prediction result of the sand spreading pipe area according to the fusion features;

[0101] S4: Outputting the prediction result of the sand spreading pipe area as the detection result of the locomotive sand spreading device image.

[0102] In this embodiment, after the prediction result of the sand spreading pipe area is obtained, it can be used to determine whether sand spreading is being performed through the image.

[0103] In the present invention, the low-light enhancement network generates an enhanced locomotive sand spreading device image through a geometric sparse representation branch and a low-light enhancement branch. First, the geometric sparse representation branch extracts and enhances the geometric structure information in the image by analyzing the geometric sparse coefficient map of the image. These structural information are crucial for identifying objects with specific shapes and positions such as sand spreading pipes, which helps to stably locate the sand spreading pipe area under low-light conditions and improve the stability of sand spreading detection of locomotive sand spreading devices. Secondly, the low-light enhancement branch focuses on improving the brightness, contrast and clarity of the image, effectively suppressing noise, so that the image details that were originally blurred in low-light environments can be restored, which helps subsequent target detection tasks to more accurately identify the sand spreading pipe area; at the same time, the quality and features of the image processed by the low-light enhancement network have been significantly improved, thereby improving the accuracy and robustness of locomotive sand spreading device detection in low-light environments. Finally, the output before the fusion layer of the low-light enhancement branch and the output of the penultimate layer of the geometric sparse representation branch are fused, which can make full use of the complementary features extracted by the two branches, enhance the overall quality of the image, and retain the key geometric structure information. The image quality, including brightness, contrast and detail information, is significantly improved, making the target that was originally difficult to identify under low-light conditions become clearly visible, thereby greatly improving the accuracy of sand spreading detection of the locomotive sand spreading device.

[0104] Based on the generation of enhanced locomotive sand-spreading device images through a low-light enhancement network, the present invention further predicts the sand-spreading pipe area through a target detection model. First, the backbone network of the target detection model can efficiently extract multi-level features in the image through a combination of convolution modules, C2F modules, spatial pyramid pooling modules and attention modules. These features not only contain rich detail information, but also cover global context information, providing a solid foundation for subsequent feature fusion and target detection; at the same time, the attention module is introduced into the backbone network, so that the model can pay more attention to multi-scale information, which is particularly important for detecting targets of different sizes and shapes, because in practical applications, the size and shape of targets are often diverse. By enhancing the processing ability of multi-scale information, the target in the image can be more accurately identified. Secondly, the neck network realizes the effective fusion and splicing of features of different scales through the cooperation of convolution modules, cross-layer connection modules, upsampling modules and C2F modules, which not only improves the robustness and expression ability of the features, but also helps the model to better adapt to sand-spreading pipe areas of different scales and shapes. Finally, the head network uses advanced detection algorithms (such as YOLO and Faster R-CNN) based on the fusion features to accurately locate the sand-spreading pipe area. Through regression prediction and classification judgment, the model can accurately identify the position and shape of the sand-spreading pipe, providing a reliable basis for subsequent decision-making and control.

[0105] In order to better introduce the technical solution of the present invention, this embodiment is described through the following parts.

[0106] 1. Geometric Sparse Coefficient Graph

[0107] In this embodiment, the geometric sparse coefficient graph is calculated by the following formula:

[0108]

[0109] Where: J(a k ,∈ k ) represents the geometric sparse coefficient of the locomotive sand spreading device image I; ω k Represents an image block; a k represents the geometric sparse coefficient of pixel k; k represents pixel; D represents dictionary image (in geometric sparse representation, dictionary image is a set of basic elements used to represent signals or data. These dictionary elements can be regarded as building blocks of images. By linearly combining these elements, the target image can be effectively reconstructed or approximated); λ represents a k The regularization parameter on ∈ k Represents coefficient; μ k Represents the image block ω k The mean value of D in the Represents the image block ω k The mean value of I in Represents the image block ω k The variance of D in .

[0110] 2. Low-light Enhanced Network

[0111] Combination Figure 2 As shown, the low-light enhancement network includes a geometric sparse representation branch and a low-light enhancement branch.

[0112] 1. Geometric sparse representation branch

[0113] The geometric sparse representation branch includes a first convolution module, a combination of four modules, a second convolution module, a combination of convolution modules, and a third convolution module, which are sequentially connected end to end;

[0114] The module combination 1 includes a series module, three RRDB modules and a convolution module connected end to end in sequence;

[0115] The convolutional module combination includes two convolutional modules connected end to end in sequence;

[0116] The feature map output by the first convolution module and the feature map output by the second convolution module are concatenated and used as the input of the convolution module combination; the output of the convolution module combination is used as the enhanced geometric sparse coefficient map output by the geometric sparse representation branch.

[0117] 2. Low-light enhancement branch

[0118] The low-light enhancement branch includes the first convolution module, the five module combination two, the second convolution module and the convolution module combination connected end to end in sequence;

[0119] Module combination 2 includes two channel attention dense blocks and three residual dense modules connected end to end in sequence;

[0120] The convolutional module combination includes two convolutional modules connected end to end in sequence;

[0121] The feature map output by the first convolution module and the feature map output by the combined convolution module are concatenated as the low-light enhancement feature map output by the low-light enhancement branch;

[0122] The feature maps output by the first to fourth module combination 2 are used as prior features and correspond one-to-one as the input of the series modules in the first to fourth module combination 1 in the geometric sparse representation branch.

[0123] In this embodiment, the residual dense module (RRDB module) enhances information flow and gradient propagation through multi-level feature extraction and residual connection. In terms of working logic, RRDB nests multiple dense blocks together, and each dense block directly connects the input and output to enhance feature reuse, while using residual learning to reduce the difficulty of training. This can effectively improve image quality, especially in terms of details and texture.

[0124] 3. Fusion of two branches

[0125] The output of the penultimate layer of the low-light enhancement branch and the geometric sparse representation branch is fused through a fusion network;

[0126] The fusion network includes a module combination three connected end to end in sequence and a convolution module combination;

[0127] Module combination three includes a series module and a fusion module connected end to end in sequence;

[0128] The convolutional module combination includes two convolutional modules connected end to end in sequence;

[0129] The low-light enhanced feature map and the enhanced geometric sparse coefficient map are used as the input of the series module in module combination three; the feature map output by the convolution module combination is used as the enhanced locomotive sand spreading device image output by the fusion network.

[0130] 4. Channel attention dense block

[0131] Combination Figure 3 As shown, the channel attention dense block includes two dense modules and a channel attention module connected end to end in sequence;

[0132] The dense module includes four module combinations connected end to end in sequence and a convolution module; the module combination four includes a convolution module connected end to end in sequence and a ReLU activation function layer;

[0133] The channel attention module includes a global pooling layer, a fully connected layer (FC layer), a ReLU activation function layer, a FC layer, and a Sigmoid activation function layer connected end to end in sequence;

[0134] The original feature map input to the channel attention dense block is concatenated with the feature map output by the first dense module and used as the input to the second dense module; the feature maps input and output by the second dense module are concatenated and used as the input to the channel attention module; the feature maps input and output by the channel attention module are concatenated and then concatenated with the original feature map as the feature map output by the channel attention dense block.

[0135] 3. Training Geometric Sparse Representation Branch

[0136] The geometric sparse representation branch is trained based on the generative adversarial network. The specific training steps are as follows:

[0137] S2021: Acquiring low-light images I Low and real images and geometric sparse coefficient maps for low-light and real images;

[0138] S2022: Train the generator: fix the parameters of the discriminator; use the geometric sparse coefficient map of the low-light image as the input of the generator, and combine the prior features of the low-light enhancement branch to generate an enhanced geometric sparse coefficient map Based on enhanced geometric sparse coefficient graph The loss function of the generator is calculated by using the geometric sparse coefficient graph of the real image. And the pixel loss function And further calculate the total loss function Update the generator parameters through the back-propagation algorithm to minimize its total loss function

[0139] The formula is:

[0140]

[0141] Where: β GSR and γ GSR They represent the enhanced geometric sparse coefficient graphs Pixel loss weight and adversarial loss weight; GSR (I Low ) and A Low Both represent the geometric sparse coefficient maps of low-light images; represents the computational expectation; G GSR , D GSRRepresent the generator and discriminator respectively; G GSR (GSR(I Low )) and G(A Low ) both represent enhanced geometric sparse coefficient maps generated by the generator according to the geometric sparse coefficient map of the low-light image; A geometric sparse coefficient map representing a real image;

[0142] S2023: Training the discriminator: fix the parameters of the generator; use the geometric sparse coefficient map of the low-light image as the input of the generator, and combine the prior features of the low-light enhancement branch to generate an enhanced geometric sparse coefficient map Enhanced Geometric Sparse Coefficient Graph Perform image reconstruction to obtain the restored image Will restore the image and real images Input the discriminator and calculate the discriminator loss function Update the discriminator parameters through the back-propagation algorithm to minimize the discriminator loss function

[0143] The formula of the discriminator loss function is expressed as:

[0144]

[0145] Where: All represent computational expectations; Represents the discriminator's restoration of the image The output probability of is the probability of a real image; Represents the discriminator for the real image The output probability of the discriminator is that the real image is the probability of a real image;

[0146] Restore the image The calculation formula is expressed as:

[0147]

[0148] Where: Represents the value of the enhanced geometric sparse coefficient map at pixel k; Represents the value of the restored image at pixel k; Represents the value of the real image at pixel k; represents the representation error at pixel k, which is used to represent the residual or uncertainty in the image reconstruction process;

[0149] S2023: Repeat steps S2021 to S2022 to alternately train the generator and the discriminator;

[0150] S2024: Use the completed generator as the trained geometric sparse representation branch.

[0151] 4. Training the Low-light Enhancement Branch

[0152] The low-light enhancement branch is trained based on the generative adversarial network. The specific training steps are as follows:

[0153] S2031: Acquiring low-light image I Low and real images And the geometric sparse representation branch is based on the low-light image I Low Enhanced geometric sparse coefficient map generated by the geometric sparse coefficient map

[0154] S2032: Training the discriminator: fix the parameters of the generator; combine the low-light image and the enhanced geometric sparse coefficient map generated by the geometric sparse representation branch As the input of the generator, generate low-light enhancement feature map The low light enhancement feature map and real images Input the discriminator, which distinguishes whether the input image is a low-light enhanced feature map generated by the generator or a real image, and calculates the discriminator loss function Update the discriminator parameters through the back-propagation algorithm to minimize the discriminator loss function

[0155] The formula of the discriminator loss function is expressed as:

[0156]

[0157] Where: All represent computational expectations; Represents the probability that the discriminator believes that the low-light enhanced feature map generated by the generator is a real image; Represents the probability that the discriminator considers the real image to be a real image;

[0158] S2033: Training the generator: fix the parameters of the discriminator; combine the low-light image and the enhanced geometric sparse coefficient map generated by the geometric sparse representation branch As the input of the generator, generate low-light enhancement feature map The generated low-light enhancement feature map With real image Input into the discriminator to obtain the discriminator's judgment result; according to the discriminator's judgment result and the low-light enhancement feature map With the real feature map The difference between them is used to calculate the generator loss function Perceptual Loss Function Pixel loss function And the pixel loss function after introducing the geometric sparse coefficient map And further calculate the total loss function Update the parameters of the generator through the back-propagation algorithm to minimize its total loss function

[0159] The formula is:

[0160]

[0161] Where: γ I Represent the low light enhancement feature map The pixel loss weight, the pixel loss weight after introducing the geometric sparse coefficient graph, and the adversarial loss weight; represents the computational expectation; G(I Low ) indicates that the generator is based on the low-light image I Low Generated low-light enhancement feature map; GSR(G(I Low )) represents an enhanced geometric sparse coefficient map generated by the root geometric sparse representation branch according to the geometric sparse coefficient map of the low-light image; C is the geometric sparse coefficient graph representing the real image; j W represents the number of feature maps in the jth layer of the generator; j H represents the width of the feature map of the jth layer of the generator; j represents the height of the feature map of the jth layer of the generator; φ J represents the j-th layer output of the generator; G represents the generator; D I represents the discriminator;

[0162] S2034: Repeat steps S2021 to S2022 to alternately train the generator and the discriminator;

[0163] S2035: Use the completed generator as the trained low-light enhancement branch.

[0164] 5. Object Detection Model

[0165] Combination Figure 4As shown in the figure, the backbone network includes a combination of three feature extraction modules connected end to end, a pyramid pooling module and an attention module; the feature extraction module combination includes a convolution module and a C2F module connected end to end; the attention module includes two sub-modules, channel attention and spatial attention; in the channel attention module: the input features are subjected to maximum pooling and average pooling to obtain two different features, which are then sent to a multi-layer perceptron, and then the two feature maps are merged, and the merged feature maps are sent to the spatial attention module, in which the input feature map is subjected to maximum pooling and average pooling along the channel axis to obtain two feature maps, the two feature maps are spliced, and then a convolution layer with a kernel size of 7 is applied to generate a spatial attention map;

[0166] In this embodiment, the C2F module and the pyramid pooling module are both existing network modules, and their network structure diagrams are as follows: Figure 5 and Figure 6 shown.

[0167] The neck module includes an upsampling module, a first cross-layer connection, a first C2F module, a convolution module, a second cross-layer connection, and a second C2F module, which are connected end to end in sequence; the cross-layer connection can splice the feature information between different layers, merge the features of different layers, and prevent information loss;

[0168] The head network generates feature maps for predicting categories and bounding boxes, producing the final detection results.

[0169] 6. Training the Object Detection Model

[0170] The classification loss function when training the target detection model is as follows:

[0171]

[0172] Where: L i represents the loss of the i-th sample; N represents the total number of samples; y i represents the true label of the i-th sample; p i Indicates the probability that the i-th sample predicted by the model belongs to the positive class (target exists);

[0173] Bounding box regression loss function when training object detection models as follows:

[0174]

[0175]

[0176] Where: represents the weighting coefficient; represents IOU loss; W g Indicates the actual frame width; Hg Indicates the real frame height; W i Indicates the width of the overlapping area; H i Indicates the height of the overlapping area; S u Represents the area of ​​the predicted box and the true box; * indicates that W g , H g Separate from the computational graph; x represents the horizontal coordinate of the center point of the prediction box; y represents the vertical coordinate of the center point of the prediction box; x gt Indicates the horizontal coordinate of the center point of the real frame; y gt Indicates the vertical coordinate of the center point of the real box.

[0177] After training is completed, the target detection model can be obtained, and the performance of the target detection model can be tested using the test data set. The evaluation indicators are as follows:

[0178] Accuracy: The ratio of the number of correctly detected targets to the total number of detected targets;

[0179] Precision: The ratio of all positive samples detected as positive samples.

[0180] Recall: The ratio of all detected positive samples to the actual positive samples.

[0181] Where TP represents the number of correctly predicted positive samples;

[0182] FP represents the number of incorrectly predicted positive samples;

[0183] FN represents the number of incorrectly predicted negative samples;

[0184] F1 score: the harmonic mean of precision and recall.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. An intelligent detection method for a locomotive sand spreading device in a low-light environment, characterized in that: include: S1: Acquire an image of a locomotive sand spreading device to be detected; S2: Perform image enhancement on the locomotive sand spreading device image through a low-light enhancement network to obtain an enhanced locomotive sand spreading device image; The processing steps of the low-light enhancement network are as follows: S201: Calculating a geometric sparse coefficient map of a sand spreading device image of a vehicle; S202: Input the geometric sparse coefficient graph into the geometric sparse representation branch, and output the corresponding enhanced geometric sparse coefficient graph; S203: inputting the locomotive sand spreading device image into the low-light enhancement branch, and outputting a corresponding low-light enhancement feature map; S204: Fusing the outputs of the penultimate layer of the low-light enhancement branch and the geometric sparse representation branch to obtain a corresponding enhanced locomotive sand spreading device image; S3: Input the enhanced locomotive sand spreading device image into the trained object detection model and output the prediction result of the sand spreading pipe area; S4: Outputting the prediction result of the sand spreading pipe area as the detection result of the locomotive sand spreading device image.

2. The intelligent detection method for a locomotive sand spreading device in a low light environment according to claim 1, characterized in that: In step S201, the geometric sparse coefficient graph is calculated by the following formula: Where: J(a k ,∈ k ) represents the geometric sparse coefficient of the locomotive sand spreading device image; ω k Represents an image block; a k represents the geometric sparse coefficient of pixel k; k represents pixel; D represents dictionary image; λ represents a k The regularization parameter on ∈ k Represents coefficient; μ k Represents the image block ω k The mean value of D in the Represents the image block ω k In, the mean of; Represents the image block ω k The variance of D in .

3. The intelligent detection method for a locomotive sand spreading device in a low light environment as claimed in claim 1, characterized in that: In step S202, the geometric sparse representation branch includes a first convolution module, a combination of four modules, a second convolution module, a combination of convolution modules, and a third convolution module connected end to end in sequence; The module combination 1 includes a series module, three RRDB modules and a convolution module connected end to end in sequence; The convolutional module combination includes two convolutional modules connected end to end in sequence; The feature map output by the first convolution module and the feature map output by the second convolution module are concatenated and used as the input of the convolution module combination; the output of the convolution module combination is used as the enhanced geometric sparse coefficient map output by the geometric sparse representation branch.

4. The intelligent detection method for a locomotive sand spreading device in a low light environment as claimed in claim 3, characterized in that: In step S203, the low-light enhancement branch includes a first convolution module, a second combination of five modules, a second convolution module, and a convolution module combination connected end to end in sequence; Module combination 2 includes two channel attention dense blocks and three residual dense modules connected end to end in sequence; The convolutional module combination includes two convolutional modules connected end to end in sequence; The feature map output by the first convolution module and the feature map output by the combined convolution module are concatenated as the low-light enhancement feature map output by the low-light enhancement branch; The feature maps output by the first to fourth module combination 2 are used as prior features and correspond one-to-one as the input of the series modules in the first to fourth module combination 1 in the geometric sparse representation branch.

5. The intelligent detection method for a locomotive sand spreading device in a low light environment as claimed in claim 4, characterized in that: In step S204, the outputs of the penultimate layer of the low-light enhancement branch and the geometric sparse representation branch are fused through a fusion network; The fusion network includes a module combination three connected end to end in sequence and a convolution module combination; Module combination three includes a series module and a fusion module connected end to end in sequence; The convolutional module combination includes two convolutional modules connected end to end in sequence; The low-light enhanced feature map and the enhanced geometric sparse coefficient map are used as the input of the series module in module combination three; the feature map output by the convolution module combination is used as the enhanced locomotive sand spreading device image output by the fusion network.

6. The intelligent detection method for a locomotive sand spreading device in a low light environment as claimed in claim 4, characterized in that: The channel attention dense block consists of two dense modules and a channel attention module connected end to end in sequence; The dense module includes four module combinations connected end to end in sequence and a convolution module; the module combination four includes a convolution module connected end to end in sequence and a ReLU activation function layer; The channel attention module includes a global pooling layer, a fully connected layer, a ReLU activation function layer, a fully connected layer, and a Sigmoid activation function layer connected end to end in sequence; The original feature map input to the channel attention dense block is concatenated with the feature map output by the first dense module and used as the input to the second dense module; the feature maps input and output by the second dense module are concatenated and used as the input to the channel attention module; the feature maps input and output by the channel attention module are concatenated and then concatenated with the original feature map as the feature map output by the channel attention dense block.

7. The intelligent detection method for a locomotive sand spreading device in a low light environment as claimed in claim 4, characterized in that: In step S202, the geometric sparse representation branch is trained based on the generative adversarial network. The specific training steps are as follows: S2021: Acquiring low-light images I Low and real images and geometric sparse coefficient maps for low-light and real images; S2022: Train the generator: fix the parameters of the discriminator; use the geometric sparse coefficient map of the low-light image as the input of the generator, and combine the prior features of the low-light enhancement branch to generate an enhanced geometric sparse coefficient map Based on enhanced geometric sparse coefficient graph The loss function of the generator is calculated by using the geometric sparse coefficient graph of the real image. And the pixel loss function And further calculate the total loss function Update the generator parameters through the back-propagation algorithm to minimize its total loss function The formula is: Where: β GSR and γGS R They represent the enhanced geometric sparse coefficient graphs Pixel loss weight and adversarial loss weight; GSR (I Low ) and A Low Both represent the geometric sparse coefficient maps of low-light images; represents the computational expectation; G GSR , D GSR Represent the generator and discriminator respectively; G GSR (GSR(I Low )) and G(A Low ) all represent enhanced geometric sparse coefficient maps generated by the generator according to the geometric sparse coefficient map of the low-light image; A geometric sparse coefficient map representing a real image; S2023: Training the discriminator: fix the parameters of the generator; use the geometric sparse coefficient map of the low-light image as the input of the generator, and combine the prior features of the low-light enhancement branch to generate an enhanced geometric sparse coefficient map Enhanced Geometric Sparse Coefficient Graph Perform image reconstruction to obtain the restored image Will restore the image and real images Input the discriminator and calculate the discriminator loss function Update the discriminator parameters through the back-propagation algorithm to minimize the discriminator loss function The formula of the discriminator loss function is expressed as: Where: All represent computational expectations; Represents the discriminator's restoration of the image The output probability of is the probability of a real image; Represents the discriminator for the real image The output probability of the discriminator is that the real image is the probability of a real image; Restore Image The calculation formula is expressed as: Where: Represents the value of the enhanced geometric sparse coefficient map at pixel k; Represents the value of the restored image at pixel k; Represents the value of the real image at pixel k; represents the representation error at pixel k, which is used to represent the residual or uncertainty in the image reconstruction process; S2023: Repeat steps S2021 to S2022 to alternately train the generator and the discriminator; S2024: Use the completed generator as the trained geometric sparse representation branch.

8. The intelligent detection method for a locomotive sand spreading device in a low light environment as claimed in claim 4, characterized in that: In step S203, the low-light enhancement branch is trained based on the generative adversarial network. The specific training steps are as follows: S2031: Acquiring low-light image I Low and real images And the geometric sparse representation branch is based on the low-light image I Low Enhanced geometric sparse coefficient map generated by the geometric sparse coefficient map S2032: Training the discriminator: fix the parameters of the generator; combine the low-light image and the enhanced geometric sparse coefficient map generated by the geometric sparse representation branch As the input of the generator, generate low-light enhancement feature map The low light enhancement feature map and real images Input the discriminator, which distinguishes whether the input image is a low-light enhanced feature map generated by the generator or a real image, and calculates the discriminator loss function Update the discriminator parameters through the back-propagation algorithm to minimize the discriminator loss function The formula of the discriminator loss function is expressed as: Where: All represent computational expectations; Represents the probability that the discriminator believes that the low-light enhanced feature map generated by the generator is a real image; Represents the probability that the discriminator considers the real image to be a real image; S2033: Training the generator: fix the parameters of the discriminator; combine the low-light image and the enhanced geometric sparse coefficient map generated by the geometric sparse representation branch As the input of the generator, generate low-light enhancement feature map The generated low-light enhancement feature map With real image Input into the discriminator to obtain the discriminator's judgment result; according to the discriminator's judgment result and the low-light enhancement feature map With the real feature map The difference between them is used to calculate the generator loss function Perceptual Loss Function Pixel loss function And the pixel loss function after introducing the geometric sparse coefficient map And further calculate the total loss function Update the parameters of the generator through the back-propagation algorithm to minimize its total loss function The formula is: Where: γ I Represent the low light enhancement feature map The pixel loss weight, the pixel loss weight after introducing the geometric sparse coefficient graph, and the adversarial loss weight; represents the computational expectation; G(I Low ) indicates that the generator is based on the low-light image I Low Generated low-light enhancement feature map; GSR(G(I Low )) represents an enhanced geometric sparse coefficient map generated by the root geometric sparse representation branch according to the geometric sparse coefficient map of the low-light image; C represents the geometric sparse coefficient graph of the real image; j W represents the number of feature maps in the jth layer of the generator; j H represents the width of the feature map of the jth layer of the generator; j represents the height of the feature map of the jth layer of the generator; φ J represents the j-th layer output of the generator; G represents the generator; D U represents the discriminator; S2034: Repeat steps S2021 to S2022 to alternately train the generator and the discriminator; S2035: Use the completed generator as the trained low-light enhancement branch.

9. The intelligent detection method for a locomotive sand spreading device in a low light environment as claimed in claim 1, characterized in that: In step S301, the backbone network includes a combination of three feature extraction modules, a pyramid pooling module, and an attention module connected end to end in sequence; the feature extraction module combination includes a convolution module and a C2F module connected end to end in sequence; the attention module includes two submodules, channel attention and spatial attention; in the channel attention module: the input feature is subjected to maximum pooling and average pooling to obtain two different features, which are then sent to a multi-layer perceptron, and then the two feature maps are merged, and the merged feature maps are sent to the spatial attention module, in which the input feature map is subjected to maximum pooling and average pooling along the channel axis to obtain two feature maps, the two feature maps are spliced, and then a convolution layer with a kernel size of 7 is applied to generate a spatial attention map; In step S302, the neck module includes an upsampling module, a first cross-layer connection, a first C2F module, a convolution module, a second cross-layer connection, and a second C2F module connected end to end in sequence; the cross-layer connection can splice feature information between different layers and merge features of different layers; In step S303, the head network generates a feature map for predicting categories and bounding boxes, and produces the final detection result.

10. The intelligent detection method for a locomotive sand spreading device in a low light environment according to claim 9, characterized in that: The classification loss function when training the target detection model is as follows: Where: L i represents the loss of the i-th sample; N represents the total number of samples; y i represents the true label of the i-th sample; p i Indicates the probability that the i-th sample predicted by the model belongs to the positive class; Bounding box regression loss function when training object detection models as follows: Where: represents the weighting coefficient; represents IOU loss; W g Indicates the real frame width; H g Indicates the real frame height; W i Indicates the width of the overlapping area; H i Indicates the height of the overlapping area; S u Represents the area of ​​the predicted box and the true box; * indicates that W g , H g Separate from the computational graph; x represents the horizontal coordinate of the center point of the prediction box; y represents the vertical coordinate of the center point of the prediction box; x gt Indicates the horizontal coordinate of the center point of the real frame; y gt Indicates the vertical coordinate of the center point of the real box.

Images