Roller detection method, device and system for night environment

Abstract

The application provides a night environment roller detection method, device and roller detection system. The method comprises the following steps: acquiring a first night roller image, correcting the brightness of the first night roller image to obtain a second night roller image, converting the second night roller image into a gray image to obtain a third night roller image; inputting the third night roller image into a detection model for recognition and detection to obtain position information of a target roller; extracting a contour image of the target roller in a position box corresponding to the position information, the position box being a bounding box of the target roller; and determining that the target roller is faulty under the condition that a first predetermined condition and / or a second predetermined condition are met, the first predetermined condition being that the coincidence degree of the contour image and a standard roller contour image is less than a predetermined coincidence degree, and the second predetermined condition being that the distance between the position corresponding to the position information and a predetermined position is greater than a predetermined distance, thereby solving the problem of low fault detection accuracy of the night environment roller detection method.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, and more specifically, to a method, apparatus, computer-readable storage medium, and roller inspection system for nighttime environments. Background Technology

[0002] With the development of machine learning, deep learning, computer computing hardware, and computer vision processing technologies, methods based on target data learning and representation learning have provided a solid technological foundation for the entire field. Currently, with the advancement of electronic information technology, monitoring modern coal mines using wireless sensors, computers, and signal transmission technologies has become relatively mature. Compared to the past, these technologies have significantly reduced coal mine accidents and improved the safety factor of coal mines.

[0003] With the expansion of production scale and the improvement of comprehensive mechanical capabilities, the use of automated inspection monitoring systems in coal mines is increasing. Coal mine conveyor belts experience frequent malfunctions during production, especially belt cracks, tears, misalignment, and slippage, which are extremely destructive. These accidents not only affect coal mine production volume and work progress but also threaten worker safety, resulting in significant economic losses for enterprises. Idler rollers are a crucial component of conveyor belts, directly impacting their performance and service life. The condition of idler rollers is critical to coal transportation safety, making idler roller condition monitoring essential.

[0004] Nighttime detection, under stable artificial lighting conditions, mitigates the impact of varying light intensity and uneven illumination during the day on idler roller recognition, providing technical support for the application of nighttime robots. Research on nighttime object recognition faces the following challenges: low light levels result in lower object recognition rates; nighttime recognition methods are complex, employing multi-stage manual feature extraction, which is unsuitable for large datasets, and the recognition effect needs further improvement; currently, nighttime object recognition methods are inefficient. Furthermore, some studies indicate that the detection methods used are limited by factors such as temperature, equipment cost, and shooting angle, leading to increased detection costs and complexity. Therefore, researching nighttime inspection using coal mine robots to accurately identify and locate idler roller positions has significant theoretical and engineering application value. Summary of the Invention

[0005] The main objective of this application is to provide a method, apparatus, computer-readable storage medium, and idler detection system for idler detection in nighttime environments, so as to at least solve the problem of low fault detection accuracy in existing idler detection methods for nighttime environments.

[0006] To achieve the above objectives, according to one aspect of this application, a method for detecting idler rollers in a nighttime environment is provided, characterized by comprising: acquiring a first nighttime idler roller image, correcting the brightness of the first nighttime idler roller image to obtain a second nighttime idler roller image, converting the second nighttime idler roller image into a grayscale image to obtain a third nighttime idler roller image, wherein the first nighttime idler roller image is an image of a target idler roller in a nighttime environment, and the brightness of the second nighttime idler roller image is greater than the brightness of the first nighttime idler roller image; inputting the third nighttime idler roller image into a detection model for recognition and detection to obtain the position information of the target idler roller, wherein the detection model is a Des-YOLO model trained on a dataset. The training data set, obtained by training a v3 deep convolutional neural network, includes grayscale images of idler rollers with location annotation information. The grayscale images of the training idler rollers are grayscale images corresponding to idler roller images collected in a scene where the ambient brightness is greater than a predetermined ambient brightness. The contour image of the target idler roller is extracted within the location box corresponding to the location information. The location box is the bounding box of the target idler roller in the third nighttime idler roller image. If a first predetermined condition and / or a second predetermined condition are met, it is determined that the target idler roller has a fault. The first predetermined condition is that the overlap between the contour image and the standard idler roller contour image is less than a predetermined overlap. The second predetermined condition is that the distance between the location corresponding to the location information and the predetermined location is greater than a predetermined distance. The predetermined location is the installation location of the target idler roller.

[0007] Optionally, correcting the brightness of the first nighttime idler image to obtain a second nighttime idler image includes: converting the format of the first nighttime idler image to HSV format to obtain a fourth nighttime idler image; determining the brightness information, hue information, and saturation information of the fourth nighttime idler image based on the fourth nighttime idler image; correcting the brightness information using a Gamma function to obtain corrected brightness information, such that the brightness corresponding to the corrected brightness information is greater than the brightness corresponding to the original brightness information; and synthesizing the second nighttime idler image in HSV format based on the corrected brightness information, the hue information, and the saturation information.

[0008] Optionally, converting the format of the first nighttime idler image to HSV format to obtain the fourth nighttime idler image includes: converting the format of the first nighttime idler image to RGB format to obtain the fifth nighttime idler image; and converting the format of the fifth nighttime idler image to HSV format to obtain the fourth nighttime idler image.

[0009] Optionally, converting the second nighttime idler image into a grayscale image to obtain a third nighttime idler image includes: converting the format of the second nighttime idler image into RGB format to obtain a sixth nighttime idler image; and converting the sixth nighttime idler image into a grayscale image to obtain the third nighttime idler image.

[0010] Optionally, the third nighttime idler image is input into a detection model for recognition and detection to obtain the position information of the target idler. This includes: sequentially performing multiple sets of convolution processing, multiple sets of batch normalization processing, and multiple sets of activation function processing on the third nighttime idler image to obtain multiple first feature maps; performing pooling processing on the multiple first feature maps to obtain multiple second feature maps; and performing one densely connected convolution processing and three residual convolution processing on the multiple second feature maps to obtain the position information. The densely connected convolution processing is performed using the convolutional layers of the DenseNet densely connected convolutional network, and the residual convolution processing is performed by modifying the output value of the convolutional layer to the sum of the output value and the input value of the convolutional layer.

[0011] Optionally, the brightness of the first nighttime idler image is corrected to obtain the second nighttime idler image, including: processing the first nighttime idler image using an AE automatic exposure algorithm to obtain the second nighttime idler image.

[0012] Optionally, after determining that the target idler roller has malfunctioned under the condition of satisfying the first predetermined condition and / or the second predetermined condition, the method further includes: issuing an alarm signal, the alarm signal being used to prompt maintenance personnel to perform fault repair on the target idler roller.

[0013] According to another aspect of this application, a roller detection device for nighttime environments is provided, comprising: a processing unit configured to acquire a first nighttime roller image, correct the brightness of the first nighttime roller image to obtain a second nighttime roller image, and convert the second nighttime roller image into a grayscale image to obtain a third nighttime roller image, wherein the first nighttime roller image is an image of a target roller in a nighttime environment, and the brightness of the second nighttime roller image is greater than the brightness of the first nighttime roller image; and a detection unit configured to input the third nighttime roller image into a detection model for recognition and detection to obtain the position information of the target roller, wherein the detection model is a Des-YOLO model trained on a dataset. The training data set, obtained by training a v3 deep convolutional neural network, includes grayscale images of idler rollers with location annotation information. The grayscale images of the training idler rollers are grayscale images corresponding to idler roller images collected in a scene where the ambient brightness is greater than a predetermined ambient brightness. A processing unit is used to extract the contour image of the target idler roller within the location box corresponding to the location information. The location box is the bounding box of the target idler roller in the third nighttime idler roller image. A determination unit is used to determine that the target idler roller has a fault if a first predetermined condition and / or a second predetermined condition are met. The first predetermined condition is that the overlap between the contour image and the standard idler roller contour image is less than a predetermined overlap. The second predetermined condition is that the distance between the location corresponding to the location information and the predetermined location is greater than a predetermined distance. The predetermined location is the installation location of the target idler roller.

[0014] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform any of the methods described.

[0015] According to another aspect of this application, a roller detection system is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing any one of the methods described.

[0016] Applying the technical solution of this application, in the above-mentioned method for detecting idler rollers in a nighttime environment, firstly, a first nighttime idler roller image is acquired, and the brightness of the first nighttime idler roller image is corrected to obtain a second nighttime idler roller image. The second nighttime idler roller image is then converted to a grayscale image to obtain a third nighttime idler roller image. The first nighttime idler roller image is an image of the target idler roller in a nighttime environment, and the brightness of the second nighttime idler roller image is greater than the brightness of the first nighttime idler roller image. Then, the third nighttime idler roller image is input into a detection model for recognition and detection to obtain the position information of the target idler roller. The detection model is a Des-YOLO model trained on a dataset. The training data set, obtained by training a v3 deep convolutional neural network, includes grayscale images of idlers with location annotation information. These grayscale images are grayscale images of idlers acquired in scenarios where the ambient brightness is greater than a predetermined ambient brightness. Then, the contour image of the target idler is extracted within the location bounding box corresponding to the location information. This location bounding box is the bounding box of the target idler in the third nighttime idler image. Finally, if a first predetermined condition and / or a second predetermined condition are met, it is determined that the target idler has malfunctioned. The first predetermined condition is that the overlap between the contour image and the standard idler contour image is less than a predetermined overlap. The second predetermined condition is that the distance between the location corresponding to the location information and a predetermined location is greater than a predetermined distance. The predetermined location is the installation location of the target idler. This method improves the brightness of idler roller images acquired at night by performing brightness correction. The improved images are then input into a detection model for identification and detection to obtain the position information of the target idler roller. Based on the contour image and position information, it can determine whether the target idler roller is faulty. Since the detection model is trained better on images with higher brightness, brightness correction is used to improve the accuracy of identification and detection of idler roller images acquired at night, thereby improving the accuracy of detection and positioning. This enhances the accuracy of fault detection, improves the safety of automated coal mine transportation, saves labor costs, and solves the problem of low fault detection accuracy in existing idler roller detection methods for nighttime environments. Attached Figure Description

[0017] Figure 1 A hardware structure block diagram of a mobile terminal for performing a roller detection method in a nighttime environment, according to an embodiment of this application, is shown.

[0018] Figure 2 A schematic flowchart of a method for detecting idler rollers in a nighttime environment according to an embodiment of this application is shown.

[0019] Figure 3 A schematic diagram of the structure of a detection model provided according to an embodiment of this application is shown;

[0020] Figure 4A schematic flowchart of another method for detecting idler rollers in a nighttime environment according to an embodiment of this application is shown;

[0021] Figure 5 A structural block diagram of a roller detection device for nighttime environments provided according to an embodiment of this application is shown. Detailed Implementation

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] As described in the background section, the fault detection accuracy of existing idler roller detection methods in nighttime environments is low. To solve this problem, embodiments of this application provide an idler roller detection method, apparatus, computer-readable storage medium, and idler roller detection system for nighttime environments.

[0026] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0027] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a nighttime environment idler roller detection method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0028] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the device information display method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0029] This embodiment provides a method for detecting idler rollers in a nighttime environment that operates on a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0030] Figure 2 This is a flowchart of a nighttime environment idler roller detection method according to an embodiment of this application. Figure 2 As shown, the method includes the following steps:

[0031] Step S201: Obtain a first nighttime idler image, correct the brightness of the first nighttime idler image to obtain a second nighttime idler image, convert the second nighttime idler image into a grayscale image to obtain a third nighttime idler image, wherein the first nighttime idler image is an image of the target idler in a nighttime environment, and the brightness of the second nighttime idler image is greater than the brightness of the first nighttime idler image.

[0032] Specifically, the first nighttime idler roller image is processed to achieve automatic exposure and improve the image brightness, so as to facilitate subsequent identification and detection of the outline image and position information of the target idler roller.

[0033] Step S202: Input the above-mentioned third nighttime idler image into the detection model for recognition and detection to obtain the target idler position information. The detection model is obtained by training the Des-YOLO v3 deep convolutional neural network through the training data set. The training data set includes idler grayscale images with position labeling information. The training idler grayscale images are grayscale images corresponding to idler images collected in a scene where the ambient brightness is greater than the predetermined ambient brightness.

[0034] Specifically, during the nighttime idler roll detection test, it was found that the YOLO v3 network had the drawback of missing target detection, and the detection accuracy needed to be improved. DarkNet53, which has higher detection accuracy and speed, was selected as the base network of YOLO v3. By drawing on residual networks and dense connection networks, multiple improvements and experimental verifications were carried out on the basis of YOLO v3 deep convolutional neural network to obtain the Des-YOLO v3 network, which is the detection model mentioned above. The detection model enhances the recognition of small targets and the robustness to occluded and overlapping targets, and effectively reduces the number of convolutional layers of the original network, significantly improving the detection accuracy and speed of idler rolls. Its network parameters are shown in Table 1.

[0035] Step S203: Extract the outline image of the target idler within the location box corresponding to the above location information. The location box is the bounding box of the target idler in the third night idler image.

[0036] Specifically, the image of the third nighttime idler roller is tagged based on the location information to generate a bounding box (bbox), i.e., the aforementioned location box, and the image of the location box is then extracted to obtain the outline image of the target idler roller.

[0037] Step S204: If the first predetermined condition and / or the second predetermined condition are met, it is determined that the target idler roller has malfunctioned. The first predetermined condition is that the overlap between the outline image and the standard idler roller outline image is less than a predetermined overlap. The second predetermined condition is that the distance between the position corresponding to the position information and the predetermined position is greater than a predetermined distance. The predetermined position is the installation position of the target idler roller.

[0038] Specifically, after the detection model outputs the position information of the target idler, the contour image of the target idler is extracted within the position box corresponding to the position information. It is then compared with the contour image and installation position of the standard idler. If the difference is too large, the target idler is determined to be faulty, that is, the overlap between the contour image and the standard idler contour image is less than a predetermined overlap, and / or, the distance between the position corresponding to the position information and the predetermined position is greater than a predetermined distance.

[0039] In the aforementioned method for detecting idler rollers in nighttime environments, firstly, a first nighttime idler roller image is acquired, and the brightness of the first nighttime idler roller image is corrected to obtain a second nighttime idler roller image. The second nighttime idler roller image is then converted to a grayscale image to obtain a third nighttime idler roller image. The first nighttime idler roller image is the image of the target idler roller in a nighttime environment, and the brightness of the second nighttime idler roller image is greater than that of the first nighttime idler roller image. Then, the third nighttime idler roller image is input into a detection model for recognition and detection to obtain the position information of the target idler roller. The detection model is a Des-YOLO model trained on a dataset. The training data set, obtained by training a v3 deep convolutional neural network, includes grayscale images of idlers with location annotation information. These grayscale images are grayscale images of idlers acquired in scenarios where the ambient brightness is greater than a predetermined ambient brightness. Then, the contour image of the target idler is extracted within the location bounding box corresponding to the location information. This location bounding box is the bounding box of the target idler in the third nighttime idler image. Finally, if a first predetermined condition and / or a second predetermined condition are met, it is determined that the target idler has malfunctioned. The first predetermined condition is that the overlap between the contour image and the standard idler contour image is less than a predetermined overlap. The second predetermined condition is that the distance between the location corresponding to the location information and a predetermined location is greater than a predetermined distance. The predetermined location is the installation location of the target idler. This method improves the brightness of idler roller images acquired at night by performing brightness correction. The improved images are then input into a detection model for identification and detection to obtain the position information of the target idler roller. Based on the contour image and position information, it can determine whether the target idler roller is faulty. Since the detection model is trained better on images with higher brightness, brightness correction is used to improve the accuracy of identification and detection of idler roller images acquired at night, thereby improving the accuracy of detection and positioning. This enhances the accuracy of fault detection, improves the safety of automated coal mine transportation, saves labor costs, and solves the problem of low fault detection accuracy in existing idler roller detection methods for nighttime environments.

[0040] Table 1

[0041]

[0042]

[0043] To reduce the difficulty of fault detection, in one optional implementation, step S201 includes:

[0044] Step S2011: Convert the format of the first nighttime idler roller image to HSV format to obtain the fourth nighttime idler roller image;

[0045] Step S2012: Determine the brightness information, hue information, and saturation information of the fourth nighttime idler image based on the aforementioned fourth nighttime idler image;

[0046] Step S2013: The above brightness information is corrected using the Gamma function to obtain corrected brightness information, such that the brightness corresponding to the corrected brightness information is greater than the brightness corresponding to the above brightness information.

[0047] Step S2014: The second nighttime roller image in HSV format is synthesized based on the above-mentioned corrected brightness information, the above-mentioned hue information and the above-mentioned saturation information.

[0048] Specifically, the first nighttime idler image is converted into the HSV color space, and the image's luminance information V, hue information H, and saturation information S are obtained. The luminance information is processed by multi-scale Gaussian convolution to obtain the illumination component I(x, y). The 2D Gamma function is calculated, and after parameter adjustment, the luminance is corrected to obtain the corrected luminance component V', making the luminance corresponding to the corrected luminance information greater than the luminance corresponding to the above luminance information. The corrected luminance component is combined with the hue and saturation S information to re-synthesize the image, obtaining the second nighttime idler image in HSV format. The luminance correction improves the brightness of the idler image, making it easier to identify and detect the outline and position of the idler image, and reducing the difficulty of fault detection.

[0049] In an optional implementation, to convert the image format to HSV format, step S2011 above includes:

[0050] Step S20111: Convert the format of the first nighttime idler roller image to RGB format to obtain the fifth nighttime idler roller image;

[0051] Step S20112: Convert the format of the fifth nighttime idler roller image to HSV format to obtain the fourth nighttime idler roller image.

[0052] Specifically, since the RGB color space and the HSV color space can be converted to each other, the first night roller image is determined to be an RGB image. If it is a GRB image, it is directly converted to the HSV color space. If not, it is converted to an RGB image and then converted to the HSV color space. This process converts the first night roller image into the fourth night roller image in HSV format. The HSV format image includes brightness information, which is convenient for brightness correction.

[0053] To reduce the difficulty of fault detection, in one optional implementation, step S201 includes:

[0054] Step S2015: Convert the format of the second nighttime idler roller image to RGB format to obtain the sixth nighttime idler roller image;

[0055] Step S2016: Convert the above-mentioned sixth nighttime idler roller image into a grayscale image to obtain the above-mentioned third nighttime idler roller image.

[0056] Specifically, the HSV format of the second nighttime idler roller image is converted to RGB format, and then to grayscale image. Grayscale image makes it easier to extract contour features and position features, which facilitates the identification and detection of the contour and position of the idler roller image and reduces the difficulty of fault detection.

[0057] To reduce the difficulty of fault detection, in one optional implementation, step S202 includes:

[0058] Step S2021: Perform multiple sets of convolution processing, multiple sets of batch normalization processing, and multiple sets of activation function processing on the above-mentioned third nighttime roller image in sequence to obtain multiple first feature maps;

[0059] Step S2022: Pooling is performed on the multiple first feature maps to obtain multiple second feature maps;

[0060] Step S2023: Perform one densely connected convolution and three residual convolutions on the multiple second feature maps to obtain the position information. The densely connected convolution is performed using the convolutional layers of the DenseNet network. The residual convolution is performed by modifying the output value of the convolutional layer to the sum of the output value and the input value of the convolutional layer.

[0061] Specifically, the Des-YOLO v3 network extracts features from the input image through one densely connected convolution and three residual convolutions. Compared to the original framework DarkNet53, Des-YOLO v3's shallow network uses densely connected convolutional blocks, allowing shallow features to be transmitted to deeper convolutions more effectively and quickly. This enables the reuse and fusion of features across multiple network layers, further improving the overall information transfer and gradient transfer efficiency of the network. This is beneficial for combining upsampling with shallow features in regression prediction for target detection. In the latter part of the feature extraction network, Des-YOLO v3 retains the residual network structure, deepening the feature extraction network structure with three sets of residual blocks to improve the model's ability to select and extract deep image features. Finally, Des-YOLO v3 uses a scale pyramid structure similar to FPN networks, performing two upsampling operations and concatenating them with feature maps of the same size from the upper layers of the network for secondary regression prediction, achieving multi-scale detection of targets of different sizes. Furthermore, the Des-YOLO v3 network reduces the number of convolutional layers, thereby decreasing model size and computational cost, while maintaining high prediction accuracy. The Des-YOLO v3 network architecture is as follows: Figure 3As shown, convolution is first performed through a Convolutional layer (Conv). After processing with an activation function, it enters a Max pooling layer for pooling to extract feature maps. These feature maps are then subjected to one densely connected convolution and three residual convolutions to obtain the aforementioned location information. CBR is a basic component of the Des-YOLO v3 network, including convolution, batch normalization (BN), and ReLU (Leaky ReLU) operations. Dense is a densely connected block containing two convolution operations. This module concatenates the feature maps before and after the convolution operations, enabling the reuse and fusion of multi-layer features. Res Block is a residual block, where Res_unit*n represents n residual units. Using residual structures allows for deeper network structures, obtaining high-dimensional features to improve prediction accuracy.

[0062] To improve the accuracy of fault detection, in one optional implementation, step S201 includes:

[0063] Step S2017: The first nighttime idler image is processed using the AE automatic exposure algorithm to obtain the second nighttime idler image.

[0064] Specifically, the AE automatic exposure algorithm is used to improve the brightness of the idler roller image, enhance the performance of idler roller target detection at night, and thus improve the accuracy of fault detection.

[0065] To avoid missing faults, in an optional implementation, after step S203, the method further includes:

[0066] Step S301: Issue an alarm signal. The alarm signal is used to prompt maintenance personnel to perform fault repair on the target idler roller.

[0067] Specifically, after a fault is detected in the target idler roller, an alarm signal is issued to remind maintenance personnel to repair the fault and avoid missing any faults that could affect production progress.

[0068] To enable those skilled in the art to better understand the technical solution of this application, the implementation process of the roller detection method for nighttime environments of this application will be described in detail below with reference to specific embodiments.

[0069] This embodiment relates to a specific method for detecting idler rollers in a nighttime environment, such as... Figure 4 As shown, it includes the following steps:

[0070] Step S1: Determine whether the input image is an RGB image. If it is a GRB image, convert it directly to the HSV color space. If not, convert it to an RGB image and then to the HSV color space.

[0071] Step S2: Obtain the luminance (V) information, hue (H) and saturation (S) information from the HSV color space. Perform multi-scale Gaussian convolution on the luminance information to obtain the illumination component I(x, y). Calculate the 2D Gamma function, adjust the parameters, and then correct the luminance to obtain the corrected luminance component. Combine the hue and saturation (S) information to re-synthesize the image. Then, convert the HSV color space back to the RGB color space and then back to a grayscale image.

[0072] Step S3: Input the obtained grayscale image into the improved Des-YOLOv3 network for recognition and detection to obtain the position information of the idler roller.

[0073] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0074] This application also provides a roller detection device for nighttime environments. It should be noted that the roller detection device for nighttime environments in this application can be used to execute the roller detection method for nighttime environments provided in this application. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0075] The following describes the idler roller detection device for nighttime environments provided in the embodiments of this application.

[0076] Figure 5 This is a schematic diagram of a roller detection device for nighttime environments according to an embodiment of this application. Figure 5 As shown, the device includes:

[0077] The first processing unit 10 is used to acquire a first nighttime idler roller image, correct the brightness of the first nighttime idler roller image to obtain a second nighttime idler roller image, convert the second nighttime idler roller image into a grayscale image to obtain a third nighttime idler roller image, wherein the first nighttime idler roller image is an image of the target idler roller in a nighttime environment, and the brightness of the second nighttime idler roller image is greater than the brightness of the first nighttime idler roller image.

[0078] Specifically, the first nighttime idler roller image is processed to achieve automatic exposure and improve the image brightness, so as to facilitate subsequent identification and detection of the outline image and position information of the target idler roller.

[0079] The detection unit 20 is used to input the above-mentioned third nighttime idler image into the detection model for recognition and detection, and obtain the position information of the target idler. The detection model is obtained by training the Des-YOLO v3 deep convolutional neural network through the training data set. The training data set includes idler grayscale images with position annotation information. The training idler grayscale images are grayscale images corresponding to idler images collected in a scene where the ambient brightness is greater than the predetermined ambient brightness.

[0080] Specifically, during the nighttime idler roll detection test, it was found that the YOLO v3 network had the drawback of missing target detection, and the detection accuracy needed to be improved. DarkNet53, which has higher detection accuracy and speed, was selected as the base network of YOLO v3. By drawing on residual networks and dense connection networks, multiple improvements and experimental verifications were carried out on the basis of YOLO v3 deep convolutional neural network to obtain the Des-YOLO v3 network, which is the detection model mentioned above. The detection model enhances the recognition of small targets and the robustness to occluded and overlapping targets, and effectively reduces the number of convolutional layers of the original network, significantly improving the detection accuracy and speed of idler rolls. Its network parameters are shown in Table 1.

[0081] The second processing unit 30 extracts the outline image of the target idler within the position box corresponding to the above position information. The position box is the boundary box of the target idler in the third night idler image.

[0082] Specifically, the image of the third nighttime idler roller is tagged based on the location information to generate a bounding box (bbox), i.e., the aforementioned location box, and the image of the location box is then extracted to obtain the outline image of the target idler roller.

[0083] The determining unit 40 is used to determine that the target idler has malfunctioned when a first predetermined condition and / or a second predetermined condition are met. The first predetermined condition is that the overlap between the outline image and the standard idler outline image is less than a predetermined overlap. The second predetermined condition is that the distance between the position corresponding to the position information and the predetermined position is greater than a predetermined distance. The predetermined position is the installation position of the target idler.

[0084] Specifically, after the detection model outputs the position information of the target idler, the contour image of the target idler is extracted within the position box corresponding to the position information. It is then compared with the contour image and installation position of the standard idler. If the difference is too large, the target idler is determined to be faulty, that is, the overlap between the contour image and the standard idler contour image is less than a predetermined overlap, and / or, the distance between the position corresponding to the position information and the predetermined position is greater than a predetermined distance.

[0085] In the aforementioned nighttime idler detection device, a first processing unit acquires a first nighttime idler image and corrects the brightness of the first nighttime idler image to obtain a second nighttime idler image. The second nighttime idler image is then converted to a grayscale image to obtain a third nighttime idler image. The first nighttime idler image is an image of the target idler in a nighttime environment, and the brightness of the second nighttime idler image is greater than that of the first nighttime idler image. A detection unit inputs the third nighttime idler image into a detection model for recognition and detection to obtain the position information of the target idler. The detection model is a Des-YOLO dataset trained on the target idler. The training data set, obtained by training a v3 deep convolutional neural network, includes grayscale images of idlers with location annotation information. These grayscale images are grayscale images of idlers acquired in scenarios where the ambient brightness is greater than a predetermined ambient brightness. The second processing unit extracts the contour image of the target idler within the location box corresponding to the location information. The location box is the bounding box of the target idler in the third nighttime idler image. The determining unit determines that the target idler has malfunctioned if the first predetermined condition and / or the second predetermined condition are met. The first predetermined condition is that the overlap between the contour image and the standard idler contour image is less than a predetermined overlap. The second predetermined condition is that the distance between the location corresponding to the location information and the predetermined location is greater than a predetermined distance. The predetermined location is the installation location of the target idler. This device performs brightness correction on idler roller images acquired at night to enhance image brightness. The enhanced image is then input into a detection model for identification and detection to obtain the position information of the target idler roller. Based on the contour image and position information, it determines whether the target idler roller is faulty. Since the detection model is trained better using images with higher brightness, brightness correction enhances the nighttime idler roller images to ensure the accuracy of identification and detection, improving the detection accuracy and positioning precision. This improves the accuracy of fault detection, enhances the safety of automated coal mine transportation, saves labor costs, and solves the problem of low fault detection accuracy in existing idler roller detection methods for nighttime environments.

[0086] To reduce the difficulty of fault detection, in one optional implementation, the first processing unit includes:

[0087] The first processing module is used to convert the format of the first nighttime idler roller image to HSV format to obtain the fourth nighttime idler roller image;

[0088] The second processing module is used to determine the brightness information, hue information and saturation information of the fourth night idler image based on the fourth night idler image.

[0089] The third processing module is used to correct the above brightness information using the Gamma function to obtain corrected brightness information, so that the brightness corresponding to the corrected brightness information is greater than the brightness corresponding to the above brightness information.

[0090] The fourth processing module is used to synthesize the second nighttime idler roller image in HSV format based on the above-mentioned corrected brightness information, the above-mentioned hue information and the above-mentioned saturation information.

[0091] Specifically, the first nighttime idler image is converted into the HSV color space, and the image's luminance information V, hue information H, and saturation information S are obtained. The luminance information is processed by multi-scale Gaussian convolution to obtain the illumination component I(x, y). The 2D Gamma function is calculated, and after parameter adjustment, the luminance is corrected to obtain the corrected luminance component V', making the luminance corresponding to the corrected luminance information greater than the luminance corresponding to the above luminance information. The corrected luminance component is combined with the hue and saturation S information to re-synthesize the image, obtaining the second nighttime idler image in HSV format. The luminance correction improves the brightness of the idler image, making it easier to identify and detect the outline and position of the idler image, and reducing the difficulty of fault detection.

[0092] In an optional implementation, to convert the image format to HSV format, the first processing module described above includes:

[0093] The first processing submodule is used to convert the format of the first nighttime idler roller image to RGB format to obtain the fifth nighttime idler roller image;

[0094] The second processing submodule is used to convert the format of the fifth nighttime idler roller image to HSV format to obtain the fourth nighttime idler roller image.

[0095] Specifically, since the RGB color space and the HSV color space can be converted to each other, the first night roller image is determined to be an RGB image. If it is a GRB image, it is directly converted to the HSV color space. If not, it is converted to an RGB image and then converted to the HSV color space. This process converts the first night roller image into the fourth night roller image in HSV format. The HSV format image includes brightness information, which is convenient for brightness correction.

[0096] To reduce the difficulty of fault detection, in one optional implementation, the first processing unit further includes:

[0097] The fifth processing module is used to convert the format of the second nighttime idler roller image to RGB format to obtain the sixth nighttime idler roller image;

[0098] The sixth processing module is used to convert the aforementioned sixth nighttime idler roller image into a grayscale image to obtain the aforementioned third nighttime idler roller image.

[0099] Specifically, the HSV format of the second nighttime idler roller image is converted to RGB format, and then to grayscale image. Grayscale image makes it easier to extract contour features and position features, which facilitates the identification and detection of the contour and position of the idler roller image and reduces the difficulty of fault detection.

[0100] To reduce the difficulty of fault detection, in one optional implementation, the detection unit includes:

[0101] The seventh processing module is used to sequentially perform multiple sets of convolution processing, multiple sets of batch normalization processing, and multiple sets of activation function processing on the above-mentioned third nighttime roller image to obtain multiple first feature maps;

[0102] The eighth processing module is used to perform pooling processing on the first feature map to obtain the second feature map;

[0103] The ninth processing module is used to perform one densely connected convolution and three residual convolutions on the second feature map to obtain the position information. The densely connected convolution is performed using the convolutional layers of the DenseNet densely connected convolutional network. The residual convolution is performed by modifying the output value of the convolutional layer to the sum of the output value and the input value of the convolutional layer.

[0104] Specifically, the Des-YOLO v3 network extracts features from the input image through one densely connected convolution and three residual convolutions. Compared to the original framework DarkNet53, Des-YOLO v3's shallow network uses densely connected convolutional blocks, allowing shallow features to be transmitted to deeper convolutions more effectively and quickly. This enables the reuse and fusion of features across multiple network layers, further improving the overall information transfer and gradient transfer efficiency of the network. This is beneficial for combining upsampling with shallow features in regression prediction for target detection. In the latter part of the feature extraction network, Des-YOLO v3 retains the residual network structure, deepening the feature extraction network structure with three sets of residual blocks to improve the model's ability to select and extract deep image features. Finally, Des-YOLO v3 uses a scale pyramid structure similar to FPN networks, performing two upsampling operations and concatenating them with feature maps of the same size from the upper layers of the network for secondary regression prediction, achieving multi-scale detection of targets of different sizes. Furthermore, the Des-YOLO v3 network reduces the number of convolutional layers, thereby decreasing model size and computational cost, while maintaining high prediction accuracy. The Des-YOLO v3 network architecture is as follows: Figure 3As shown, convolution is first performed through a Convolutional layer (Conv). After processing with an activation function, it enters a Max pooling layer for pooling to extract feature maps. These feature maps are then subjected to one densely connected convolution and three residual convolutions to obtain the aforementioned location information. CBR is a basic component of the Des-YOLO v3 network, including convolution, batch normalization (BN), and ReLU (Leaky ReLU) operations. Dense is a densely connected block containing two convolution operations. This module concatenates the feature maps before and after the convolution operations, enabling the reuse and fusion of multi-layer features. Res Block is a residual block, where Res_unit*n represents n residual units. Using residual structures allows for deeper network structures, obtaining high-dimensional features to improve prediction accuracy.

[0105] To improve the accuracy of fault detection, in one optional implementation, the processing unit includes:

[0106] The tenth processing module is used to process the first nighttime idler roller image using the AE automatic exposure algorithm to obtain the second nighttime idler roller image.

[0107] Specifically, the AE automatic exposure algorithm is used to improve the brightness of the idler roller image, enhance the performance of idler roller target detection at night, and thus improve the accuracy of fault detection.

[0108] To avoid missing faults, in an optional embodiment, the above-mentioned device further includes:

[0109] The sending unit is used to issue an alarm signal after determining that the target idler has malfunctioned under the condition of satisfying the first predetermined condition and / or the second predetermined condition. The alarm signal is used to prompt maintenance personnel to carry out fault repair on the target idler.

[0110] Specifically, after a fault is detected in the target idler roller, an alarm signal is issued to remind maintenance personnel to repair the fault and avoid missing any faults that could affect production progress.

[0111] The aforementioned idler roller detection device for nighttime environments includes a processor and a memory. The processing unit, detection unit, and determination unit are all stored as program units in the memory, and the processor executes these program units to achieve the corresponding functions. All of the above modules reside in the same processor; alternatively, the modules may be located in different processors in any combination.

[0112] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the low fault detection accuracy problem of existing roller detection methods in nighttime environments.

[0113] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0114] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the aforementioned nighttime environment idler roller detection method.

[0115] Specifically, the above-mentioned methods for detecting idler rollers in nighttime environments include:

[0116] Step S201: Obtain a first nighttime idler image, correct the brightness of the first nighttime idler image to obtain a second nighttime idler image, convert the second nighttime idler image into a grayscale image to obtain a third nighttime idler image, wherein the first nighttime idler image is an image of the target idler in a nighttime environment, and the brightness of the second nighttime idler image is greater than the brightness of the first nighttime idler image.

[0117] Specifically, the first nighttime idler roller image is processed to achieve automatic exposure and improve the image brightness, so as to facilitate subsequent identification and detection of the outline image and position information of the target idler roller.

[0118] Step S202: Input the above-mentioned third nighttime idler image into the detection model for recognition and detection to obtain the position information of the target idler. The detection model is obtained by training the Des-YOLO v3 deep convolutional neural network through the training data set. The training data set includes idler grayscale images with position annotation information. The training idler grayscale images are grayscale images corresponding to idler images collected in a scene where the ambient brightness is greater than the predetermined ambient brightness.

[0119] Specifically, during the nighttime idler roll detection test, it was found that the YOLO v3 network had the drawback of missing target detection, and the detection accuracy needed to be improved. DarkNet53, which has higher detection accuracy and speed, was selected as the base network of YOLO v3. By drawing on residual networks and dense connection networks, multiple improvements and experimental verifications were carried out on the basis of YOLO v3 deep convolutional neural network to obtain the Des-YOLO v3 network, which is the detection model mentioned above. The detection model enhances the recognition of small targets and the robustness to occluded and overlapping targets, and effectively reduces the number of convolutional layers of the original network, significantly improving the detection accuracy and speed of idler rolls. Its network parameters are shown in Table 1.

[0120] Step S203: Extract the outline image of the target idler within the location box corresponding to the above location information. The location box is the bounding box of the target idler in the third night idler image.

[0121] Specifically, the image of the third nighttime idler roller is tagged based on the location information to generate a bounding box (bbox), i.e., the aforementioned location box, and the image of the location box is then extracted to obtain the outline image of the target idler roller.

[0122] Step S204: If the first predetermined condition and / or the second predetermined condition are met, it is determined that the target idler roller has malfunctioned. The first predetermined condition is that the overlap between the outline image and the standard idler roller outline image is less than a predetermined overlap. The second predetermined condition is that the distance between the position corresponding to the position information and the predetermined position is greater than a predetermined distance. The predetermined position is the installation position of the target idler roller.

[0123] Specifically, after the detection model outputs the contour image and position information of the target idler, it compares it with the contour image and installation position of the standard idler. If the difference is too large, the target idler is determined to be faulty, that is, the overlap between the contour image and the standard idler contour image is less than the predetermined overlap, and / or, the distance between the position corresponding to the position information and the predetermined position is greater than the predetermined distance.

[0124] This invention provides a processor for running a program, wherein the program executes the roller detection method for nighttime environments.

[0125] Specifically, the above-mentioned methods for detecting idler rollers in nighttime environments include:

[0126] Step S201: Obtain a first nighttime idler image, correct the brightness of the first nighttime idler image to obtain a second nighttime idler image, convert the second nighttime idler image into a grayscale image to obtain a third nighttime idler image, wherein the first nighttime idler image is an image of the target idler in a nighttime environment, and the brightness of the second nighttime idler image is greater than the brightness of the first nighttime idler image.

[0127] Specifically, the first nighttime idler roller image is processed to achieve automatic exposure and improve the image brightness, so as to facilitate subsequent identification and detection of the outline image and position information of the target idler roller.

[0128] Step S202: Input the above-mentioned third nighttime idler image into the detection model for recognition and detection to obtain the position information of the target idler. The detection model is obtained by training the Des-YOLO v3 deep convolutional neural network through the training data set. The training data set includes idler grayscale images with position annotation information. The training idler grayscale images are grayscale images corresponding to idler images collected in a scene where the ambient brightness is greater than the predetermined ambient brightness.

[0129] Specifically, during the nighttime idler roll detection test, it was found that the YOLO v3 network had the drawback of missing target detection, and the detection accuracy needed to be improved. DarkNet53, which has higher detection accuracy and speed, was selected as the base network of YOLO v3. By drawing on residual networks and dense connection networks, multiple improvements and experimental verifications were carried out on the basis of YOLO v3 deep convolutional neural network to obtain the Des-YOLO v3 network, which is the detection model mentioned above. The detection model enhances the recognition of small targets and the robustness to occluded and overlapping targets, and effectively reduces the number of convolutional layers of the original network, significantly improving the detection accuracy and speed of idler rolls. Its network parameters are shown in Table 1.

[0130] Step S203: Extract the outline image of the target idler within the location box corresponding to the above location information. The location box is the bounding box of the target idler in the third night idler image.

[0131] Specifically, the image of the third nighttime idler roller is tagged based on the location information to generate a bounding box (bbox), i.e., the aforementioned location box, and the image of the location box is then extracted to obtain the outline image of the target idler roller.

[0132] Step S204: If the first predetermined condition and / or the second predetermined condition are met, it is determined that the target idler roller has malfunctioned. The first predetermined condition is that the overlap between the outline image and the standard idler roller outline image is less than a predetermined overlap. The second predetermined condition is that the distance between the position corresponding to the position information and the predetermined position is greater than a predetermined distance. The predetermined position is the installation position of the target idler roller.

[0133] Specifically, after the detection model outputs the position information of the target idler, the contour image of the target idler is extracted within the position box corresponding to the position information. It is then compared with the contour image and installation position of the standard idler. If the difference is too large, the target idler is determined to be faulty, that is, the overlap between the contour image and the standard idler contour image is less than a predetermined overlap, and / or, the distance between the position corresponding to the position information and the predetermined position is greater than a predetermined distance.

[0134] This invention provides a roller detection system, the device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs at least the following steps:

[0135] Step S201: Obtain a first nighttime idler image, correct the brightness of the first nighttime idler image to obtain a second nighttime idler image, convert the second nighttime idler image into a grayscale image to obtain a third nighttime idler image, wherein the first nighttime idler image is an image of the target idler in a nighttime environment, and the brightness of the second nighttime idler image is greater than the brightness of the first nighttime idler image.

[0136] Specifically, the first nighttime idler roller image is processed to achieve automatic exposure and improve the image brightness, so as to facilitate subsequent identification and detection of the outline image and position information of the target idler roller.

[0137] Step S202: Input the above-mentioned third nighttime idler image into the detection model for recognition and detection to obtain the position information of the target idler. The detection model is obtained by training the Des-YOLO v3 deep convolutional neural network through the training data set. The training data set includes idler grayscale images with position annotation information. The training idler grayscale images are grayscale images corresponding to idler images collected in a scene where the ambient brightness is greater than the predetermined ambient brightness.

[0138] Specifically, during the nighttime idler roll detection test, it was found that the YOLO v3 network had the drawback of missing target detection, and the detection accuracy needed to be improved. DarkNet53, which has higher detection accuracy and speed, was selected as the base network of YOLO v3. By drawing on residual networks and dense connection networks, multiple improvements and experimental verifications were carried out on the basis of YOLO v3 deep convolutional neural network to obtain the Des-YOLO v3 network, which is the detection model mentioned above. The detection model enhances the recognition of small targets and the robustness to occluded and overlapping targets, and effectively reduces the number of convolutional layers of the original network, significantly improving the detection accuracy and speed of idler rolls. Its network parameters are shown in Table 1.

[0139] Step S203: Extract the outline image of the target idler within the location box corresponding to the above location information. The location box is the bounding box of the target idler in the third night idler image.

[0140] Specifically, the image of the third nighttime idler roller is tagged based on the location information to generate a bounding box (bbox), i.e., the aforementioned location box, and the image of the location box is then extracted to obtain the outline image of the target idler roller.

[0141] Step S204: If the first predetermined condition and / or the second predetermined condition are met, it is determined that the target idler roller has malfunctioned. The first predetermined condition is that the overlap between the outline image and the standard idler roller outline image is less than a predetermined overlap. The second predetermined condition is that the distance between the position corresponding to the position information and the predetermined position is greater than a predetermined distance. The predetermined position is the installation position of the target idler roller.

[0142] Specifically, after the detection model outputs the position information of the target idler, the contour image of the target idler is extracted within the position box corresponding to the position information. It is then compared with the contour image and installation position of the standard idler. If the difference is too large, the target idler is determined to be faulty, that is, the overlap between the contour image and the standard idler contour image is less than a predetermined overlap, and / or, the distance between the position corresponding to the position information and the predetermined position is greater than a predetermined distance.

[0143] Optionally, step S201 includes: step S2011, converting the format of the first nighttime idler image to HSV format to obtain a fourth nighttime idler image; step S2012, determining the brightness information, hue information, and saturation information of the fourth nighttime idler image based on the fourth nighttime idler image; step S2013, correcting the brightness information using the Gamma function to obtain corrected brightness information, such that the brightness corresponding to the corrected brightness information is greater than the brightness corresponding to the brightness information; and step S2014, synthesizing the second nighttime idler image in HSV format based on the corrected brightness information, the hue information, and the saturation information.

[0144] Specifically, the first nighttime idler image is converted into the HSV color space, and the image's luminance information V, hue information H, and saturation information S are obtained. The luminance information is processed by multi-scale Gaussian convolution to obtain the illumination component I(x, y). The 2D Gamma function is calculated, and after parameter adjustment, the luminance is corrected to obtain the corrected luminance component V', making the luminance corresponding to the corrected luminance information greater than the luminance corresponding to the above luminance information. The corrected luminance component is combined with the hue and saturation S information to re-synthesize the image, obtaining the second nighttime idler image in HSV format. The luminance correction improves the brightness of the idler image, making it easier to identify and detect the outline and position of the idler image, and reducing the difficulty of fault detection.

[0145] Optionally, step S2011 includes: step S20111, converting the format of the first night roller image to RGB format to obtain the fifth night roller image; step S20112, converting the format of the fifth night roller image to HSV format to obtain the fourth night roller image.

[0146] Specifically, since the RGB color space and the HSV color space can be converted to each other, the first night roller image is determined to be an RGB image. If it is a GRB image, it is directly converted to the HSV color space. If not, it is converted to an RGB image and then converted to the HSV color space. This process converts the first night roller image into the fourth night roller image in HSV format. The HSV format image includes brightness information, which is convenient for brightness correction.

[0147] Optionally, step S201 includes: step S2015, converting the format of the second nighttime idler image to RGB format to obtain the sixth nighttime idler image; step S2016, converting the sixth nighttime idler image to grayscale image to obtain the third nighttime idler image.

[0148] Specifically, the HSV format of the second nighttime idler roller image is converted to RGB format, and then to grayscale image. Grayscale image makes it easier to extract contour features and position features, which facilitates the identification and detection of the contour and position of the idler roller image and reduces the difficulty of fault detection.

[0149] Optionally, step S202 includes: step S2021, sequentially performing multiple sets of convolution processing, multiple sets of batch normalization processing, and multiple sets of activation function processing on the third nighttime roller image to obtain multiple first feature maps; step S2022, performing pooling processing on the multiple first feature maps to obtain multiple second feature maps; step S2023, performing one densely connected convolution processing and three residual convolution processing on the multiple second feature maps to obtain the position information, wherein the densely connected convolution processing is performed using the convolutional layers of the DenseNet densely connected convolutional network, and the residual convolution processing is performed by modifying the output value of the convolutional layer to the sum of the output value and the input value of the convolutional layer.

[0150] Specifically, the Des-YOLO v3 network extracts features from the input image through one densely connected convolution and three residual convolutions. Compared to the original framework DarkNet53, Des-YOLO v3's shallow network uses densely connected convolutional blocks, allowing shallow features to be transmitted to deeper convolutions more effectively and quickly. This enables the reuse and fusion of features across multiple network layers, further improving the overall information transfer and gradient transfer efficiency of the network. This is beneficial for combining upsampling with shallow features in regression prediction for target detection. In the latter part of the feature extraction network, Des-YOLO v3 retains the residual network structure, deepening the feature extraction network structure with three sets of residual blocks to improve the model's ability to select and extract deep image features. Finally, Des-YOLO v3 uses a scale pyramid structure similar to FPN networks, performing two upsampling operations and concatenating them with feature maps of the same size from the upper layers of the network for secondary regression prediction, achieving multi-scale detection of targets of different sizes. Furthermore, the Des-YOLO v3 network reduces the number of convolutional layers, thereby decreasing model size and computational cost, while maintaining high prediction accuracy. The Des-YOLO v3 network architecture is as follows: Figure 3 As shown, convolution is first performed through a Convolutional layer (Conv). After processing with an activation function, it enters a Max pooling layer for pooling to extract feature maps. These feature maps are then subjected to one densely connected convolution and three residual convolutions to obtain the aforementioned location information. CBR is a basic component of the Des-YOLO v3 network, including convolution, batch normalization (BN), and ReLU (Leaky ReLU) operations. Dense is a densely connected block containing two convolution operations. This module concatenates the feature maps before and after the convolution operations, enabling the reuse and fusion of multi-layer features. Res Block is a residual block, where Res_unit*n represents n residual units. Using residual structures allows for deeper network structures, obtaining high-dimensional features to improve prediction accuracy.

[0151] Optionally, step S201 includes: step S2017, processing the first nighttime idler image using an AE automatic exposure algorithm to obtain the second nighttime idler image.

[0152] Specifically, the AE automatic exposure algorithm is used to improve the brightness of the idler roller image, enhance the performance of idler roller target detection at night, and thus improve the accuracy of fault detection.

[0153] Optionally, after step S203, the method further includes step S301, issuing an alarm signal, which is used to prompt maintenance personnel to perform fault repair on the target idler roller.

[0154] Specifically, after a fault is detected in the target idler roller, an alarm signal is issued to remind maintenance personnel to repair the fault and avoid missing any faults that could affect production progress.

[0155] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having at least the following method steps:

[0156] Step S201: Obtain a first nighttime idler image, correct the brightness of the first nighttime idler image to obtain a second nighttime idler image, convert the second nighttime idler image into a grayscale image to obtain a third nighttime idler image, wherein the first nighttime idler image is an image of the target idler in a nighttime environment, and the brightness of the second nighttime idler image is greater than the brightness of the first nighttime idler image.

[0157] Specifically, the first nighttime idler roller image is processed to achieve automatic exposure and improve the image brightness, so as to facilitate subsequent identification and detection of the outline image and position information of the target idler roller.

[0158] Step S202: Input the above-mentioned third nighttime idler image into the detection model for recognition and detection to obtain the position information of the target idler. The detection model is obtained by training the Des-YOLO v3 deep convolutional neural network through the training data set. The training data set includes idler grayscale images with position annotation information. The training idler grayscale images are grayscale images corresponding to idler images collected in a scene where the ambient brightness is greater than the predetermined ambient brightness.

[0159] Specifically, during the nighttime idler roll detection test, it was found that the YOLO v3 network had the drawback of missing target detection, and the detection accuracy needed to be improved. DarkNet53, which has higher detection accuracy and speed, was selected as the base network of YOLO v3. By drawing on residual networks and dense connection networks, multiple improvements and experimental verifications were carried out on the basis of YOLO v3 deep convolutional neural network to obtain the Des-YOLO v3 network, which is the detection model mentioned above. The detection model enhances the recognition of small targets and the robustness to occluded and overlapping targets, and effectively reduces the number of convolutional layers of the original network, significantly improving the detection accuracy and speed of idler rolls. Its network parameters are shown in Table 1.

[0160] Step S203: Extract the outline image of the target idler within the location box corresponding to the above location information. The location box is the bounding box of the target idler in the third night idler image.

[0161] Specifically, the image of the third nighttime idler roller is tagged based on the location information to generate a bounding box (bbox), i.e., the aforementioned location box, and the image of the location box is then extracted to obtain the outline image of the target idler roller.

[0162] Step S204: If the first predetermined condition and / or the second predetermined condition are met, it is determined that the target idler roller has malfunctioned. The first predetermined condition is that the overlap between the outline image and the standard idler roller outline image is less than a predetermined overlap. The second predetermined condition is that the distance between the position corresponding to the position information and the predetermined position is greater than a predetermined distance. The predetermined position is the installation position of the target idler roller.

[0163] Specifically, after the detection model outputs the position information of the target idler, the contour image of the target idler is extracted within the position box corresponding to the position information. It is then compared with the contour image and installation position of the standard idler. If the difference is too large, the target idler is determined to be faulty, that is, the overlap between the contour image and the standard idler contour image is less than a predetermined overlap, and / or, the distance between the position corresponding to the position information and the predetermined position is greater than a predetermined distance.

[0164] Optionally, step S201 includes: step S2011, converting the format of the first nighttime idler image to HSV format to obtain a fourth nighttime idler image; step S2012, determining the brightness information, hue information, and saturation information of the fourth nighttime idler image based on the fourth nighttime idler image; step S2013, correcting the brightness information using the Gamma function to obtain corrected brightness information, such that the brightness corresponding to the corrected brightness information is greater than the brightness corresponding to the brightness information; and step S2014, synthesizing the second nighttime idler image in HSV format based on the corrected brightness information, the hue information, and the saturation information.

[0165] Specifically, the first nighttime idler image is converted into the HSV color space, and the image's luminance information V, hue information H, and saturation information S are obtained. The luminance information is processed by multi-scale Gaussian convolution to obtain the illumination component I(x, y). The 2D Gamma function is calculated, and after parameter adjustment, the luminance is corrected to obtain the corrected luminance component V', making the luminance corresponding to the corrected luminance information greater than the luminance corresponding to the above luminance information. The corrected luminance component is combined with the hue and saturation S information to re-synthesize the image, obtaining the second nighttime idler image in HSV format. The luminance correction improves the brightness of the idler image, making it easier to identify and detect the outline and position of the idler image, and reducing the difficulty of fault detection.

[0166] Optionally, step S2011 includes: step S20111, converting the format of the first night roller image to RGB format to obtain the fifth night roller image; step S20112, converting the format of the fifth night roller image to HSV format to obtain the fourth night roller image.

[0167] Specifically, since the RGB color space and the HSV color space can be converted to each other, the first night roller image is determined to be an RGB image. If it is a GRB image, it is directly converted to the HSV color space. If not, it is converted to an RGB image and then converted to the HSV color space. This process converts the first night roller image into the fourth night roller image in HSV format. The HSV format image includes brightness information, which is convenient for brightness correction.

[0168] Optionally, step S201 includes: step S2015, converting the format of the second nighttime idler image to RGB format to obtain the sixth nighttime idler image; step S2016, converting the sixth nighttime idler image to grayscale image to obtain the third nighttime idler image.

[0169] Specifically, the HSV format of the second nighttime idler roller image is converted to RGB format, and then to grayscale image. Grayscale image makes it easier to extract contour features and position features, which facilitates the identification and detection of the contour and position of the idler roller image and reduces the difficulty of fault detection.

[0170] Optionally, step S202 includes: step S2021, sequentially performing multiple sets of convolution processing, multiple sets of batch normalization processing, and multiple sets of activation function processing on the third nighttime roller image to obtain multiple first feature maps; step S2022, performing pooling processing on the multiple first feature maps to obtain multiple second feature maps; step S2023, performing one densely connected convolution processing and three residual convolution processing on the multiple second feature maps to obtain the position information, wherein the densely connected convolution processing is performed using the convolutional layers of the DenseNet densely connected convolutional network, and the residual convolution processing is performed by modifying the output value of the convolutional layer to the sum of the output value and the input value of the convolutional layer.

[0171] Specifically, the Des-YOLO v3 network extracts features from the input image through one densely connected convolution and three residual convolutions. Compared to the original framework DarkNet53, Des-YOLO v3's shallow network uses densely connected convolutional blocks, allowing shallow features to be transmitted to deeper convolutions more effectively and quickly. This enables the reuse and fusion of features across multiple network layers, further improving the overall information transfer and gradient transfer efficiency of the network. This is beneficial for combining upsampling with shallow features in regression prediction for target detection. In the latter part of the feature extraction network, Des-YOLO v3 retains the residual network structure, deepening the feature extraction network structure with three sets of residual blocks to improve the model's ability to select and extract deep image features. Finally, Des-YOLO v3 uses a scale pyramid structure similar to FPN networks, performing two upsampling operations and concatenating them with feature maps of the same size from the upper layers of the network for secondary regression prediction, achieving multi-scale detection of targets of different sizes. Furthermore, the Des-YOLO v3 network reduces the number of convolutional layers, thereby decreasing model size and computational cost, while maintaining high prediction accuracy. The Des-YOLO v3 network architecture is as follows: Figure 3 As shown, convolution is first performed through a Convolutional layer (Conv). After processing with an activation function, it enters a Max pooling layer for pooling to extract feature maps. These feature maps are then subjected to one densely connected convolution and three residual convolutions to obtain the aforementioned location information. CBR is a basic component of the Des-YOLO v3 network, including convolution, batch normalization (BN), and ReLU (Leaky ReLU) operations. Dense is a densely connected block containing two convolution operations. This module concatenates the feature maps before and after the convolution operations, enabling the reuse and fusion of multi-layer features. Res Block is a residual block, where Res_unit*n represents n residual units. Using residual structures allows for deeper network structures, obtaining high-dimensional features to improve prediction accuracy.

[0172] Optionally, step S201 includes: step S2017, processing the first nighttime idler image using an AE automatic exposure algorithm to obtain the second nighttime idler image.

[0173] Specifically, the AE automatic exposure algorithm is used to improve the brightness of the idler roller image, enhance the performance of idler roller target detection at night, and thus improve the accuracy of fault detection.

[0174] Optionally, after step S203, the method further includes step S301, issuing an alarm signal, which is used to prompt maintenance personnel to perform fault repair on the target idler roller.

[0175] Specifically, after a fault is detected in the target idler roller, an alarm signal is issued to remind maintenance personnel to repair the fault and avoid missing any faults that could affect production progress.

[0176] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0177] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0178] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0179] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0180] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0181] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0182] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0183] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0184] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0185] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0186] 1) In the nighttime environment idler detection method of this application, firstly, a first nighttime idler image is acquired, and the brightness of the first nighttime idler image is corrected to obtain a second nighttime idler image. The second nighttime idler image is then converted into a grayscale image to obtain a third nighttime idler image. The first nighttime idler image is an image of the target idler in a nighttime environment, and the brightness of the second nighttime idler image is greater than the brightness of the first nighttime idler image. Then, the third nighttime idler image is input into a detection model for recognition and detection to obtain the position information of the target idler. The detection model is a Des-YOLO model trained on a dataset. The training data set, obtained by training a v3 deep convolutional neural network, includes grayscale images of idlers with location annotation information. These grayscale images are grayscale images of idlers acquired in scenarios where the ambient brightness is greater than a predetermined ambient brightness. Then, the contour image of the target idler is extracted within the location bounding box corresponding to the location information. This location bounding box is the bounding box of the target idler in the third nighttime idler image. Finally, if a first predetermined condition and / or a second predetermined condition are met, it is determined that the target idler has malfunctioned. The first predetermined condition is that the overlap between the contour image and the standard idler contour image is less than a predetermined overlap. The second predetermined condition is that the distance between the location corresponding to the location information and a predetermined location is greater than a predetermined distance. The predetermined location is the installation location of the target idler. This method improves the brightness of idler roller images acquired at night by performing brightness correction. The improved images are then input into a detection model for identification and detection to obtain the position information of the target idler roller. Based on the contour image and position information, it can determine whether the target idler roller is faulty. Since the detection model is trained better on images with higher brightness, brightness correction is used to improve the accuracy of identification and detection of idler roller images acquired at night, thereby improving the accuracy of detection and positioning. This enhances the accuracy of fault detection, improves the safety of automated coal mine transportation, saves labor costs, and solves the problem of low fault detection accuracy in existing idler roller detection methods for nighttime environments.

[0187] 2) In the nighttime environment idler detection device of this application, the first processing unit acquires a first nighttime idler image, corrects the brightness of the first nighttime idler image to obtain a second nighttime idler image, converts the second nighttime idler image into a grayscale image to obtain a third nighttime idler image. The first nighttime idler image is an image of the target idler in a nighttime environment, and the brightness of the second nighttime idler image is greater than the brightness of the first nighttime idler image. The detection unit inputs the third nighttime idler image into a detection model for recognition and detection to obtain the position information of the target idler. The detection model is a Des-YOLO model trained on a dataset. The training data set, obtained by training a v3 deep convolutional neural network, includes grayscale images of idlers with location annotation information. These grayscale images are grayscale images of idlers acquired in scenarios where the ambient brightness is greater than a predetermined ambient brightness. The second processing unit extracts the contour image of the target idler within the location box corresponding to the location information. The location box is the bounding box of the target idler in the third nighttime idler image. The determining unit determines that the target idler has malfunctioned if the first predetermined condition and / or the second predetermined condition are met. The first predetermined condition is that the overlap between the contour image and the standard idler contour image is less than a predetermined overlap. The second predetermined condition is that the distance between the location corresponding to the location information and the predetermined location is greater than a predetermined distance. The predetermined location is the installation location of the target idler. This device performs brightness correction on idler roller images acquired at night to enhance image brightness. The enhanced image is then input into a detection model for identification and detection to obtain the position information of the target idler roller. Based on the contour image and position information, it determines whether the target idler roller is faulty. Since the detection model is trained better using images with higher brightness, brightness correction enhances the nighttime idler roller images to ensure the accuracy of identification and detection, improving the detection accuracy and positioning precision. This improves the accuracy of fault detection, enhances the safety of automated coal mine transportation, saves labor costs, and solves the problem of low fault detection accuracy in existing idler roller detection methods for nighttime environments.

[0188] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for detecting idler rollers in a nighttime environment, characterized in that, include: A first nighttime idler image is acquired, and the brightness of the first nighttime idler image is corrected to obtain a second nighttime idler image. The second nighttime idler image is then converted into a grayscale image to obtain a third nighttime idler image. The first nighttime idler image is an image of the target idler in a nighttime environment, and the brightness of the second nighttime idler image is greater than the brightness of the first nighttime idler image. The third nighttime idler image is input into the detection model for recognition and detection to obtain the position information of the target idler. The detection model is obtained by training the Des-YOLO v3 deep convolutional neural network through a training data set. The training data set includes a grayscale image of the idler with position labeling information. The grayscale image of the idler is the grayscale image corresponding to the idler image collected in a scene where the ambient brightness is greater than a predetermined ambient brightness. Extract the outline image of the target idler within the location box corresponding to the location information, where the location box is the bounding box of the target idler in the third night idler image; If the first predetermined condition and / or the second predetermined condition are met, it is determined that the target idler roller has malfunctioned. The first predetermined condition is that the overlap between the contour image and the standard idler roller contour image is less than a predetermined overlap. The second predetermined condition is that the distance between the position corresponding to the position information and the predetermined position is greater than a predetermined distance. The predetermined position is the installation position of the target idler roller. The third nighttime idler image is input into a detection model for recognition and detection to obtain the position information of the target idler. This includes: sequentially performing multiple sets of convolution processing, multiple sets of batch normalization processing, and multiple sets of activation function processing on the third nighttime idler image to obtain multiple first feature maps; performing pooling processing on the multiple first feature maps to obtain multiple second feature maps; and performing one densely connected convolution processing and three residual convolution processing on the multiple second feature maps to obtain the position information. The densely connected convolution processing is performed using the convolutional layers of the DenseNet densely connected convolutional network, and the residual convolution processing modifies the output value of the convolutional layer to be the sum of the output value and the input value of the convolutional layer.

2. The method according to claim 1, characterized in that, The brightness of the first nighttime idler image is corrected to obtain a second nighttime idler image, including: The format of the first nighttime idler roller image is converted to HSV format to obtain the fourth nighttime idler roller image; The brightness, hue, and saturation information of the fourth nighttime idler roller image are determined based on the image. The brightness information is corrected using the Gamma function to obtain corrected brightness information, such that the brightness corresponding to the corrected brightness information is greater than the brightness corresponding to the original brightness information. The second nighttime roller image in HSV format is synthesized based on the corrected brightness information, the hue information, and the saturation information.

3. The method according to claim 2, characterized in that, The format of the first nighttime idler roller image is converted to HSV format to obtain the fourth nighttime idler roller image, including: The format of the first nighttime idler roller image is converted to RGB format to obtain the fifth nighttime idler roller image; The format of the fifth nighttime idler roller image is converted to HSV format to obtain the fourth nighttime idler roller image.

4. The method according to claim 1, characterized in that, The second nighttime idler image is converted to a grayscale image to obtain the third nighttime idler image, including: The format of the second nighttime idler roller image is converted to RGB format to obtain the sixth nighttime idler roller image; The sixth nighttime idler image is converted into a grayscale image to obtain the third nighttime idler image.

5. The method according to any one of claims 1 to 3, characterized in that, The brightness of the first nighttime idler image is corrected to obtain a second nighttime idler image, including: The first nighttime idler image is processed using the AE automatic exposure algorithm to obtain the second nighttime idler image.

6. The method according to any one of claims 1 to 3, characterized in that, After determining that the target idler roller has failed when a first predetermined condition and / or a second predetermined condition are met, the method further includes: An alarm signal is issued to prompt maintenance personnel to perform fault repair on the target idler roller.

7. A roller detection device for nighttime environments, characterized in that, include: The first processing unit is used to acquire a first nighttime idler image, correct the brightness of the first nighttime idler image to obtain a second nighttime idler image, convert the second nighttime idler image into a grayscale image to obtain a third nighttime idler image, wherein the first nighttime idler image is an image of the target idler in a nighttime environment, and the brightness of the second nighttime idler image is greater than the brightness of the first nighttime idler image. The detection unit is used to input the third nighttime idler image into the detection model for recognition and detection to obtain the position information of the target idler. The detection model is obtained by training the Des-YOLO v3 deep convolutional neural network through a training data set. The training data set includes a grayscale image of the idler with position labeling information. The grayscale image of the idler is a grayscale image corresponding to the idler image collected in a scene where the ambient brightness is greater than a predetermined ambient brightness. The second processing unit is used to extract the outline image of the target idler within the position box corresponding to the position information, wherein the position box is the bounding box of the target idler in the third night idler image; The determining unit is configured to determine that the target idler roller has malfunctioned when a first predetermined condition and / or a second predetermined condition are met. The first predetermined condition is that the overlap between the contour image and the standard idler roller contour image is less than a predetermined overlap. The second predetermined condition is that the distance between the position corresponding to the position information and a predetermined position is greater than a predetermined distance. The predetermined position is the installation position of the target idler roller. The detection unit includes: a seventh processing module, used to sequentially perform multiple sets of convolution processing, multiple sets of batch normalization processing, and multiple sets of activation function processing on the third nighttime roller image to obtain multiple first feature maps; an eighth processing module, used to perform pooling processing on the multiple first feature maps to obtain multiple second feature maps; and a ninth processing module, used to perform one densely connected convolution processing and three residual convolution processing on the multiple second feature maps to obtain the position information. The densely connected convolution processing is a convolution processing performed using the convolutional layers of the DenseNet densely connected convolutional network, and the residual convolution processing is to modify the output value of the convolutional layer to the sum of the output value and the input value of the convolutional layer.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method of any one of claims 1 to 6.

9. A roller detection system, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising methods for performing any one of claims 1 to 6.

Images