A method and apparatus for detecting foreign objects on a conveyor belt

Abstract

The application provides a conveyor belt foreign matter detection method and device, and relates to the technical field of foreign matter detection.The method comprises the following steps: collecting a first long-exposure image and a first short-exposure image alternately shot at preset time intervals during the transportation of a conveyor belt, performing fuzzy integration on the first long-exposure image to obtain a second long-exposure image; modeling the first short-exposure image according to brightness and additive noise to obtain a second short-exposure image; performing alignment processing on the second long-exposure image and the second short-exposure image to obtain a long-exposure aligned image and a short-exposure aligned image, and then performing image smoothing processing on the long-exposure aligned image and the short-exposure aligned image to obtain a target image; in the application, the collected image is subjected to exposure compensation preprocessing, alignment processing and image smoothing processing, the obtained target image is input into a foreign matter detection model for foreign matter detection, all-weather automatic intelligent detection is adopted, the intensity of manual labor is effectively reduced, and the efficiency of foreign matter detection is improved.

Description

Technical Field

[0001] This invention relates to the field of foreign object detection technology, specifically to a method and apparatus for detecting foreign objects on a conveyor belt. Background Technology

[0002] Currently, my country's demand for energy and raw materials is increasing rapidly, leading to a year-on-year increase in coal mining output. The coal mining industry faces unprecedented challenges, such as unsafe working conditions, high occupational risks, and labor shortages. Furthermore, conveyor belt tearing is a common problem in coal mining. Conveyor belts experience significant wear and tear during prolonged operation, and sometimes, due to worker negligence or accidents, foreign objects such as wood, stones, and tools may accumulate on them. Therefore, it is essential to inspect conveyor belts for foreign objects and take preventative measures to avoid conveyor belt tearing.

[0003] In existing technologies, foreign object detection methods still largely rely on traditional manual and contact inspection. Manual inspection is susceptible to uncertainties such as the physical condition, work experience, and work status of staff, leading to a gradual increase in personnel and management costs and a continuous decline in the efficiency of foreign object detection. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in the prior art. Therefore, the object of this invention is to provide a method and apparatus for detecting foreign objects on conveyor belts, which has the advantages of reducing manual labor intensity and improving the efficiency of foreign object detection.

[0005] Technical solution: A method and device for detecting foreign objects on a conveyor belt, comprising:

[0006] Real-time images of the conveyor belt transportation process are acquired, including multiple first long exposure images and multiple first short exposure images taken alternately at preset time intervals;

[0007] By performing fuzzy integration on multiple first long exposure images, multiple second long exposure images are obtained;

[0008] Multiple first short exposure images are modeled based on brightness and additive noise, and the modeled multiple first short exposure images are subjected to histogram equalization to obtain multiple second short exposure images.

[0009] Multiple second long exposure images and multiple second short exposure images are aligned using a two-level complementary alignment algorithm to obtain multiple long exposure aligned images and multiple short exposure aligned images.

[0010] Image smoothing is performed on multiple long-exposure aligned images and multiple short-exposure aligned images to obtain the target image;

[0011] The target image is input into the foreign object detection model for foreign object detection.

[0012] Furthermore, the step of performing blur integration on the plurality of first long-exposure images includes:

[0013] By performing a blur integration on multiple first long-exposure images during the exposure time τ, multiple second long-exposure images are obtained, wherein...

[0014] The fuzzy integral is expressed as:

[0015]

[0016] Where L is the long-exposure image over a period of time from 0 to τ, τ is the exposure time, and I... t It is the i-th unblurred latent image per unit exposure time.

[0017] Furthermore, after performing blur integration on the plurality of the first long-exposure images, the process further includes:

[0018] Multiple first short-exposure images are modeled based on brightness and additive noise, wherein,

[0019] The modeling formula is expressed as follows:

[0020]

[0021] Where S is a short exposure image, where For the exposure time ratio, t L t S These represent the exposure time for long exposure images and the exposure time for short exposure images, respectively. Represents the gamma curve, with γ taking an empirical value of 2.3, n~N(0,σ) 2 ) represents Gaussian white noise in a short-exposure image, where n is a natural number.

[0022] Furthermore, the step of performing histogram equalization on the modeled first short-exposure images to obtain multiple second short-exposure images includes:

[0023] A histogram compensation algorithm was used for equalization processing to obtain multiple second short-exposure images, among which...

[0024] The histogram compensation algorithm is expressed as follows:

[0025]

[0026] Where M is the histogram matching function, S i It is the i-th short-exposure image within the exposure time τ. This indicates that the output histogram is the same as the long exposure image L.i+1 The matching second short exposure image, L i+1 It is the (i+1)th long exposure image within the exposure time τ.

[0027] Furthermore, the alignment process of the plurality of second long-exposure images and the plurality of second short-exposure images using a two-level complementary alignment algorithm includes:

[0028] Construct multiple short-exposure alignment frameworks and long-exposure alignment frameworks based on motion estimation and motion compensation in image processing;

[0029] Two adjacent second short exposure images or two adjacent second long exposure images and one second long exposure image or a second short exposure image between them are input into the corresponding long exposure alignment frame or short exposure alignment frame for first alignment processing to generate the output result of the first-level complementary alignment network. The output result of the first-level complementary alignment network includes multiple first-level long exposure processed images and first-level short exposure processed images.

[0030] Two adjacent first-level long exposure processed images or two adjacent first-level short exposure processed images and one first-level short exposure processed image or one first-level long exposure processed image between them are input into the corresponding short exposure alignment frame or long exposure alignment frame for second alignment processing to generate the output result of the second-level complementary alignment network.

[0031] The alignment process is iterated on the output of the second-level complementary alignment network until a long-exposure aligned image and a short-exposure aligned image are generated.

[0032] Furthermore, including:

[0033] Constructing multiple short-exposure alignment frameworks F based on motion estimation and motion compensation in image processing S Alignment frame F with long exposure L ;

[0034] Specifically, two adjacent second short exposure images or two adjacent second long exposure images, along with one second long exposure image or a second short exposure image between them, are used as the first-level complementary alignment network:

[0035]

[0036] The first-level complementary alignment network is input into the corresponding long-exposure alignment framework or short-exposure alignment framework for the first alignment process, generating the output of the first-level complementary alignment network, which is then used as the second-level complementary alignment network.

[0037]

[0038] The second-level complementary alignment network is input into the corresponding long-exposure alignment frame or short-exposure alignment frame for a second alignment process, generating the output result of the second-level complementary alignment network, which is represented as:

[0039]

[0040] Furthermore, when performing the first alignment process on the plurality of second long exposure images and the plurality of second short exposure images, or when performing the second alignment process on the plurality of first-level long exposure images and the plurality of first-level short exposure images, the corresponding plurality of short exposure alignment frames and long exposure alignment frames share weights.

[0041] Furthermore, the image smoothing process performed on the plurality of long-exposure aligned images and the plurality of short-exposure aligned images includes:

[0042] Multiple long-exposure aligned images and multiple short-exposure aligned images are input into a stream enhancement network, which then splits them into multiple image frames. The stream enhancement network uses a scaled recurrent network (SRN) to split the images.

[0043] Multiple image frames are input into a stream enhancement alignment framework for alignment processing to obtain the corresponding target image.

[0044] Furthermore, the step of inputting the target image into the foreign object detection model to detect foreign objects includes:

[0045] Multiple multi-attention layers are constructed and added to the foreign object detection model. The multi-attention layers include channel attention, spatial attention, and self-attention.

[0046] Based on the multiple attention layers, extract the key features of the target image;

[0047] The key features are compared with data in the database to detect foreign objects.

[0048] A conveyor belt foreign object detection device, characterized in that the device comprises:

[0049] The acquisition module is used to acquire real-time images during the conveyor belt transportation process. The real-time images include multiple first long exposure images and multiple first short exposure images that are taken alternately at preset time intervals.

[0050] An integration module is used to perform fuzzy integration on multiple first long exposure images to obtain multiple second long exposure images;

[0051] The modeling module is used to model multiple first short exposure images based on brightness and additive noise, and to perform histogram equalization processing on the modeled multiple first short exposure images to obtain multiple second short exposure images;

[0052] The alignment processing module is used to align multiple second long exposure images and multiple second short exposure images using a two-level complementary alignment algorithm to obtain multiple long exposure aligned images and multiple short exposure aligned images.

[0053] The smoothing module is used to perform image smoothing processing on multiple long-exposure aligned images and multiple short-exposure aligned images to obtain a target image;

[0054] The detection module is used to input the target image into the foreign object detection model for foreign object detection.

[0055] Beneficial effects: In this invention, a first long exposure image and a first short exposure image are captured alternately at preset time intervals during the conveyor belt transportation process. These images are then subjected to exposure compensation preprocessing, alignment processing, and image smoothing processing to obtain a low-blur and low-noise target image. The obtained target image is then input into a foreign object detection model for foreign object detection. This all-weather automated intelligent detection effectively reduces the intensity of manual labor and improves the efficiency of foreign object detection. Attached Figure Description

[0056] Figure 1 A flowchart illustrating a method for detecting foreign objects on a conveyor belt provided by the present invention;

[0057] Figure 2 A schematic diagram of the complementary alignment network in a conveyor belt foreign object detection method provided by the present invention;

[0058] Figure 3 A schematic diagram of the frame smoothing network in a conveyor belt foreign object detection method provided by the present invention;

[0059] Figure 4 This is a schematic diagram of the target detection network in a conveyor belt foreign object detection method provided by the present invention;

[0060] Figure 5 This is a schematic diagram of the SPP layer in a conveyor belt foreign object detection method provided by the present invention;

[0061] Figure 6 This is a schematic diagram of the structure of layer C3 in a conveyor belt foreign object detection method provided by the present invention;

[0062] Figure 7 This is a schematic diagram of the CBL layer structure in a conveyor belt foreign object detection method provided by the present invention;

[0063] Figure 8 This is a schematic diagram of the structure of the multi-attention layer in a conveyor belt foreign object detection method provided by the present invention;

[0064] Figure 9 This is a schematic diagram of the structure of a conveyor belt foreign object detection device provided by the present invention. Detailed Implementation

[0065] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0068] To address the problems of the prior art, the present invention provides a method and apparatus for detecting foreign objects on a conveyor belt. The method and apparatus for detecting foreign objects on a conveyor belt provided by the embodiments of the present invention are described below.

[0069] Figure 1 A schematic flowchart of a conveyor belt foreign object detection method provided by an embodiment of the present invention is shown. Figure 1 As shown, the conveyor belt foreign object detection method may specifically include the following steps:

[0070] S1. Collect real-time images during the conveyor belt transportation process. The real-time images include multiple first long exposure images and multiple first short exposure images that are taken alternately at preset time intervals.

[0071] S2. Perform fuzzy integration on multiple first long exposure images to obtain multiple second long exposure images;

[0072] S3. Model multiple first short exposure images based on brightness and additive noise, and perform histogram equalization on the modeled multiple first short exposure images to obtain multiple second short exposure images.

[0073] S4. Align the multiple second long exposure images and multiple second short exposure images using a two-level complementary alignment algorithm to obtain multiple long exposure aligned images and multiple short exposure aligned images.

[0074] S5. Perform image smoothing processing on multiple long-exposure aligned images and multiple short-exposure aligned images to obtain the target image;

[0075] S6. Input the target image into the foreign object detection model for foreign object detection.

[0076] Therefore, in this invention, a low-blur, low-noise target image is obtained by taking a first long-exposure image and a first short-exposure image alternately at a preset time interval during the conveyor belt transportation process, and performing exposure compensation preprocessing, alignment processing, and image smoothing processing on the image. The obtained target image is then input into a foreign object detection model for foreign object detection. This all-weather automated intelligent detection effectively reduces the intensity of manual labor and improves the efficiency of foreign object detection.

[0077] The specific implementation methods for each of the above steps are described below.

[0078] In some embodiments, in S1, the camera captures a plurality of first long exposure images L and a plurality of first short exposure images S, which are taken alternately at preset time intervals during the conveyor belt transport process.

[0079] As an example, to obtain low-blur, low-noise, high-frame-rate images, a video enhancement method based on hybrid exposure is proposed. Input: Low-frame-rate images with alternating exposures {S} i L i+1 |i=0:2:2N}。 S i It is the first short-exposure image, characterized by low blur and high texture; L i+1 It is the first long exposure image, characterized by high brightness and low noise. The camera captures the exposure image by periodically opening or closing the shutter.

[0080] In some embodiments, in S2, the image captured by the camera when the shutter is open is called an exposure, and a blur integration is performed on multiple first long exposure images.

[0081] As an example, one could assume that the i-th unblurred latent image within a unit exposure time τ is I. t By performing a fuzzy integration on the first long-exposure image during the exposure time τ, multiple second long-exposure images are obtained. The fuzzy integration formula is expressed as:

[0082]

[0083] Where L is the long-exposure image over a period of time from 0 to τ, τ is the exposure time, and I... t It is the i-th unblurred latent image per unit exposure time.

[0084] In some embodiments, in S3, the camera captures multiple first short-exposure images when the shutter opens. These first short-exposure images under low-light conditions can be modeled using brightness adjustment and additive noise, where the modeling formula is expressed as:

[0085]

[0086] Where S is a short exposure image, where For the exposure time ratio, t L t S These represent the exposure time for long exposure images and the exposure time for short exposure images, respectively. Represents the gamma curve, with γ taking an empirical value of 2.3, n~N(0,σ) 2 ) represents Gaussian white noise in a short-exposure image, where n is a natural number.

[0087] As an example, since a long-exposure image collects more photons within the exposure time τ than a short-exposure image, it is brighter. To compensate for this brightness difference, a histogram matching compensation mechanism is used. Multiple modeled first short-exposure images are processed using a histogram compensation algorithm for histogram equalization. The histogram compensation algorithm is expressed as follows:

[0088]

[0089] Where M is the histogram matching function, S i It is the i-th short-exposure image within the exposure time τ. This indicates that the output histogram is the same as the long exposure image L. i+1 The matching second short exposure image, L i+1 It is the (i+1)th long exposure image within the exposure time τ.

[0090] In some embodiments, in S4, such as Figure 2 As shown, Figure 2 This is a flowchart illustrating the complementary alignment network in a conveyor belt foreign object detection method provided by the present invention. The obtained multiple second long exposure images and multiple second short exposure images are aligned using a two-level complementary alignment algorithm.

[0091] In order to obtain low-blur, low-noise images and to make the second long-exposure image and the second short-exposure image clearer, the above-mentioned S4 may specifically include:

[0092] S4-1: Construct multiple short-exposure alignment frameworks and long-exposure alignment frameworks based on motion estimation and motion compensation in image processing.

[0093] As an example, such as Figure 2 As shown, construct F L and F S They serve as long exposure alignment frames and short exposure alignment frames, respectively, and F L and F S It is an alignment framework based on motion estimation and compensation, consisting of three optical flow estimation networks Flownet2 and one synthetic network SRN-DeblurNet.

[0094] S4-2: Input two adjacent second short exposure images or two adjacent second long exposure images and one second long exposure image or second short exposure image between them into the corresponding long exposure alignment frame or short exposure alignment frame for first alignment processing, and generate the output result of the first-level complementary alignment network. The output result of the first-level complementary alignment network includes multiple first-level long exposure processed images and first-level short exposure processed images.

[0095] As an example, consider two adjacent second short exposure images. and And a second long exposure image L between the two. i-1 Input the long exposure alignment frame F L In the process, a first-level long exposure image is generated. Similarly, take two adjacent second long exposure images L i-1 and L i+1 And a second short exposure image between the two. Input to short exposure alignment frame F S In the process, a first-level short exposure image is generated. In the first alignment process for multiple second long-exposure images and multiple second short-exposure images, the corresponding short-exposure alignment frameworks FS and long-exposure alignment frameworks FL share weights. This shared weight parameter reduces the number of parameters required for network training, thereby reducing the time required for the first alignment process and improving operational efficiency. For example... Figure 2 As shown, the input image is denoted as... After motion estimation and motion compensation in image processing, the output of the first-level complementary alignment network is as follows: Represented as

[0096] S4-3: Input two adjacent first-level long exposure processed images or two adjacent first-level short exposure processed images and one first-level short exposure processed image or one first-level long exposure processed image between them into the corresponding short exposure alignment frame or long exposure alignment frame for second alignment processing, and generate the output result of the second-level complementary alignment network.

[0097] As an example, the output of the first-level complementary alignment network can be used as the input to the second-level complementary alignment network, for instance, when processing an image with a first-level long exposure. and And a first-level short exposure processed image between the two. Input to the corresponding short exposure alignment frame F S In the middle, the output of the second-level complementary alignment network is generated. Similarly, process images with a first-level short exposure. and And a first-level long exposure processed image between the two. Input to the corresponding long exposure alignment frame F L In the middle, the output of the second-level complementary alignment network is generated. When performing second alignment processing on multiple first-level long-exposure processed images and multiple first-level short-exposure processed images, the corresponding multiple short-exposure alignment frames F S Alignment frame F with long exposure L Sharing weights between images reduces the number of parameters required for network training. It also reduces the time required for second alignment processing of first-level long-exposure and first-level short-exposure images, thus improving operational efficiency. For example... Figure 2 As shown, the output of the second-level complementary alignment network is

[0098] S4-4: Iterate through the alignment process of the output of the second-level complementary alignment network until a long-exposure aligned image and a short-exposure aligned image are generated.

[0099] As an example, consider the output of a second-level complementary alignment network. and Perform alignment iterations, aligning each short-exposure frame F S Alignment frame F with long exposure L The corresponding output is a long exposure aligned image. A short exposure aligned image

[0100] In some embodiments, in S5, such as Figure 3 As shown, Figure 3This is a flowchart illustrating the frame smoothing network in a conveyor belt foreign object detection method provided by the present invention, showing the generation of multiple long-exposure aligned images. Aligned with multiple short exposure images Perform image smoothing processing.

[0101] To enhance the smoothness of long-exposure aligned images and short-exposure aligned images, S5 above may specifically include:

[0102] S5-1: Input multiple long-exposure aligned images and multiple short-exposure aligned images into the stream enhancement network, and the stream enhancement network splits them into multiple image frames. The stream enhancement network uses a scaled recurrent network (SRN) to split the images.

[0103] As an example, the complementary alignment algorithm used in the above steps processes 6 frames of images into 2 frames, generating a short-exposure aligned image with reduced noise. Aligned images with long exposures and reduced blur. To improve the sharpness and dynamic effect of the generated aligned image, the output of the second-level complementary alignment network is incorporated into the frame smoothing network. The input is fed into the Stream Enhancement Network, which splits it into multiple distorted image frames V. The Stream Enhancement Network uses a Scale Recurrent Network (SRN, Single Registration Number) to process the image splitting. It learns the features of different images by training on a large amount of image data, so as to accurately identify and segment images in real-time scenes.

[0104] S5-2: Input multiple image frames into the stream enhancement alignment framework for alignment processing to obtain the corresponding target image.

[0105] As an example, the multiple distorted image frames V obtained from the splitting are input into the stream enhancement alignment frame F. i Alignment processing is performed during the image frame V synthesis to enhance the clarity of the synthesized image frame V, improve the dynamic effect of the synthesized image frame V, and reduce image artifacts and distortion caused by the lack of image smoothing. This results in better performance in complex scenes, producing a target image with low noise, low blur, and high frame rate. Among these, F... i Is with F S and F L Alignment networks with the same frame.

[0106] In some embodiments, in S6, such as Figure 4 As shown, Figure 4This is a schematic diagram of the target detection network in a conveyor belt foreign object detection method provided by the present invention. The target image is input into the target detection network based on the improved YOLOv5 foreign object detection algorithm to detect foreign objects in the target image.

[0107] To improve the accuracy of foreign object identification in a single draft, the aforementioned S6 may specifically include:

[0108] S6-1: Construct multiple multi-attention layers and add them to the foreign object detection model. The multi-attention layers include channel attention, spatial attention, and self-attention.

[0109] S6-2: Extract key features of the target image based on multiple attention layers;

[0110] S6-3: Compare key features with data in the database to detect foreign objects.

[0111] As an example, such as Figure 4 As shown, the improved YOLOv5 structure retains the original Backbone feature extraction layer and constructs a multi-attention layer before each CBL layer in the Neck part, such as... Figure 8 As shown, the multi-attention layer includes channel attention, spatial attention, and self-attention. Channel attention focuses on which features in the image are meaningful. Spatial attention focuses on where features in the image are meaningful. Self-attention makes the network better at capturing the internal correlations of features and extracting more important image features; for example... Figure 5 As shown, the SPP layer includes input, CBL, max pooling layer, and feature fusion layer; as Figure 6 As shown, layer C3 includes the input layer, CBL layer, and feature fusion layer; as Figure 7 As shown, the CBL includes an input layer, a convolutional layer, a batch normalization layer, and a LeakyReLU activation function. By constructing a multi-attention layer, the recognition accuracy of the YOLOv5 foreign object detection algorithm is effectively improved.

[0112] Therefore, in this invention, a low-blur, low-noise target image is obtained by acquiring a first long-exposure image and a first short-exposure image alternately captured at a preset time interval during the conveyor belt transportation process, and performing exposure compensation preprocessing, alignment processing, and image smoothing processing on these images. The obtained target image is then input into a foreign object detection model for foreign object detection. This model employs all-weather automated intelligent detection and is based on an improved YOLOv5 foreign object detection algorithm. By constructing multiple attention layers in the Neck part of the YOLOv5 structure, the efficiency of foreign object detection is effectively improved, and the accuracy of foreign object identification is increased.

[0113] Based on the same inventive concept, the present invention also provides a conveyor belt foreign object detection device. (Specifically combined with...) Figure 9 Please provide a detailed explanation.

[0114] Figure 9 This is a schematic diagram of the structure of a conveyor belt foreign object detection device provided by the present invention.

[0115] like Figure 9 As shown, the conveyor belt foreign object detection device 900 may include:

[0116] The acquisition module 901 is used to acquire real-time images during the conveyor belt transportation process. The real-time images include multiple first long exposure images and multiple first short exposure images that are taken alternately at preset time intervals.

[0117] The integration module 902 is used to perform fuzzy integration on multiple first long exposure images to obtain multiple second long exposure images;

[0118] Modeling module 903 is used to model multiple first short exposure images based on brightness and additive noise, and to perform histogram equalization on the modeled multiple first short exposure images to obtain multiple second short exposure images;

[0119] Alignment processing module 904 is used to align multiple second long exposure images and multiple second short exposure images using a two-level complementary alignment algorithm to obtain multiple long exposure aligned images and multiple short exposure aligned images.

[0120] The smoothing module 905 is used to perform image smoothing on multiple long-exposure aligned images and multiple short-exposure aligned images to obtain the target image;

[0121] The detection module 906 is used to input the target image into the foreign object detection model for foreign object detection.

[0122] Therefore, in this invention, a low-blur, low-noise target image is obtained by taking a first long-exposure image and a first short-exposure image alternately at a preset time interval during the conveyor belt transportation process, and performing exposure compensation preprocessing, alignment processing, and image smoothing processing on the image. The obtained target image is then input into a foreign object detection model for foreign object detection. This all-weather automated intelligent detection effectively reduces the intensity of manual labor and improves the efficiency of foreign object detection.

[0123] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

Claims

1. A method for detecting foreign objects on a conveyor belt, characterized in that, include: Real-time images of the conveyor belt transportation process are acquired, including multiple first long exposure images and multiple first short exposure images taken alternately at preset time intervals; By performing fuzzy integration on multiple first long exposure images, multiple second long exposure images are obtained; Multiple first short exposure images are modeled based on brightness and additive noise, and the modeled multiple first short exposure images are subjected to histogram equalization to obtain multiple second short exposure images. Multiple second long-exposure images and multiple second short-exposure images are aligned using a two-level complementary alignment algorithm to obtain multiple long-exposure aligned images and multiple short-exposure aligned images, including: Construct multiple short-exposure alignment frameworks and long-exposure alignment frameworks based on motion estimation and motion compensation in image processing; Two adjacent second short exposure images or two adjacent second long exposure images and one second long exposure image or a second short exposure image between them are input into the corresponding long exposure alignment frame or short exposure alignment frame for first alignment processing to generate the output result of the first-level complementary alignment network. The output result of the first-level complementary alignment network includes multiple first-level long exposure processed images and first-level short exposure processed images. Two adjacent first-level long exposure processed images or two adjacent first-level short exposure processed images and one first-level short exposure processed image or one first-level long exposure processed image between them are input into the corresponding short exposure alignment frame or long exposure alignment frame for second alignment processing to generate the output result of the second-level complementary alignment network. The alignment process is iterated on the output of the second-level complementary alignment network until a long-exposure aligned image and a short-exposure aligned image are generated. Image smoothing is performed on multiple long-exposure aligned images and multiple short-exposure aligned images to obtain a target image, including: Multiple long-exposure aligned images and multiple short-exposure aligned images are input into a stream enhancement network, which then splits them into multiple image frames. The stream enhancement network uses a scaled recurrent network (SRN) to split the images. Multiple image frames are input into a stream enhancement alignment framework for alignment processing to obtain the corresponding target image; The target image is input into the foreign object detection model for foreign object detection.

2. The method for detecting foreign objects on a conveyor belt according to claim 1, characterized in that, The step of performing blur integration on multiple first long-exposure images includes: Regarding exposure time Multiple first long exposure images during the period are subjected to blur integration to obtain multiple second long exposure images, wherein, The fuzzy integral is expressed as: Where L is from 0 to Long exposure images over a period of time, It's the exposure time, I t It is the i-th unblurred latent image per unit exposure time.

3. The method for detecting foreign objects on a conveyor belt according to claim 2, characterized in that, After performing blur integration on multiple first long-exposure images, the method further includes: Multiple first short-exposure images are modeled based on brightness and additive noise, wherein, The modeling formula is expressed as follows: Where S is a short exposure image, where For the exposure time ratio, t L t S These represent the exposure time for long exposure images and the exposure time for short exposure images, respectively. Represents the gamma curve. Take the empirical value of 2.

3. This represents Gaussian white noise in a short-exposure image, where n is a natural number.

4. The method for detecting foreign objects on a conveyor belt according to claim 3, characterized in that, The step of performing histogram equalization on the modeled first short-exposure images to obtain multiple second short-exposure images includes: A histogram compensation algorithm was used for equalization processing to obtain multiple second short-exposure images, among which... The histogram compensation algorithm is expressed as follows: Where M is the histogram matching function, S i Exposure time The i-th short exposure image within, This indicates the output histogram and the long exposure image L. i+1 The matching second short exposure image, L i+1 Exposure time The (i+1)th long exposure image within.

5. The method for detecting foreign objects on a conveyor belt according to claim 4, characterized in that, include: Constructing multiple short-exposure alignment frameworks F based on motion estimation and motion compensation in image processing S Alignment frame F with long exposure L ; Specifically, two adjacent second short exposure images or two adjacent second long exposure images, along with one second long exposure image or a second short exposure image between them, are used as the first-level complementary alignment network: The first-level complementary alignment network is input into the corresponding long-exposure alignment framework or short-exposure alignment framework for the first alignment process, generating the output of the first-level complementary alignment network, which is then used as the second-level complementary alignment network. The second-level complementary alignment network is input into the corresponding long-exposure alignment frame or short-exposure alignment frame for a second alignment process, generating the output result of the second-level complementary alignment network, which is represented as: 。 6. The method for detecting foreign objects on a conveyor belt according to claim 5, characterized in that, When performing a first alignment process on the plurality of second long exposure images and the plurality of second short exposure images, or when performing a second alignment process on the plurality of first-level long exposure images and the plurality of first-level short exposure images, the corresponding plurality of short exposure alignment frames and long exposure alignment frames share weights.

7. The method for detecting foreign objects on a conveyor belt according to claim 1, characterized in that, The step of inputting the target image into the foreign object detection model to detect foreign objects includes: Multiple multi-attention layers are constructed and added to the foreign object detection model. The multi-attention layers include channel attention, spatial attention, and self-attention. Based on the multiple attention layers, extract the key features of the target image; The key features are compared with data in the database to detect foreign objects.

8. A conveyor belt foreign object detection device, implementing the method as described in any one of claims 1-7, characterized in that, The device includes: The acquisition module is used to acquire real-time images during the conveyor belt transportation process. The real-time images include multiple first long exposure images and multiple first short exposure images that are taken alternately at preset time intervals. An integration module is used to perform fuzzy integration on multiple first long exposure images to obtain multiple second long exposure images; The modeling module is used to model multiple first short exposure images based on brightness and additive noise, and to perform histogram equalization processing on the modeled multiple first short exposure images to obtain multiple second short exposure images; The alignment processing module is used to align multiple second long exposure images and multiple second short exposure images using a two-level complementary alignment algorithm to obtain multiple long exposure aligned images and multiple short exposure aligned images. The smoothing module is used to perform image smoothing processing on multiple long-exposure aligned images and multiple short-exposure aligned images to obtain a target image; The detection module is used to input the target image into the foreign object detection model for foreign object detection.

Images