Vision system for identifying foreign bodies in a belt based on image processing

Abstract

The application discloses a kind of based on image processing's belt foreign matter vision identification system, it is related to image processing technical field;Image enhancement module obtains initial image, and the first image is obtained by enhancing initial image to first feature by default illumination enhancement algorithm and is substituted into feature extraction module;Characteristic enhancement module obtains first enhanced feature by substituting first feature into characteristic enhancement module;Pooling enhanced feature module obtains first pooling enhanced feature by substituting first enhanced feature into pooling enhanced feature;Foreign matter identification module obtains target detection result by substituting first feature, first enhanced feature and first pooling enhanced feature into foreign matter detection model.The problem such as uneven illumination, low contrast of initial image is corrected by default illumination enhancement algorithm, avoid foreign matter missed detection caused by light interference, and then the confidence of model to detection result is more reliable by carrying out multi-stage feature extraction fusion to first image, improve the accuracy and efficiency of foreign matter detection.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a visual recognition system for foreign objects on belts based on image processing. Background Technology

[0002] In modern coal mining operations, belt conveyor systems, as the primary means of coal transportation, directly impact the stability and reliability of the entire production system. The continuous and stable operation of belt conveyors not only affects coal production efficiency but is also a crucial factor in ensuring the safety of underground workers. However, in actual production, due to complex geological conditions and limitations in mining technology, various foreign objects such as coal gangue, anchor bolts, angle iron, and large pieces of coal can easily become mixed into the raw coal. If these foreign objects are not identified and handled in a timely manner, they can trigger a series of serious operational failures: from minor issues like belt misalignment and accelerated equipment wear to major accidents such as belt tearing and transmission system blockage, affecting not only the safe and efficient operation of the belt conveyor but also potentially threatening the safe production of the entire mine.

[0003] Patent CN119006466B discloses a method for detecting foreign objects in a coal mine belt conveyor. The method involves acquiring RGB and grayscale images of the coal mine; determining the degree of local disorder within the neighborhood of each pixel based on the color deviation of each pixel in each connected region of the grayscale image; determining the probability of the presence of metallic minerals within each connected region based on the theoretical centroid position of the disorder level; and obtaining an enhanced RGB image of the coal mine based on the enhanced color value of the first color channel of each pixel in the RGB image. This allows for the identification of metallic foreign objects in the coal mine conveyor belt. This invention improves the accuracy of foreign object detection. However, when faced with low-quality acquired images, the accuracy of metallic foreign object detection remains insufficient, resulting in low detection efficiency. Summary of the Invention

[0004] The purpose of this invention is to solve the problem that the accuracy of metal foreign object detection is still not high enough when the quality of the acquired images is low, resulting in low foreign object detection efficiency. Therefore, this invention proposes a belt foreign object visual recognition system based on image processing.

[0005] This invention proposes a visual recognition system for foreign objects on belts based on image processing, the system comprising:

[0006] An image enhancement module is used to acquire an initial image and enhance the initial image using a preset illumination enhancement algorithm to obtain a first image;

[0007] The feature extraction module is used to substitute the first image into the feature extraction module to obtain the first feature;

[0008] A feature enhancement module is used to substitute the first feature into the feature enhancement module to obtain a first enhanced feature;

[0009] A pooling enhancement feature module is used to substitute the first enhancement feature into the pooling enhancement feature module to obtain the first pooling enhancement feature;

[0010] The foreign object recognition module is used to substitute the first feature, the first enhanced feature, and the first pooling enhanced feature into the foreign object detection model to obtain the target detection result; the target detection result includes the target detection box and the confidence score.

[0011] Optionally, the image enhancement module includes:

[0012] The grayscale conversion module is used to perform Gaussian blurring on the initial image and then perform grayscale conversion to obtain a grayscale image.

[0013] An initial weight matrix construction module is used to initialize the connection weights between a target pixel and its neighboring pixels to obtain an initial weight matrix for the grayscale image; the target pixel is any pixel in the grayscale image.

[0014] An updated weight matrix construction module is used to calculate the gradient magnitude between the target pixel and its neighboring pixels to obtain an updated weight matrix; the initial weight matrix is ​​updated based on the updated weight matrix.

[0015] The target illumination image generation module is used to obtain a sparse Laplacian matrix based on the initial weight matrix updated for all pixels, and to fuse the initial image and the sparse Laplacian matrix through sparse Cholesky decomposition to obtain the target illumination image.

[0016] The first image generation module is used to correct the initial image based on the target illumination image to obtain the first image.

[0017] Optionally, the first image generation module includes:

[0018] An underexposed image generation module is used to obtain an underexposed image by dividing the initial image by the target illumination image;

[0019] An overexposed image generation module is used to process the initial image and the target illumination image using a preset algorithm to obtain an overexposed image;

[0020] The contrast weight determination module is used to calculate the Laplacian response corresponding to the underexposed image and the overexposed image respectively, and determine a first contrast weight and a second contrast weight based on the Laplacian response of each pixel in the underexposed image and the overexposed image; the first contrast weight is the weight corresponding to each pixel in the underexposed image; the second contrast weight is the weight corresponding to each pixel in the overexposed image.

[0021] The first fused image determination module is used to fuse the underexposed image and the overexposed image according to the first contrast weight and the second contrast weight to obtain a first fused image;

[0022] The saturation weight determination module is used to perform HSV color space transformation on both the underexposed image and the overexposed image. For the transformed image, it calculates the saturation of each pixel in each image to obtain a first saturation weight and a second saturation weight. The first saturation weight is the weight corresponding to the pixel in the underexposed image. The second saturation weight is the weight corresponding to the pixel in the overexposed image.

[0023] The second fused image determination module is used to fuse the underexposed image and the overexposed image according to the first saturation weight and the second saturation weight to obtain a second fused image;

[0024] The first image generation module is used to superimpose the initial image, the first fused image, and the second fused image and calculate the average to obtain the first image.

[0025] Optionally, the feature extraction module includes a first convolutional layer, a first deep convolutional layer, a second deep convolutional layer, a third deep convolutional layer, and a fully connected layer;

[0026] The first image is sequentially passed through the first convolutional layer, the first deep convolutional layer, the second deep convolutional layer, the third deep convolutional layer, and the fully connected layer to obtain the first feature.

[0027] Optionally, the feature enhancement module includes a first edge feature extraction module, a second edge feature extraction module, and a second convolutional layer;

[0028] The first feature is sequentially processed by the first edge feature extraction module and the second edge feature extraction module to obtain the first edge feature and the second edge feature;

[0029] The first edge feature and the second edge feature are fused to obtain a fused edge feature, and the fused edge feature is substituted into the second convolutional layer to obtain a first enhanced feature.

[0030] Optionally, the pooling enhancement feature module includes a max pooling layer, a fourth depthwise convolution, a fifth depthwise convolution, a first extended depthwise convolution, and a second extended depthwise convolution.

[0031] The first enhanced feature is sequentially passed through the fourth depth convolution, the fifth depth convolution, the first extended depth convolution, and the second extended depth convolution to obtain the target extended depth convolution feature;

[0032] Multiply the first enhanced feature and the target extended depth convolutional feature to obtain the fused extended depth convolutional feature;

[0033] Substituting the first enhancement feature into the max pooling layer yields the pooling enhancement feature. The fused extended depth convolution feature and the pooling enhancement feature are then fused and subjected to a 1x1 convolution to obtain the first pooling enhancement feature.

[0034] Optional, the foreign object detection module includes:

[0035] The first splicing feature determination module is used to upsample the first pooling enhancement feature and then splice it with the first feature to obtain the first splicing feature;

[0036] The first splicing enhancement feature determination module is used to substitute the first splicing feature into the second feature enhancement model to obtain the first splicing enhancement feature;

[0037] The second splicing feature determination module is used to upsample the first splicing enhancement feature and then splice it with the first feature to obtain the second splicing feature;

[0038] The third splicing feature determination module is used to perform a 1x1 convolution on the second splicing feature and then splice it with the first splicing enhancement feature to obtain the third splicing feature;

[0039] The second splicing enhancement feature determination module is used to substitute the third splicing feature into the second feature enhancement model to obtain the second splicing enhancement feature;

[0040] The fourth splicing feature determination module is used to perform a 1x1 convolution on the second splicing enhancement feature and then splice it with the first enhancement feature to obtain the fourth splicing feature;

[0041] The target detection result determination module is used to substitute the second splicing feature, the second splicing enhancement feature and the fourth splicing feature into the detection head to obtain the target detection result.

[0042] Optionally, the second feature enhancement model includes:

[0043] The local pooling module is used to perform local pooling on the input features to obtain locally pooled features;

[0044] A global pooling module is used to perform global pooling on the local pooling features to obtain global pooling features;

[0045] The pooling feature convolution module is used to perform one-dimensional convolution on the local pooling features and the global pooling features respectively to obtain the target local pooling features and the target global pooling features;

[0046] The attention weight map generation module is used to upsample the target local pooling features and the target global pooling features respectively to obtain a global attention weight map and a local spatial attention weight map;

[0047] The element-wise addition module is used to add the global attention weight map and the local spatial attention weight map element by element to obtain the target attention weight;

[0048] The feature weighting module is used to multiply the target attention weights by the input features to obtain the output features of the second feature enhancement model.

[0049] The beneficial effects of this invention are:

[0050] This invention proposes a belt foreign object visual recognition system based on image processing. The system comprises an image enhancement module for acquiring an initial image and enhancing it using a preset illumination enhancement algorithm to obtain a first image; a feature extraction module for substituting the first image into the feature extraction module to obtain a first feature; a feature enhancement module for substituting the first feature into the feature enhancement module to obtain a first enhanced feature; a pooling enhancement module for substituting the first enhanced feature into the pooling enhancement feature to obtain a first pooled enhanced feature; and a foreign object recognition module for substituting the first feature, the first enhanced feature, and the first pooled enhanced feature into a foreign object detection model to obtain a target detection result. The target detection result includes a target detection box and a confidence score. The preset illumination enhancement algorithm corrects uneven illumination and low contrast in the initial image, restoring true colors and details and avoiding missed detections of foreign objects due to light interference. Furthermore, multi-stage feature extraction and fusion of the first image makes the model's confidence score assessment of the detection result more reliable, improving the accuracy and efficiency of foreign object detection. Attached Figure Description

[0051] The invention will now be further described with reference to the accompanying drawings.

[0052] Figure 1 A framework diagram of a belt foreign object visual recognition system based on image processing provided in an embodiment of the present invention;

[0053] Figure 2 This is a framework diagram of a second feature enhancement model provided in an embodiment of the present invention. Detailed Implementation

[0054] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0055] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] This invention provides a visual recognition system for foreign objects on a belt based on image processing. See also... Figure 1 , Figure 1 This is a framework diagram of a belt foreign object visual recognition system based on image processing, provided as an embodiment of the present invention. The system includes:

[0057] The image enhancement module is used to acquire an initial image and enhance the initial image using a preset illumination enhancement algorithm to obtain a first image;

[0058] The feature extraction module is used to input the first image into the feature extraction module to obtain the first feature;

[0059] A feature enhancement module is used to substitute the first feature into the feature enhancement module to obtain the first enhanced feature;

[0060] The pooling enhancement feature module is used to substitute the first enhancement feature into the pooling enhancement feature module to obtain the first pooling enhancement feature;

[0061] The foreign object recognition module is used to input the first feature, the first enhanced feature, and the first pooling enhanced feature into the foreign object detection model to obtain the target detection result; the target detection result includes the target detection box and the confidence score.

[0062] The belt foreign object visual recognition system based on image processing provided by this invention corrects problems such as uneven lighting and low contrast in the initial image by using a preset illumination enhancement algorithm, restores true colors and details, and avoids missed detection of foreign objects due to light interference. Furthermore, by performing multi-stage feature extraction and fusion on the first image, the confidence assessment of the detection results by the model becomes more reliable, thereby improving the accuracy and efficiency of foreign object detection.

[0063] In one implementation, a preset algorithm is used to correct problems such as uneven lighting and low contrast in the initial image, restore true colors and details, and avoid missing foreign objects (such as tiny impurities covered by shadows) due to light interference.

[0064] In one implementation, the initial image is a picture taken by a camera of the conveyor belt that transports minerals in the mine. Since the light intensity is low in the mine, additional lighting is used to compensate for the light intensity, which also makes the captured image subject to a lot of underexposure and overexposure.

[0065] In one implementation, a feature extraction module captures low-level features such as edges and textures in the image, and then a feature enhancement module further amplifies the effective features (such as the shape differences of foreign objects) and suppresses background noise (such as interference from uniform textures), thereby significantly improving the distinguishability between foreign object features and normal areas.

[0066] In one implementation, when the confidence level is greater than a preset threshold, the target detection result is considered abnormal. At this time, an alarm is issued and the target detection box is sent to the display to remind technicians to remove it.

[0067] In one implementation, scale invariance enhancement is performed on the feature map through pooling operations (such as adaptive pooling and dilated pooling) to ensure that foreign objects can be stably detected at different sizes and positions. In coal mine conveyor belt detection, objects may move due to the movement of the conveyor belt. Pooling enhancement can avoid missed detections caused by feature misalignment and improve detection stability.

[0068] In one embodiment, the image enhancement module includes:

[0069] The grayscale conversion module is used to convert the initial image to grayscale after Gaussian blurring to obtain a grayscale image.

[0070] The initial weight matrix construction module is used to initialize the connection weights between the target pixel and its neighboring pixels for a grayscale image to obtain the initial weight matrix; the target pixel is any pixel in the grayscale image.

[0071] The updated weight matrix construction module is used to calculate the gradient magnitude between the target pixel and its neighboring pixels to obtain the updated weight matrix; the initial weight matrix is ​​then updated based on the updated weight matrix.

[0072] The target illumination image generation module is used to obtain a sparse Laplacian matrix based on the initial weight matrix updated from all pixels, and to fuse the initial image and the sparse Laplacian matrix through sparse Cholesky decomposition to obtain the target illumination image.

[0073] The first image generation module is used to correct the initial image based on the target illumination image to obtain the first image.

[0074] In one implementation, the initial image (A) is Gaussian blurred (kernel size 3×3) to reduce the interference of high-frequency noise on illumination estimation. Then, the image is converted to grayscale to preserve brightness information and simplify calculations. The connectivity between the target pixel (any pixel in the grayscale image) and its neighboring pixels (8-neighborhood) is initialized (i.e., the weights between neighboring pixels are set to 1). The gradient magnitude of the target pixel (Sobel operator) is calculated, and the weights between the target pixel and its neighboring pixels are changed according to the gradient magnitude. Regions with larger gradient magnitudes (usually edges or details) have smaller weights (allowing for more drastic illumination changes while preserving edge details). The Sobel operators between neighboring pixels and the target pixel are calculated, and then sorted from smallest to largest. The first sorted set is obtained by sorting the data. Then, the total value is obtained by summing all the calculated magnitudes. The weighted sorted set is obtained according to the proportion of each data in the total value in the first sorted set. The weighted sorted set is reversed and used as the weight of the corresponding adjacent pixels in the first sorted set. For example, if the pixel order in the first sorted set is (S1, S2, S3, S4, S5, S6, S7, S8, S9), then their corresponding weights should be (the proportion of S9 in the total value, the proportion of S8 in the total value, the proportion of S7 in the total value, the proportion of S6 in the total value, the proportion of S5 in the total value, the proportion of S4 in the total value, the proportion of S3 in the total value, the proportion of S2 in the total value, and the proportion of S1 in the total value). The smaller the gradient magnitude (such as a smooth surface), the greater the weight (forced lighting smoothing, suppressing noise).

[0075] In one implementation, for the grayscale image after weight calculation, the gradient magnitude of all pixels is calculated to obtain a sparse Laplacian matrix; the initial image and the sparse Laplacian matrix are processed by sparse Cholesky decomposition (B=A+λF, λ is a preset weight, usually 0.5, ranging from 0 to 1. The larger λ is, the smoother the illumination map is (suitable for noisy scenes), and the smaller λ is, the closer the illumination map is to the original image (suitable for scenes with uniform illumination), and F is the sparse Laplacian matrix) to obtain the illumination map B.

[0076] In one implementation, the initial image is convolved with a Gaussian kernel to smooth out sharp transitions between pixels and reduce sensor noise and texture detail interference; the pixel connection weights across the object boundaries are automatically reduced to avoid incorrect association between the foreground and background during illumination estimation; and the weights of pixels with the same texture are reduced in high-texture areas to prevent illumination estimation bias caused by repeated calculations.

[0077] In one embodiment, the first image generation module includes:

[0078] The underexposed image generation module is used to obtain an underexposed image by dividing the initial image by the target illumination image;

[0079] The overexposed image generation module is used to process the initial image and the target illumination image using a preset algorithm to obtain an overexposed image;

[0080] The contrast weight determination module is used to calculate the Laplacian response corresponding to the underexposed image and the overexposed image respectively, and determine the first contrast weight and the second contrast weight based on the Laplacian response of each pixel in the underexposed image and the overexposed image respectively; the first contrast weight is the weight corresponding to each pixel in the underexposed image; the second contrast weight is the weight corresponding to each pixel in the overexposed image.

[0081] The first fused image determination module is used to fuse the underexposed image and the overexposed image according to the first contrast weight and the second contrast weight to obtain the first fused image;

[0082] The saturation weight determination module is used to perform HSV color space transformation on both underexposed and overexposed images. For the transformed image, it calculates the saturation of each pixel in each image to obtain a first saturation weight and a second saturation weight. The first saturation weight is the weight corresponding to the pixel in the underexposed image; the second saturation weight is the weight corresponding to the pixel in the overexposed image.

[0083] The second fused image determination module is used to fuse the underexposed image and the overexposed image according to the first saturation weight and the second saturation weight to obtain the second fused image;

[0084] The first image generation module is used to overlay the initial image, the first fused image, and the second fused image and calculate the average to obtain the first image.

[0085] In one implementation, the dynamic range of bright areas is compressed through linear mapping or logarithmic transformation to prevent the pixel values ​​of overexposed areas from saturating to 255. Compared with traditional histogram truncation, this method can preserve the texture details of metallic reflections and avoid the loss of information in overexposed areas.

[0086] In one implementation, underexposed images have low contrast in dark areas and weak Laplacian response, requiring increased weighting to enhance details; overexposed images have easily oversaturated bright areas and their Laplacian response may be too high due to noise amplification, requiring decreased weighting to suppress artifacts; underexposed images contribute dark details, while overexposed images preserve bright structures. The optimal source is dynamically selected through contrast weighting to avoid the dynamic range limitations of a single image.

[0087] In one implementation, an underexposed image M is obtained by dividing the initial image A by the illumination image B (the corrected pixel value may exceed 1, so it needs to be truncated to the [0,1] interval to avoid oversaturation); the initial image A is normalized by the illumination image B, and the pixel value of the dark area (small B value) is amplified (e.g. when B=0.2, the pixel value of A=0.1 is corrected to 0.5, and the brightness is increased by 5 times), thereby revealing the details of the dark area (such as the outline of foreign objects in the shadow); the overexposed image N is obtained by the formula (preset algorithm) C=1-(1-A) / (1-B), and for the bright area (B close to 1), the denominator 1-B approaches 0, thereby compressing 1-A (the pixel value of the overexposed area) and restoring the details of the bright area (such as the texture of the reflective surface).

[0088] In one implementation, the contrast weight of the underexposed image is obtained by calculating the Laplacian response for both the underexposed image M and the overexposed image N. Contrast weighting of overexposed images Rule: At the same location, if the response value of the underexposed image M is larger, then the weight of that region is biased towards the underexposed image M (indicating that the dark area correction map retains more edge details). The contrast weight corresponding to the same pixel point... and contrast weight Normalization is performed so that the weights sum to 1; the image is converted to the HSV color space, and the saturation (S component) of each pixel is calculated to obtain the saturation weights of the underexposed image. Saturation weights of overexposed images Rule: Regions with lower saturation (such as overexposed whites or underexposed grays) have lower weights (for noise suppression or invalid regions). The saturation weight for the same pixel is... and saturation weight Normalization is performed so that the weights sum to 1; the underexposed image M and the overexposed image N are fused according to the contrast weights (Z1=M*). +N* The first fused image Z1 is obtained; the underexposed image M and the overexposed image N are fused according to the saturation weight (Z2=M*). +N* The second fused image Z2 is obtained, and the first image is obtained by averaging the initial image A, the first fused image Z1, and the second fused image Z2.

[0089] In one embodiment, the feature extraction module includes:

[0090] The feature extraction module includes a first convolutional layer, a first deep convolutional layer, a second deep convolutional layer, a third deep convolutional layer, and a fully connected layer;

[0091] The first image is sequentially passed through a first convolutional layer, a first deep convolutional layer, a second deep convolutional layer, a third deep convolutional layer, and a fully connected layer to obtain the first feature.

[0092] In one implementation, the first, second, and third depth convolutional layers have the same structure, with the first convolutional layer having a 3x3 kernel and a stride of 2.

[0093] In one implementation, the working principle of the deep convolutional layer includes: acquiring an input image; substituting the image into a convolutional layer with a kernel of 3x3 and a stride of 2 to obtain a first convolutional feature; passing the first convolutional feature through two fully connected layers with identical structures to obtain a first fully connected feature and a second fully connected feature; multiplying the first and second fully connected features and then passing them through another fully connected layer with identical structures to obtain a third fully connected feature; substituting the third fully connected feature into a convolutional layer with a kernel of 3x3 and a stride of 2 to obtain a second convolutional feature; and averaging the first and second convolutional features to obtain a first feature.

[0094] In one embodiment, the feature extraction module includes a first edge feature extraction module, a second edge feature extraction module, and a second convolutional layer;

[0095] The first feature is sequentially processed by the first edge feature extraction module and the second edge feature extraction module to obtain the first edge feature and the second edge feature;

[0096] The first edge feature and the second edge feature are fused to obtain the fused edge feature, and the fused edge feature is substituted into the second convolutional layer to obtain the first enhanced feature;

[0097] The first edge feature extraction module and the second edge feature extraction module have the same structure; the edge feature extraction module includes a third convolutional layer, a Sobel convolutional layer and a fourth convolutional layer;

[0098] The working principle of the edge feature extraction module is as follows:

[0099] Obtain the input features, and substitute the input features into the third convolutional layer and the Sobel convolutional layer respectively to obtain the third convolutional features and the Sobel convolutional features;

[0100] The third convolutional feature and the Sobel convolutional feature are fused to obtain the Sobel fused feature;

[0101] Substituting the Sobel fusion features into the fourth convolutional layer yields the output features of the edge feature extraction module.

[0102] In one implementation, there is a second convolutional layer (with a 3x3 kernel and a stride of 2), a third convolutional layer (with a 3x3 kernel and a stride of 1), and a fourth convolutional layer (with a 1x1 kernel and a stride of 1).

[0103] In one implementation, edge information at different scales and levels of abstraction can be captured by using two edge feature extraction modules with identical structures but independent parameters (first and second edge feature extraction modules).

[0104] In one implementation, the first edge feature extraction module and the second edge feature extraction module have the same structure; the edge feature extraction module includes a third convolutional layer, a Sobel convolutional layer, and a fourth convolutional layer; the working principle of the edge feature extraction module is as follows: obtain input features, substitute the input features into the third convolutional layer and the Sobel convolutional layer respectively to obtain the third convolutional features and the Sobel convolutional features; fuse the third convolutional features and the Sobel convolutional features to obtain the Sobel fused features; substitute the Sobel fused features into the fourth convolutional layer to obtain the output features of the edge feature extraction module.

[0105] In one implementation, the Sobel operator calculates the gradient magnitude through horizontal and vertical convolution kernels, which strongly responds to step edges in the image (such as object contours and material boundaries). Compared with traditional CNNs that learn edge features implicitly through training, the Sobel operator directly extracts edges based on prior geometric knowledge, which has clear physical meaning and can quickly locate the boundaries of foreign objects.

[0106] In one implementation, a data-driven edge pattern is adaptively learned using a 3×3 convolutional kernel to supplement non-ideal edges that the Sobel operator cannot capture. The shallow layer (first module) extracts fine-grained edges (such as sharp edges of small foreign objects and pixel-level noisy edges) on the high-resolution feature map. The deep layer (second module) extracts coarse-grained edges (such as the overall outline of large foreign objects and cross-regional structural edges) on the low-resolution feature map. The shallow edge features retain pixel-level localization accuracy, while the deep edge features provide semantic understanding. When a foreign object is partially occluded, the shallow module can detect local edges in the unoccluded area, while the deep module infers the overall shape through the global edge distribution, improving detection robustness.

[0107] In one implementation, a learnable third convolutional layer is introduced before the Sobel operator. This layer can learn noise patterns (such as Gaussian noise or salt-and-pepper noise) through training, generate a filter kernel to suppress specific noise, and avoid the Sobel operator from over-responding to noise (the traditional Sobel operator is prone to misjudging noise points as edges).

[0108] In one embodiment, the pooling enhancement feature module includes a max pooling layer, a fourth depthwise convolution, a fifth depthwise convolution, a first extended depthwise convolution, and a second extended depthwise convolution.

[0109] The first enhanced feature is sequentially passed through the fourth depth convolution, the fifth depth convolution, the first expanded depth convolution, and the second expanded depth convolution to obtain the target expanded depth convolution feature;

[0110] Multiply the first enhanced feature and the target extended depth convolutional feature to obtain the fused extended depth convolutional feature;

[0111] Substituting the first enhancement feature into the max pooling layer yields the pooling enhancement feature. After fusing the fused extended depth convolution feature and the pooling enhancement feature, a 1x1 convolution is performed to obtain the first pooling enhancement feature.

[0112] In one implementation, the kernel of the fourth depthwise convolution is 1×(2d−1), the kernel of the fifth depthwise convolution is (2d−1)×1, the kernel of the first extended depthwise convolution is 1×k / d, and the kernel of the second extended depthwise convolution is k / d×1; k represents the maximum receptive field, and d represents the dilation rate. The first enhancement feature is decomposed into a (2d−1)×(2d−1) depthwise convolution and a k / d×k / d extended depthwise convolution. Subsequently, the two-dimensional depthwise convolution kernel and the extended depthwise convolution kernel are further decomposed into one-dimensional horizontal (lateral) and vertical (vertical) convolution kernels. Then, these decomposed kernels are concatenated to maintain high computational efficiency when using large convolution kernels, effectively capturing a wide range of contextual information in the image.

[0113] In one implementation, a standard deep convolution (3×3 kernels, stride 1) is used to extract more abstract feature patterns layer by layer based on the first enhanced feature. By stacking and deepening the network layers, long-distance dependencies between features are captured.

[0114] In one implementation, feature complementarity is achieved through parallel branches: Branch 1 (deep convolution sequence): captures multi-scale context by expanding convolution; Branch 2 (max pooling layer): extracts the maximum value of local regions, enhances significant responses in the feature map (such as high-confidence regions of foreign objects), and suppresses background noise.

[0115] In one implementation, element-wise multiplication "weights" the multi-scale contextual information captured by the extended convolution onto the original feature map, highlighting feature regions related to foreign objects. If the extended convolution detects "abnormal shape" semantics (such as irregular contours) in a certain region, the multiplication operation will enhance the original features of that region (such as edge responses), forming an effect similar to an attention mask. Max pooling provides strong response features for local regions (such as the core region of foreign objects), which complements the global semantics of the extended convolution, avoiding the dilution of small target features due to the excessively large receptive field of the extended convolution.

[0116] In one embodiment, the foreign object detection module includes:

[0117] The first splicing feature determination module is used to upsample the first pooling enhancement feature and splice it with the first feature to obtain the first splicing feature;

[0118] The first splicing enhancement feature determination module is used to substitute the first splicing feature into the second feature enhancement model to obtain the first splicing enhancement feature;

[0119] The second splicing feature determination module is used to upsample the first splicing enhancement feature and then splice it with the first feature to obtain the second splicing feature;

[0120] The third splicing feature determination module is used to perform a 1x1 convolution on the second splicing feature and then splice it with the first splicing enhancement feature to obtain the third splicing feature;

[0121] The second splicing enhancement feature determination module is used to substitute the third splicing feature into the second feature enhancement model to obtain the second splicing enhancement feature;

[0122] The fourth splicing feature determination module is used to perform a 1x1 convolution on the second splicing enhancement feature and then splice it with the first enhancement feature to obtain the fourth splicing feature;

[0123] The target detection result determination module is used to substitute the second stitching feature, the second stitching enhancement feature, and the fourth stitching feature into the detection head to obtain the target detection result.

[0124] In one implementation, the first feature comes from the initial convolutional layer, which retains high-resolution spatial details (such as the pixel-level position of the foreign object edge), but has weak semantic information (only includes low-level features such as edges and textures); the first pooling enhancement feature comes from the deep network, which has rich semantic information (such as the overall shape concept of "foreign object"), but has low resolution (downsampling leads to loss of details). By concatenating through the channel dimension, the fusion of high-resolution details and high-level semantics is achieved, making foreign object detection more accurate.

[0125] In one implementation, a second feature enhancement model is used to strengthen the foreign object-related response in the spliced ​​features and suppress background noise. The final input features to the detection head include a second spliced ​​feature: high resolution, responsible for small foreign object detection; a second spliced ​​enhanced feature: medium resolution, responsible for medium-sized foreign object detection; and a fourth spliced ​​feature: low resolution, responsible for large foreign object detection. The fourth spliced ​​feature integrates the enhanced high-level features with the first enhanced features in the middle layer, realizing comparative learning of features before and after enhancement.

[0126] In one implementation, when a foreign object is partially occluded, the bottom-level splicing features can locate the target through details in the unoccluded area, and the high-level features can complete the overall shape through semantic reasoning. For deformed objects (such as bent metal wires), different levels of features can capture local edges (bottom level) and global curves (high level), improving detection consistency and increasing the accuracy of foreign object detection.

[0127] One implementation method solves the problem of small target detection in traditional deep networks by multi-level cross-layer fusion, while improving the positioning accuracy of large targets.

[0128] In one embodiment, see Figure 2 , Figure 2 A framework diagram of a second feature enhancement model is proposed, including:

[0129] The local pooling module is used to perform local pooling on the input features to obtain locally pooled features;

[0130] The global pooling module is used to perform global pooling on local pooling features to obtain global pooling features;

[0131] The pooling feature convolution module is used to perform one-dimensional convolution on local pooling features and global pooling features respectively to obtain target local pooling features and target global pooling features;

[0132] The attention weight map generation module is used to upsample the target local pooling features and the target global pooling features respectively to obtain the global attention weight map and the local spatial attention weight map;

[0133] The element-wise addition module is used to add the global attention weight map and the local spatial attention weight map element by element to obtain the target attention weight;

[0134] The feature weighting module is used to multiply the target attention weights by the input features to obtain the output features of the second feature enhancement model.

[0135] In one implementation, local pooling is achieved using a 3×3 convolution kernel; global average / max pooling is used to compress the entire feature map into a global vector, extracting cross-regional contextual information (such as the relative position of the foreign object in the image and its overall difference from the background); one-dimensional convolution is applied to the local / global pooling features; local pooling features: learn the local feature correlations between different channels through one-dimensional convolution (such as the co-activation mode of edge response channels and texture channels); global pooling features: extract channel-level global semantic dependencies (such as the feature channel combination corresponding to the semantics of "foreign object").

[0136] In one implementation, the one-dimensional vector after global pooling is upsampled to the feature map size to generate a channel-shared spatial attention mask, highlighting the global region related to the foreign object (such as the region with high activation value in the feature map). The one-dimensional convolution result based on the local pooling feature is upsampled to generate a pixel-level spatial attention mask, capturing subtle differences in local regions (such as the pixel-level position of the foreign object edge). The global weight map provides the probability distribution of the presence of the foreign object, and the local weight map refines the specific location.

[0137] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A belt foreign object visual recognition system based on image processing, characterized in that, The system includes: An image enhancement module is used to acquire an initial image and enhance the initial image using a preset illumination enhancement algorithm to obtain a first image; The feature extraction module is used to substitute the first image into the feature extraction module to obtain the first feature; A feature enhancement module is used to substitute the first feature into the feature enhancement module to obtain a first enhanced feature; A pooling enhancement feature module is used to substitute the first enhancement feature into the pooling enhancement feature module to obtain the first pooling enhancement feature; The foreign object recognition module is used to substitute the first feature, the first enhanced feature, and the first pooling enhanced feature into the foreign object detection model to obtain the target detection result; The target detection result includes the target detection bounding box and the confidence score; The image enhancement module includes: The grayscale conversion module is used to perform Gaussian blurring on the initial image and then perform grayscale conversion to obtain a grayscale image. An initial weight matrix construction module is used to initialize the connection weights between a target pixel and its neighboring pixels to obtain an initial weight matrix for the grayscale image; the target pixel is any pixel in the grayscale image. An updated weight matrix construction module is used to calculate the gradient magnitude between the target pixel and its neighboring pixels to obtain an updated weight matrix; the initial weight matrix is ​​updated based on the updated weight matrix. The target illumination image generation module is used to obtain a sparse Laplacian matrix based on the initial weight matrix updated for all pixels, and to fuse the initial image and the sparse Laplacian matrix through sparse Cholesky decomposition to obtain the target illumination image. The first image generation module is used to correct the initial image based on the target illumination image to obtain a first image; The first image generation module includes: An underexposed image generation module is used to obtain an underexposed image by dividing the initial image by the target illumination image; An overexposed image generation module is used to process the initial image and the target illumination image using a preset algorithm to obtain an overexposed image; The contrast weight determination module is used to calculate the Laplacian response corresponding to the underexposed image and the overexposed image respectively, and determine a first contrast weight and a second contrast weight based on the Laplacian response of each pixel in the underexposed image and the overexposed image; the first contrast weight is the weight corresponding to each pixel in the underexposed image; the second contrast weight is the weight corresponding to each pixel in the overexposed image; The first fused image determination module is used to fuse the underexposed image and the overexposed image according to the first contrast weight and the second contrast weight to obtain a first fused image; The saturation weight determination module is used to perform HSV color space transformation on both the underexposed image and the overexposed image. For the transformed image, it calculates the saturation of each pixel in each image to obtain a first saturation weight and a second saturation weight. The first saturation weight is the weight corresponding to the pixel in the underexposed image. The second saturation weight is the weight corresponding to the pixel in the overexposed image. The second fused image determination module is used to fuse the underexposed image and the overexposed image according to the first saturation weight and the second saturation weight to obtain a second fused image; The first image generation module is used to superimpose the initial image, the first fused image, and the second fused image and calculate the average to obtain the first image.

2. The image processing-based visual recognition system for foreign objects on a belt according to claim 1, characterized in that, The feature extraction module includes a first convolutional layer, a first deep convolutional layer, a second deep convolutional layer, a third deep convolutional layer, and a fully connected layer; The first image is sequentially passed through the first convolutional layer, the first deep convolutional layer, the second deep convolutional layer, the third deep convolutional layer, and the fully connected layer to obtain the first feature.

3. The image processing-based visual recognition system for foreign objects on a belt according to claim 1, characterized in that, The feature enhancement module includes a first edge feature extraction module, a second edge feature extraction module, and a second convolutional layer; The first feature is sequentially processed by the first edge feature extraction module and the second edge feature extraction module to obtain the first edge feature and the second edge feature; The first edge feature and the second edge feature are fused to obtain a fused edge feature, and the fused edge feature is substituted into the second convolutional layer to obtain a first enhanced feature.

4. The image processing-based visual recognition system for foreign objects on a belt according to claim 1, characterized in that, The pooling enhancement feature module includes a max pooling layer, a fourth depthwise convolution, a fifth depthwise convolution, a first extended depthwise convolution, and a second extended depthwise convolution. The first enhanced feature is sequentially passed through the fourth depth convolution, the fifth depth convolution, the first extended depth convolution, and the second extended depth convolution to obtain the target extended depth convolution feature; Multiply the first enhanced feature and the target extended depth convolutional feature to obtain the fused extended depth convolutional feature; Substituting the first enhancement feature into the max pooling layer yields the pooling enhancement feature. The fused extended depth convolution feature and the pooling enhancement feature are then fused and subjected to a 1x1 convolution to obtain the first pooling enhancement feature.

5. The image processing-based visual recognition system for foreign objects on a belt according to claim 1, characterized in that, The foreign object detection module includes: The first splicing feature determination module is used to upsample the first pooling enhancement feature and then splice it with the first feature to obtain the first splicing feature; The first splicing enhancement feature determination module is used to substitute the first splicing feature into the second feature enhancement model to obtain the first splicing enhancement feature; The second splicing feature determination module is used to upsample the first splicing enhancement feature and then splice it with the first feature to obtain the second splicing feature; The third splicing feature determination module is used to perform a 1x1 convolution on the second splicing feature and then splice it with the first splicing enhancement feature to obtain the third splicing feature; The second splicing enhancement feature determination module is used to substitute the third splicing feature into the second feature enhancement model to obtain the second splicing enhancement feature; The fourth splicing feature determination module is used to perform a 1x1 convolution on the second splicing enhancement feature and then splice it with the first enhancement feature to obtain the fourth splicing feature; The target detection result determination module is used to substitute the second splicing feature, the second splicing enhancement feature and the fourth splicing feature into the detection head to obtain the target detection result.

6. The image processing-based visual recognition system for foreign objects on a belt according to claim 5, characterized in that, The second feature enhancement model includes: The local pooling module is used to perform local pooling on the input features to obtain locally pooled features; A global pooling module is used to perform global pooling on the local pooling features to obtain global pooling features; The pooling feature convolution module is used to perform one-dimensional convolution on the local pooling features and the global pooling features respectively to obtain the target local pooling features and the target global pooling features; The attention weight map generation module is used to upsample the target local pooling features and the target global pooling features respectively to obtain a global attention weight map and a local spatial attention weight map; The element-wise addition module is used to add the global attention weight map and the local spatial attention weight map element by element to obtain the target attention weight; The feature weighting module is used to multiply the target attention weights by the input features to obtain the output features of the second feature enhancement model.

Images