Belt foreign matter visual identification system based on image processing

Abstract

The invention discloses a belt foreign matter visual identification system based on image processing, and relates to the technical field of image processing. The image enhancement module obtains an initial image, enhances the initial image through a preset illumination enhancement algorithm to obtain a first image, and substitutes the first image into the feature extraction module to obtain a first feature; the feature enhancement module substitutes the first feature into the feature enhancement module to obtain a first enhanced feature; the pooling enhancement feature module substitutes the first enhancement feature into the pooling enhancement feature to obtain a first pooling enhancement feature; and the foreign matter recognition module substitutes the first feature, the first enhanced feature and the first pooling enhanced feature into a foreign matter detection model to obtain a target detection result. The problems of uneven illumination, low contrast and the like of the initial image are corrected through the preset illumination enhancement algorithm, foreign matter leak detection caused by light interference is avoided, then multi-stage feature extraction fusion is performed on the first image, so that confidence evaluation of the model on the detection result is more reliable, and the accuracy and efficiency of foreign matter detection are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a belt foreign body visual recognition system based on image processing. Background Art

[0002] In modern coal mining operations, belt conveyor systems, the primary method of transporting coal, have a significant impact on the stability and reliability of the entire production system. The continuous and stable operation of belt conveyors not only impacts coal production efficiency but is also crucial for ensuring the safety of underground workers. However, in actual production, due to complex geological conditions and mining process limitations, various foreign objects such as gangue, anchor bolts, angle iron, and large lumps of coal can easily become mixed into the raw coal. If these foreign objects are not promptly identified and addressed, they can cause a series of serious operational failures: at the very least, belt deviation and increased equipment wear can occur, while at worst, they can lead to major accidents such as belt tearing and drive system blockage. These accidents not only affect the safe and efficient operation of the belt conveyor but can also threaten the safe production of the entire mine.

[0003] Patent CN119006466B discloses a method for detecting foreign matter in a coal mine belt conveyor. The method obtains an RGB image and a grayscale image of the coal mine. Based on the color deviation of each pixel in each connected domain of the grayscale image, the local disorder degree within the neighborhood of each pixel is obtained. Based on the theoretical centroid position of the disorder degree within each connected domain of the grayscale image, the probability of the presence of metal minerals in each connected domain is obtained. Based on the enhanced color value of the first color channel of each pixel in the RGB image, an enhanced RGB image of the coal mine is obtained. Metal foreign matter in the coal mine conveyor belt is then identified. This method makes foreign matter detection more accurate. However, when the quality of the acquired image is low, the accuracy of metal foreign matter detection is still not high enough, resulting in low efficiency of foreign matter detection. Summary of the Invention

[0004] The purpose of the present invention is to solve the problem that when the quality of the collected images is low, the accuracy of metal foreign body detection is still not high enough, resulting in low efficiency of foreign body detection, and to propose a belt foreign body visual recognition system based on image processing.

[0005] The present invention proposes a belt foreign body visual recognition system based on image processing, the system comprising:

[0006] An image enhancement module is used to obtain an initial image and enhance the initial image using a preset illumination enhancement algorithm to obtain a first image;

[0007] a feature extraction module, configured to substitute the first image into the feature extraction module to obtain a first feature;

[0008] a feature enhancement module, configured to substitute the first feature into the feature enhancement module to obtain a first enhanced feature;

[0009] A pooling enhancement feature module, configured to substitute the first enhancement feature into the pooling enhancement feature module to obtain a first pooling enhancement feature;

[0010] A foreign object recognition module is used to substitute the first feature, the first enhanced feature and the first pooled enhanced feature into a foreign object detection model to obtain a target detection result; the target detection result includes a target detection box and a confidence level.

[0011] Optionally, the image enhancement module includes:

[0012] A grayscale conversion module, configured to perform Gaussian blurring on the initial image and then perform grayscale conversion to obtain a grayscale image;

[0013] An initial weight matrix construction module is used to initialize the connection weights between the target pixel and the adjacent pixels of the grayscale image to obtain an initial weight matrix; the target pixel is any pixel in the grayscale image;

[0014] An update weight matrix construction module is used to calculate the gradient amplitude between the target pixel and the adjacent pixel points to obtain an updated weight matrix; and update the initial weight matrix according to the updated weight matrix;

[0015] A target illumination image generation module is configured to obtain a sparse Laplacian matrix based on the updated initial weight matrix of all pixels, and fuse the initial image and the sparse Laplacian matrix through sparse Cholesky decomposition to obtain a target illumination image;

[0016] The first image generation module is configured to correct the initial image according to the target illumination image to obtain a first image.

[0017] Optionally, the first image generation module includes:

[0018] an underexposed image generating module, configured to obtain an underexposed image by dividing the initial image by the target illumination image;

[0019] an overexposed image generation module, configured to process the initial image and the target illumination image using a preset algorithm to obtain an overexposed image;

[0020] a contrast weight determination module, configured to calculate the Laplace responses corresponding to the underexposed image and the overexposed image, respectively, and determine a first contrast weight and a second contrast weight based on the Laplace responses corresponding to each pixel in the underexposed image and the overexposed image; the first contrast weight being the weight corresponding to each pixel in the underexposed image; and the second contrast weight being the weight corresponding to each pixel in the overexposed image;

[0021] a first fused image determining module, configured to fuse the underexposed image and the overexposed image according to the first contrast weight and the second contrast weight to obtain a first fused image;

[0022] a saturation weight determination module, configured to transform both the underexposed image and the overexposed image in the HSV color space, and, for the transformed images, calculate the saturation of each pixel in each image to obtain a first saturation weight and a second saturation weight; the first saturation weight being the weight corresponding to the pixel in the underexposed image; and the second saturation weight being the weight corresponding to the pixel in the overexposed image;

[0023] a second fused image determining module, configured to fuse the underexposed image and the overexposed image according to the first saturation weight and the second saturation weight to obtain a second fused image;

[0024] The first image generation module is configured to superimpose and average the initial image, the first fused image, and the second fused image to obtain a first image.

[0025] Optionally, the feature extraction module includes a first convolutional layer, a first depthwise convolutional layer, a second depthwise convolutional layer, a third depthwise convolutional layer and a fully connected layer;

[0026] The first image sequentially passes through the first convolutional layer, the first depthwise convolutional layer, the second depthwise convolutional layer, the third depthwise convolutional layer and the fully connected layer to obtain a first feature.

[0027] Optionally, the feature enhancement module includes a first edge feature extraction module, a second edge feature extraction module and a second convolutional layer;

[0028] The first feature is sequentially processed by the first edge feature extraction module and the second edge feature extraction module to obtain a first edge feature and a second edge feature;

[0029] Perform feature fusion on the first edge feature and the second edge feature to obtain a fused edge feature, and substitute the fused edge feature into the second convolutional layer to obtain a first enhanced feature.

[0030] Optionally, the pooling enhancement feature module includes a maximum pooling layer, a fourth depth convolution, a fifth depth convolution, a first extended depth convolution, and a second extended depth convolution;

[0031] Sequentially subjecting the first enhanced feature to the fourth depthwise convolution, the fifth depthwise convolution, the first extended depthwise convolution, and the second extended depthwise convolution to obtain a target extended depthwise convolution feature;

[0032] Multiplying the first enhanced feature and the target extended depth convolution feature to obtain a fused extended depth convolution feature;

[0033] The first enhancement feature is substituted into the maximum pooling layer to obtain a pooling enhancement feature, and the fused extended depth convolution feature and the pooling enhancement feature are fused and then 1x1 convolution is performed to obtain a first pooling enhancement feature.

[0034] Optionally, the foreign object recognition module includes:

[0035] A first splicing feature determination module is used to upsample the first pooled enhanced feature and then splice it with the first feature to obtain a first splicing feature;

[0036] A first splicing enhancement feature determination module is configured to substitute the first splicing feature into a second feature enhancement model to obtain a first splicing enhancement feature;

[0037] a second splicing feature determination module, configured to upsample the first splicing enhancement feature and then splice it with the first feature to obtain a second splicing feature;

[0038] A third splicing feature determination module is used to perform a 1x1 convolution on the second splicing feature and then splice it with the first splicing enhancement feature to obtain a third splicing feature;

[0039] A second splicing enhancement feature determination module is used to substitute the third splicing feature into the second feature enhancement model to obtain a second splicing enhancement feature;

[0040] a fourth splicing feature determination module, configured to perform a 1x1 convolution on the second splicing enhancement feature and then splice the concatenated feature with the first enhancement feature to obtain a fourth splicing feature;

[0041] The target detection result determination module is used to substitute the second splicing feature, the second splicing enhancement feature and the fourth splicing feature into the detection head to obtain the target detection result.

[0042] Optionally, the second feature enhancement model includes:

[0043] The local pooling module is used to locally pool the input features to obtain local pooling features;

[0044] A global pooling module, configured to perform global pooling on the local pooling features to obtain global pooling features;

[0045] A pooling feature convolution module is used to perform one-dimensional convolution on the local pooling feature and the global pooling feature to obtain a target local pooling feature and a target global pooling feature;

[0046] An attention weight map generation module is used to upsample the target local pooling features and the target global pooling features respectively to obtain a global attention weight map and a local spatial attention weight map;

[0047] An element addition module, configured to perform element-by-element addition of the global attention weight map and the local spatial attention weight map to obtain a target attention weight;

[0048] A feature weighting module is used to multiply the target attention weight by the input feature to obtain the output feature of the second feature enhancement model.

[0049] Beneficial effects of the present invention:

[0050] The present invention proposes a belt foreign body visual recognition system based on image processing, comprising an image enhancement module for acquiring an initial image and enhancing the initial image through a preset illumination enhancement algorithm to obtain a first image; a feature extraction module for substituting the first image into the feature extraction module to obtain a first feature; a feature enhancement module for substituting the first feature into the feature enhancement module to obtain a first enhanced feature; a pooling enhancement feature module for substituting the first enhanced feature into the pooling enhancement feature to obtain a first pooling enhancement feature; and a foreign body recognition module for substituting the first feature, the first enhanced feature, and the first pooling enhancement feature into a foreign body detection model to obtain a target detection result; the target detection result includes a target detection frame and a confidence level. A preset illumination enhancement algorithm is used to correct problems such as uneven illumination and low contrast in the initial image, restore true colors and details, and avoid missed detection of foreign bodies due to light interference. Multi-stage feature extraction and fusion are then performed on the first image to make the model's confidence assessment of the detection result more reliable, thereby improving the accuracy and efficiency of foreign body detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The present invention will be further described below with reference to the accompanying drawings.

[0052] Figure 1 A framework diagram of a belt foreign body visual recognition system based on image processing provided by an embodiment of the present invention;

[0053] Figure 2 A framework diagram of a second feature enhancement model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0055] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0056] The embodiment of the present invention provides a belt foreign body visual recognition system based on image processing. Figure 1 , Figure 1 This is a framework diagram of a belt foreign body visual recognition system based on image processing provided by an embodiment of the present invention. The system includes:

[0057] An image enhancement module is used to obtain an initial image and enhance the initial image using a preset illumination enhancement algorithm to obtain a first image;

[0058] A feature extraction module, configured to substitute the first image into the feature extraction module to obtain a first feature;

[0059] A feature enhancement module, configured to substitute the first feature into the feature enhancement module to obtain a first enhanced feature;

[0060] A pooling enhancement feature module, configured to substitute the first enhancement feature into the pooling enhancement feature module to obtain the first pooling enhancement feature;

[0061] The foreign body recognition module is used to substitute the first feature, the first enhanced feature and the first pooled enhanced feature into the foreign body detection model to obtain a target detection result; the target detection result includes a target detection box and a confidence level.

[0062] A belt foreign body visual recognition system based on image processing provided by an embodiment of the present invention uses a preset lighting enhancement algorithm to correct problems such as uneven lighting and low contrast in the initial image, restore true colors and details, and avoid missed detection of foreign bodies due to light interference. By performing multi-stage feature extraction and fusion on the first image, the model's confidence assessment of the detection results is made more reliable, thereby improving the accuracy and efficiency of foreign body detection.

[0063] In one implementation, a preset algorithm is used to correct problems such as uneven lighting and low contrast in the initial image, restore the true colors and details, and avoid missing foreign objects due to light interference (such as tiny impurities covered by shadows).

[0064] In one implementation, the initial image is an image captured by a camera on a belt transporting minerals in a mine. Since the light intensity in the mine is low, additional lights are used to compensate for the light intensity, which also causes the captured image to be greatly affected by low exposure and high exposure.

[0065] In one implementation, the feature extraction module captures underlying features such as edges and textures in the image, and then the feature enhancement module further amplifies effective features (such as the shape differences of foreign objects) and suppresses background noise (such as interference from uniform textures), significantly improving the distinction between foreign object features and normal areas.

[0066] In one implementation, when the confidence level is greater than a preset threshold, the target detection result is considered abnormal, an alarm is issued, and the target detection frame is sent to a display to remind technicians to remove it.

[0067] In one implementation, the scale invariance of the feature map is enhanced through pooling operations (such as adaptive pooling and hole pooling) to ensure that foreign objects of different sizes and positions can be stably detected. In coal mine conveyor belt detection, objects may move due to the movement of the conveyor belt. Pooling enhancement can avoid missed detections due to feature misalignment and improve detection stability.

[0068] In one embodiment, the image enhancement module includes:

[0069] Grayscale conversion module, used to perform Gaussian blur on the initial image and then convert it into grayscale to obtain a grayscale image;

[0070] The initial weight matrix construction module is used to initialize the connection weights between the target pixel and the adjacent pixels in the grayscale image to obtain the initial weight matrix; the target pixel is any pixel in the grayscale image;

[0071] The weight matrix update construction module is used to calculate the gradient magnitude between the target pixel and the adjacent pixels to obtain the updated weight matrix; the initial weight matrix is updated according to the updated weight matrix;

[0072] The target illumination image generation module is used to obtain a sparse Laplacian matrix based on the updated initial weight matrix of all pixels, and fuse the initial image and the sparse Laplacian matrix through sparse Cholesky decomposition to obtain the target illumination image;

[0073] The first image generation module is used to correct the initial image according to the target illumination image to obtain a first image.

[0074] In one implementation, the initial image (A) is Gaussian blurred (kernel size 3×3) to reduce the interference of high-frequency noise on the illumination estimation, and then the image is converted into a grayscale image to retain the brightness information and simplify the calculation; the connection relationship between the target pixel (any pixel in the grayscale image) and the adjacent pixels (8 neighborhoods) is initialized (that is, the weight between adjacent pixels is set to 1), the gradient amplitude of the target pixel (Sobel operator) is calculated, and the weight between the target pixel and the adjacent pixels is changed according to the gradient amplitude, where the area with a larger gradient amplitude (usually an edge or detail) has a smaller weight (allowing more drastic illumination changes and retaining edge details), and the Sobel operator of the adjacent pixels and the target pixel is calculated, and then the values are sorted from small to large. Sort to get the first sorted set; then sum up all the calculated amplitudes to get the total value, get the weighted sorted set according to the proportion of each data in the total value in the first sorted set, reverse the weighted sorted set, and use it as the weight of the corresponding adjacent pixels in the first sorted set. For example, the order of the pixels in the first sorted set is (S1, S2, S3, S4, S5, S6, S7, S8, S9), then their corresponding weights should be (S9's proportion in the total value, S8's proportion in the total value, S7's proportion in the total value, S6's proportion in the total value, S5's proportion in the total value, S4's proportion in the total value, S3's proportion in the total value, S2's proportion in the total value, S1's proportion in the total value); the smaller the gradient amplitude (such as the smooth surface of the belt), the greater the weight (forced lighting smoothing, noise suppression).

[0075] In one implementation, for the weighted grayscale image, the gradient magnitudes of all pixels are calculated to obtain a sparse Laplacian matrix. The initial image and the sparse Laplacian matrix are processed using sparse Cholesky decomposition (B = A + λF, where λ is a preset weight, typically 0.5, ranging from 0 to 1. The larger the λ, the smoother the illumination map (suitable for scenes with high noise levels), and the smaller the λ, the closer the illumination map is to the original image (suitable for scenes with uniform lighting). F is a sparse Laplacian matrix) to obtain the illumination map B.

[0076] In one implementation, the initial image is convolved with a Gaussian kernel to smooth out sharp jumps between pixels and reduce interference from sensor noise and texture details. The weight of pixel connections across edges is automatically reduced at object boundaries to avoid incorrect association of foreground and background during lighting estimation. The weight of similar texture pixels is reduced in high-texture areas to prevent lighting estimation bias caused by repeated calculations.

[0077] In one embodiment, the first image generation module includes:

[0078] An underexposed image generation module, configured to obtain an underexposed image by dividing the initial image by the target illumination image;

[0079] An overexposed image generation module, used to process the initial image and the target illumination image using a preset algorithm to obtain an overexposed image;

[0080] a contrast weight determination module, configured to calculate the Laplace responses corresponding to the underexposed image and the overexposed image, respectively, and determine a first contrast weight and a second contrast weight based on the Laplace responses corresponding to each pixel in the underexposed image and the overexposed image; the first contrast weight is the weight corresponding to each pixel in the underexposed image; and the second contrast weight is the weight corresponding to each pixel in the overexposed image;

[0081] a first fused image determination module, configured to fuse the underexposed image and the overexposed image according to the first contrast weight and the second contrast weight to obtain a first fused image;

[0082] A saturation weight determination module is used to transform both the underexposed image and the overexposed image in the HSV color space. For the transformed image, the saturation of each pixel in each image is calculated to obtain a first saturation weight and a second saturation weight; the first saturation weight is the weight corresponding to the pixel in the underexposed image; the second saturation weight is the weight corresponding to the pixel in the overexposed image;

[0083] a second fused image determination module, configured to fuse the underexposed image and the overexposed image according to the first saturation weight and the second saturation weight to obtain a second fused image;

[0084] The first image generation module is configured to superimpose and average the initial image, the first fused image, and the second fused image to obtain a first image.

[0085] In one implementation, the dynamic range of the bright area is compressed through linear mapping or logarithmic transformation to prevent the pixel values in the overexposed area from being saturated to 255. Compared with traditional histogram truncation, this can retain the texture details of the metal reflection and avoid the loss of information in the overexposed area.

[0086] In one implementation, for underexposed images, the dark areas have low contrast and weak Laplace responses, so the weights need to be increased to enhance details. For overexposed images, the bright areas are prone to oversaturation, and the Laplace responses may be high due to noise amplification, so the weights need to be reduced to suppress artifacts. Underexposed images contribute dark details, while overexposed images retain bright structures. The optimal source is dynamically selected through contrast weighting to avoid the dynamic range limitations of a single image.

[0087] In one implementation, the underexposed image M is obtained by dividing the initial image A by the illumination map B (the pixel values after correction may exceed 1 and need to be truncated to the interval [0,1] to avoid oversaturation); the initial image A is normalized by the illumination map B, and the pixel values in the dark areas (small B values) are amplified (for example, when B=0.2, the pixels with A=0.1 are corrected to 0.5, and the brightness is increased by 5 times), thereby revealing the dark details (such as the outline of foreign objects in the shadows); the overexposed image N is obtained by the formula (preset algorithm) C=1-(1-A) / (1-B). For the bright areas (B is close to 1), the denominator 1-B approaches 0, thereby compressing 1-A (the pixel value of the overexposed area) and restoring the bright details (such as the texture of the reflective surface).

[0088] In one implementation, the Laplace response of the underexposed image M and the overexposed image N is calculated to obtain the contrast weight of the underexposed image: and the contrast weight of the overexposed image , rule: at the same position, if the response value of the underexposed image M is larger, the weight of the area is biased towards the underexposed image M (indicating that the dark area correction image retains more edge details), and the contrast weight corresponding to the same pixel point is and contrast weight Normalize so that the weight sum is 1; convert the image to HSV color space, calculate the saturation (S component) of each pixel to obtain the saturation weight of the underexposed image and the saturation weight of the overexposed image , rule: the lower the saturation area (such as overexposed white or underexposed gray), the lower the weight (suppressing noise or invalid area), the saturation weight corresponding to the same pixel and saturation weight Normalize the image so that the weight sum is 1; fuse the underexposed image M and overexposed image N according to the contrast weight (Z1=M* +N* ) to obtain the first fused image Z1; according to the saturation weight, the underexposed image M and the overexposed image N are fused (Z2=M* +N* ) to obtain the second fused image Z2, and the initial image A, the first fused image Z1 and the second fused image Z2 are averaged to obtain the first image.

[0089] In one embodiment, the feature extraction module includes:

[0090] The feature extraction module includes a first convolutional layer, a first depthwise convolutional layer, a second depthwise convolutional layer, a third depthwise convolutional layer, and a fully connected layer;

[0091] The first image passes through the first convolution layer, the first depth convolution layer, the second depth convolution layer, the third depth convolution layer and the fully connected layer in sequence to obtain the first feature.

[0092] In one implementation, the first depth convolution layer, the second depth convolution layer, and the third depth convolution layer have the same structure, and the convolution kernel of the first convolution layer is 3x3 with a stride of 2.

[0093] In one implementation, the working principle of the deep convolutional layer includes obtaining an input image, substituting the image into a convolution layer with a convolution kernel of 3x3 and a stride of 2 to obtain a first convolution feature, and the first convolution feature passes through two fully connected layers with the same structure to obtain a first fully connected feature and a second fully connected feature; multiplying the first fully connected feature and the second fully connected feature and then passing through the fully connected feature layer with the same structure again to obtain a third fully connected feature, substituting the third fully connected feature into a convolution layer with a convolution kernel of 3x3 and a stride of 2 again to obtain a second convolution feature, and averaging the first convolution feature and the second convolution feature to obtain the first feature.

[0094] In one embodiment, the feature extraction module includes a first edge feature extraction module, a second edge feature extraction module, and a second convolutional layer;

[0095] The first feature is sequentially passed through a first edge feature extraction module and a second edge feature extraction module to obtain a first edge feature and a second edge feature;

[0096] Performing feature fusion on the first edge feature and the second edge feature to obtain a fused edge feature, and substituting the fused edge feature into the second convolutional layer to obtain a first enhanced feature;

[0097] The first edge feature extraction module and the second edge feature extraction module have the same structure; the edge feature extraction module includes a third convolution layer, a Sobel convolution layer and a fourth convolution layer;

[0098] The working principle of the edge feature extraction module is:

[0099] Obtain input features, and substitute the input features into the third convolution layer and the Sobel convolution layer respectively to obtain the third convolution feature and the Sobel convolution feature;

[0100] The third convolution feature and the Sobel convolution feature are fused to obtain the Sobel fusion feature;

[0101] Substitute the Sobel fusion features into the fourth convolutional layer to obtain the output features of the edge feature extraction module.

[0102] In one implementation, the second convolution layer (convolution kernel is 3x3, stride is 2), the third convolution layer (convolution kernel is 3x3, stride is 1), and the fourth convolution layer (convolution kernel is 1x1, stride is 1).

[0103] In one implementation, edge information of different scales and abstraction levels can be captured by two edge feature extraction modules (first and second edge feature extraction modules) with the same structure but independent parameters.

[0104] In one implementation, the first edge feature extraction module and the second edge feature extraction module have the same structure; the edge feature extraction module includes a third convolution layer, a Sobel convolution layer and a fourth convolution layer; the working principle of the edge feature extraction module is: obtaining input features, substituting the input features into the third convolution layer and the Sobel convolution layer respectively to obtain the third convolution feature and the Sobel convolution feature; fusing the third convolution feature and the Sobel convolution feature to obtain the Sobel fusion feature; substituting the Sobel fusion feature into the fourth convolution layer to obtain the output feature of the edge feature extraction module.

[0105] In one implementation, the Sobel operator calculates the gradient amplitude through horizontal and vertical convolution kernels, and responds strongly to step edges in the image (such as object contours and material boundaries). Compared with traditional CNNs that implicitly learn edge features through training, the Sobel operator directly extracts edges based on prior geometric knowledge, has clear physical meaning, and can quickly locate the boundaries of foreign objects.

[0106] In one implementation, a 3×3 convolution kernel is used to adaptively learn data-driven edge patterns to supplement non-ideal edges that cannot be captured by the Sobel operator; the shallow layer (first module) extracts fine-grained edges (such as the sharp edges of small-sized foreign objects and pixel-level noise edges) on the high-resolution feature map; the deep layer (second module) extracts coarse-grained edges (such as the overall contours of large-sized foreign objects and cross-regional structural edges) on the low-resolution feature map. The shallow edge features retain pixel-level positioning accuracy, and the deep edge features provide semantic understanding. When the foreign object is partially obscured, the shallow module can detect the local edges of the unobstructed area, and the deep module infers the overall shape through the global edge distribution to improve the detection robustness.

[0107] In one implementation, a learnable third convolutional layer is introduced before the Sobel operator. This layer can learn noise patterns (such as Gaussian noise and salt and pepper noise) through training, generating a filter kernel to suppress specific noise and prevent the Sobel operator from over-responding to noise (the traditional Sobel operator is prone to misjudging noise points as edges).

[0108] In one embodiment, the pooling enhancement feature module includes a maximum pooling layer, a fourth depthwise convolution, a fifth depthwise convolution, a first extended depthwise convolution, and a second extended depthwise convolution;

[0109] The first enhanced feature is sequentially subjected to the fourth depthwise convolution, the fifth depthwise convolution, the first extended depthwise convolution, and the second extended depthwise convolution to obtain the target extended depthwise convolution feature;

[0110] Multiplying the first enhanced feature and the target extended depth convolution feature to obtain a fused extended depth convolution feature;

[0111] The first enhanced feature is substituted into the maximum pooling layer to obtain the pooled enhanced feature, and the fused extended depth convolution feature and the pooled enhanced feature are fused and then 1x1 convolution is performed to obtain the first pooled enhanced feature.

[0112] In one implementation, the convolution kernel of the fourth depthwise convolution is 1×(2d−1), the convolution kernel of the fifth depthwise convolution is (2d−1)×1, the convolution kernel of the first extended depthwise convolution is 1×k / d, and the convolution kernel of the second extended depthwise convolution is k / d×1. k represents the maximum receptive field, and d represents the dilation rate. The first enhanced feature is decomposed into a (2d−1)×(2d−1) depthwise convolution and a k / d×k / d depthwise extended convolution. Subsequently, the two-dimensional depthwise convolution kernel and the depthwise extended convolution kernel are further decomposed into one-dimensional horizontal (horizontal) and vertical (vertical) convolution kernels. These decomposed kernels are then concatenated, maintaining high computational efficiency even when using large convolution kernels, effectively capturing a wide range of contextual information in the image.

[0113] In one implementation, standard depth convolution (3×3 kernel, stride 1) is used to extract more abstract feature patterns layer by layer based on the first enhanced features, and the network level is deepened by stacking to capture long-distance dependencies between features.

[0114] In one implementation, through feature complementarity of parallel branches, branch 1 (deep convolution sequence) captures multi-scale context through dilated convolution; branch 2 (maximum pooling layer) extracts the maximum value of the local area, strengthens the significant response in the feature map (such as the high confidence area of foreign objects), and suppresses background noise.

[0115] In one implementation, the multi-scale contextual information captured by the expanded convolution is "weighted" to the original feature map through element-level multiplication, highlighting the feature areas related to foreign objects. If the expanded convolution detects "abnormal shape" semantics (such as irregular contours) in a certain area, the multiplication operation will enhance the original features of the area (such as edge response), forming an effect similar to the attention mask; through the maximum pooling feature, strong response features of the local area (such as the core area of the foreign object) are provided, which complement the global semantics of the expanded convolution, avoiding the dilution of small target features due to the large receptive field of the expanded convolution.

[0116] In one embodiment, the foreign object recognition module includes:

[0117] A first splicing feature determination module is used to upsample the first pooling enhancement feature and then splice it with the first feature to obtain a first splicing feature;

[0118] A first splicing enhancement feature determination module is used to substitute the first splicing feature into the second feature enhancement model to obtain the first splicing enhancement feature;

[0119] A second splicing feature determination module is used to upsample the first splicing enhancement feature and then splice it with the first feature to obtain a second splicing feature;

[0120] A third splicing feature determination module is used to perform a 1x1 convolution on the second splicing feature and then splice it with the first splicing enhancement feature to obtain a third splicing feature;

[0121] A second splicing enhancement feature determination module is used to substitute the third splicing feature into the second feature enhancement model to obtain a second splicing enhancement feature;

[0122] a fourth splicing feature determination module, configured to perform a 1x1 convolution on the second splicing enhancement feature and then splice the concatenated feature with the first enhancement feature to obtain a fourth splicing feature;

[0123] The target detection result determination module is used to substitute the second splicing feature, the second splicing enhancement feature and the fourth splicing feature into the detection head to obtain the target detection result.

[0124] In one implementation, the first feature comes from the initial convolutional layer, which retains high-resolution spatial details (such as the pixel-level position of the edge of the foreign object), but has weak semantic information (only contains low-level features such as edges and textures); the first pooled enhanced feature comes from the deep network, which is rich in semantic information (such as the overall shape concept of "foreign objects"), but has low resolution (downsampling causes loss of details). Through channel dimension splicing, the fusion of high-resolution details and high-level semantics is achieved, making foreign object detection more accurate.

[0125] In one implementation method, the foreign body-related response in the splicing feature is strengthened through the second feature enhancement model to suppress background noise; the features finally input to the detection head include the second splicing feature: high resolution, responsible for small foreign body detection; the second splicing enhancement feature: medium resolution, responsible for medium-sized foreign body detection; the fourth splicing feature: low resolution, responsible for large foreign body detection; the fourth splicing feature integrates the enhanced high-level features with the first enhanced features in the middle layer to achieve comparative learning of features before and after enhancement.

[0126] In one implementation, when a foreign object is partially obscured, the underlying stitching features can locate the target through the details of the unobstructed area, and the high-level features can complete the overall shape through semantic reasoning. For deformed objects (such as bent wires), features at different levels can capture local edges (bottom layer) and global curves (top layer), thereby improving detection consistency and the accuracy of foreign object detection.

[0127] In one implementation, multi-level cross-layer fusion is used to solve the small target detection problem of traditional deep networks while improving the positioning accuracy of large targets.

[0128] In one embodiment, see Figure 2 , Figure 2 A framework diagram of the second feature enhancement model is proposed, including:

[0129] The local pooling module is used to locally pool the input features to obtain local pooling features;

[0130] The global pooling module is used to perform global pooling on the local pooling features to obtain the global pooling features;

[0131] The pooling feature convolution module is used to perform one-dimensional convolution on the local pooling features and the global pooling features to obtain the target local pooling features and the target global pooling features respectively;

[0132] The attention weight map generation module is used to upsample the target local pooling features and the target global pooling features respectively to obtain the global attention weight map and the local spatial attention weight map;

[0133] The element-wise addition module is used to add the global attention weight map and the local spatial attention weight map element by element to obtain the target attention weight;

[0134] The feature weighting module is used to multiply the target attention weight with the input feature to obtain the output feature of the second feature enhancement model.

[0135] In one implementation, local pooling is performed through a 3×3 convolution kernel; through global average / maximum pooling, the entire feature map is compressed into a global vector to extract cross-region contextual information (such as the relative position of the foreign object in the image and the overall difference from the background); one-dimensional convolution is applied to the local / global pooling features; local pooling features: learning the local feature correlation between different channels through one-dimensional convolution (such as the co-activation pattern of the edge response channel and the texture channel); global pooling features: extracting global semantic dependencies at the channel level (such as the feature channel combination corresponding to the "foreign object" semantics).

[0136] In one implementation, the one-dimensional vector after global pooling is upsampled to the feature map size to generate a channel-shared spatial attention mask, highlighting the global area related to foreign objects (such as areas with higher activation values in the feature map). The one-dimensional convolution result based on the local pooling feature is upsampled to generate a pixel-level spatial attention mask to capture subtle differences in local areas (such as the pixel-level position of the edge of the foreign object); the global weight map provides the probability distribution of the existence of foreign objects, and the local weight map refines the specific location.

[0137] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and should not be considered to limit the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A belt foreign body visual recognition system based on image processing, characterized in that: The system comprises: An image enhancement module is used to obtain an initial image and enhance the initial image using a preset illumination enhancement algorithm to obtain a first image; a feature extraction module, configured to substitute the first image into the feature extraction module to obtain a first feature; a feature enhancement module, configured to substitute the first feature into the feature enhancement module to obtain a first enhanced feature; A pooling enhancement feature module, configured to substitute the first enhancement feature into the pooling enhancement feature module to obtain a first pooling enhancement feature; A foreign object recognition module is used to substitute the first feature, the first enhanced feature and the first pooled enhanced feature into a foreign object detection model to obtain a target detection result; the target detection result includes a target detection box and a confidence level.

2. The belt foreign body visual recognition system based on image processing according to claim 1 is characterized in that: The image enhancement module includes: A grayscale conversion module, configured to perform Gaussian blurring on the initial image and then perform grayscale conversion to obtain a grayscale image; An initial weight matrix construction module is used to initialize the connection weights between the target pixel and the adjacent pixels of the grayscale image to obtain an initial weight matrix; the target pixel is any pixel in the grayscale image; An update weight matrix construction module is used to calculate the gradient amplitude between the target pixel and the adjacent pixel points to obtain an updated weight matrix; and update the initial weight matrix according to the updated weight matrix; A target illumination image generation module is configured to obtain a sparse Laplacian matrix based on the updated initial weight matrix of all pixels, and fuse the initial image and the sparse Laplacian matrix through sparse Cholesky decomposition to obtain a target illumination image; The first image generation module is configured to correct the initial image according to the target illumination image to obtain a first image.

3. The belt foreign body visual recognition system based on image processing according to claim 2 is characterized in that: The first image generation module includes: an underexposed image generating module, configured to obtain an underexposed image by dividing the initial image by the target illumination image; an overexposed image generation module, configured to process the initial image and the target illumination image using a preset algorithm to obtain an overexposed image; a contrast weight determination module, configured to calculate the Laplace responses corresponding to the underexposed image and the overexposed image, respectively, and determine a first contrast weight and a second contrast weight based on the Laplace responses corresponding to each pixel in the underexposed image and the overexposed image; the first contrast weight being the weight corresponding to each pixel in the underexposed image; and the second contrast weight being the weight corresponding to each pixel in the overexposed image; a first fused image determining module, configured to fuse the underexposed image and the overexposed image according to the first contrast weight and the second contrast weight to obtain a first fused image; a saturation weight determination module, configured to transform both the underexposed image and the overexposed image in the HSV color space, and, for the transformed images, calculate the saturation of each pixel in each image to obtain a first saturation weight and a second saturation weight; the first saturation weight being the weight corresponding to the pixel in the underexposed image; and the second saturation weight being the weight corresponding to the pixel in the overexposed image; a second fused image determining module, configured to fuse the underexposed image and the overexposed image according to the first saturation weight and the second saturation weight to obtain a second fused image; The first image generation module is configured to superimpose and average the initial image, the first fused image, and the second fused image to obtain a first image.

4. The belt foreign body visual recognition system based on image processing according to claim 1 is characterized in that: The feature extraction module includes a first convolutional layer, a first depthwise convolutional layer, a second depthwise convolutional layer, a third depthwise convolutional layer and a fully connected layer; The first image sequentially passes through the first convolutional layer, the first depthwise convolutional layer, the second depthwise convolutional layer, the third depthwise convolutional layer and the fully connected layer to obtain a first feature.

5. The belt foreign body visual recognition system based on image processing according to claim 1 is characterized in that: The feature enhancement module includes a first edge feature extraction module, a second edge feature extraction module and a second convolutional layer; The first feature is sequentially processed by the first edge feature extraction module and the second edge feature extraction module to obtain a first edge feature and a second edge feature; Perform feature fusion on the first edge feature and the second edge feature to obtain a fused edge feature, and substitute the fused edge feature into the second convolutional layer to obtain a first enhanced feature.

6. The belt foreign body visual recognition system based on image processing according to claim 1 is characterized in that: The pooling enhancement feature module includes a maximum pooling layer, a fourth depth convolution, a fifth depth convolution, a first extended depth convolution and a second extended depth convolution; Sequentially subjecting the first enhanced feature to the fourth depthwise convolution, the fifth depthwise convolution, the first extended depthwise convolution, and the second extended depthwise convolution to obtain a target extended depthwise convolution feature; Multiplying the first enhanced feature and the target extended depth convolution feature to obtain a fused extended depth convolution feature; The first enhancement feature is substituted into the maximum pooling layer to obtain a pooling enhancement feature, and the fused extended depth convolution feature and the pooling enhancement feature are fused and then 1x1 convolution is performed to obtain a first pooling enhancement feature.

7. The belt foreign body visual recognition system based on image processing according to claim 1 is characterized in that: The foreign object recognition module includes: A first splicing feature determination module is used to upsample the first pooled enhanced feature and then splice it with the first feature to obtain a first splicing feature; A first splicing enhancement feature determination module is configured to substitute the first splicing feature into a second feature enhancement model to obtain a first splicing enhancement feature; a second splicing feature determination module, configured to upsample the first splicing enhancement feature and then splice it with the first feature to obtain a second splicing feature; A third splicing feature determination module is used to perform a 1x1 convolution on the second splicing feature and then splice it with the first splicing enhancement feature to obtain a third splicing feature; A second splicing enhancement feature determination module is used to substitute the third splicing feature into the second feature enhancement model to obtain a second splicing enhancement feature; a fourth splicing feature determination module, configured to perform a 1x1 convolution on the second splicing enhancement feature and then splice the concatenated feature with the first enhancement feature to obtain a fourth splicing feature; The target detection result determination module is used to substitute the second splicing feature, the second splicing enhancement feature and the fourth splicing feature into the detection head to obtain the target detection result.

8. The belt foreign body visual recognition system based on image processing according to claim 7 is characterized in that: The second feature enhancement model includes: The local pooling module is used to locally pool the input features to obtain local pooling features; A global pooling module, configured to perform global pooling on the local pooling features to obtain global pooling features; A pooling feature convolution module is used to perform one-dimensional convolution on the local pooling feature and the global pooling feature to obtain a target local pooling feature and a target global pooling feature; An attention weight map generation module is used to upsample the target local pooling features and the target global pooling features respectively to obtain a global attention weight map and a local spatial attention weight map; An element addition module, configured to perform element-by-element addition of the global attention weight map and the local spatial attention weight map to obtain a target attention weight; A feature weighting module is used to multiply the target attention weight by the input feature to obtain the output feature of the second feature enhancement model.

Images