Belt speed detection method and device of conveyor belt, electronic equipment and storage medium

Abstract

The invention provides a belt speed detection method and device of a conveyor belt, electronic equipment and a storage medium. The method comprises the steps of collecting multiple frames of conveyor belt images according to a preset time interval, preprocessing the conveyor belt images to obtain a target image, and determining a mapping relation between image pixel displacement and actual physical displacement; performing image processing on the target image by using a pre-trained deep learning model to obtain a pixel optical flow field, and determining a first conveyor belt speed according to the pixel optical flow field, the mapping relation and the preset time interval; determining a feature point set based on pixel points of a conveyor belt area in the target image, determining a target matching point pair set from the feature point set through a feature matching algorithm, and determining a second conveyor belt speed according to the target matching point pair set, the mapping relation and the preset time interval; and weighting the first conveyor belt speed and the second conveyor belt speed to obtain a target conveyor belt speed.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a method, apparatus, electronic device and storage medium for detecting the belt speed of a conveyor belt. Background Technology

[0002] Detecting conveyor belt speed effectively monitors its operation, and calculating speed by acquiring conveyor belt images can adapt to various complex industrial environments. However, when calculating conveyor belt speed from images, a single algorithm is typically relied upon. Conveyor belt images are easily affected by factors such as changes in lighting and image noise, leading to inaccurate speed calculations due to the reliance on a single algorithm.

[0003] Therefore, how to avoid inaccurate conveyor belt speed due to relying on a single algorithm to determine the conveyor belt speed has become an urgent technical problem to be solved. Summary of the Invention

[0004] In view of this, the purpose of this disclosure is to provide a method, apparatus, electronic device and storage medium for detecting the belt speed of a conveyor belt to solve or partially solve the above-mentioned technical problems.

[0005] To achieve the above objectives, the first aspect of this disclosure provides a method for detecting the belt speed of a conveyor belt, the method comprising: Multiple frames of conveyor belt images are acquired at preset time intervals, the conveyor belt images are preprocessed to obtain target images, and the mapping relationship between image pixel displacement and actual physical displacement is determined. The target image is processed using a pre-trained deep learning model to obtain a pixel optical flow field, and the first conveyor belt speed is determined based on the pixel optical flow field, the mapping relationship, and the preset time interval. A feature point set is determined based on the pixels in the conveyor belt region of the target image. A target matching point pair set is determined from the feature point set using a feature matching algorithm. The second conveyor belt speed is determined based on the target matching point pair set, the mapping relationship, and the preset time interval. The target conveyor belt speed is obtained by weighting the speeds of the first and second conveyor belts.

[0006] Based on the same inventive concept, a second aspect of this disclosure proposes a belt speed detection device for a conveyor belt, comprising: The preprocessing module is configured to acquire multiple frames of conveyor belt images at preset time intervals, preprocess the conveyor belt images to obtain a target image, and determine the mapping relationship between image pixel displacement and actual physical displacement. The first conveyor belt speed determination module is configured to use a pre-trained deep learning model to perform image processing on the target image to obtain a pixel optical flow field, and determine the first conveyor belt speed based on the pixel optical flow field, the mapping relationship and the preset time interval. The second conveyor belt speed determination module is configured to determine a set of feature points based on the pixels of the conveyor belt region in the target image, determine a set of target matching point pairs from the set of feature points through a feature matching algorithm, and determine the second conveyor belt speed according to the set of target matching point pairs, the mapping relationship and the preset time interval. The target conveyor belt speed determination module is configured to perform weighted processing on the first conveyor belt speed and the second conveyor belt speed to obtain the target conveyor belt speed.

[0007] Based on the same inventive concept, a third aspect of this disclosure proposes an electronic device including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method described above when executing the computer program.

[0008] Based on the same inventive concept, a fourth aspect of this disclosure provides a non-transitory computer-readable storage medium that stores computer instructions for causing a computer to perform the methods described above.

[0009] As described above, the conveyor belt speed detection method, apparatus, electronic device, and storage medium provided in this disclosure are as follows: Multiple frames of conveyor belt images are acquired at preset time intervals. These images are preprocessed to obtain a target image, and the mapping relationship between image pixel displacement and actual physical displacement is determined. A pre-trained deep learning model is used to process the target image to obtain a pixel optical flow field. Based on the pixel optical flow field, the mapping relationship, and the preset time interval, a first conveyor belt speed is determined. Thus, the first conveyor belt speed can be accurately determined from the target image using an optical flow algorithm. A feature point set is determined based on the pixels in the conveyor belt region of the target image. A target matching point pair set is determined from the feature point set using a feature matching algorithm. A second conveyor belt speed is determined based on the target image using the target image using the feature matching algorithm. Finally, the first and second conveyor belt speeds are weighted to obtain the target conveyor belt speed. This approach combines the advantages of both optical flow and feature matching algorithms, effectively improving the accuracy and reliability of conveyor belt speed detection and reducing errors that may arise from using a single algorithm to calculate the conveyor belt speed. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in this disclosure or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart of a conveyor belt speed detection method according to an embodiment of the present disclosure; Figure 2 This is a flowchart of a conveyor belt speed detection method according to an embodiment of the present disclosure; Figure 3 This is a schematic diagram of the structure of a RAFT network according to an embodiment of the present disclosure; Figure 4 This is a flowchart of the feature matching algorithm according to an embodiment of the present disclosure; Figure 5 This is a schematic diagram illustrating the comparative experimental results of an embodiment of this disclosure; Figure 6 This is a schematic diagram of the conveyor belt speed detection device according to an embodiment of the present disclosure; Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0013] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this disclosure should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in the embodiments of this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0014] Based on the background description, belt conveyors in applications such as coal mines typically operate at a constant belt speed under full load. When the material on the conveyor belt is insufficient or low, maintaining full load speed will reduce equipment efficiency, lead to excessive energy consumption, and damage conveyor components. To promote the development of energy-saving and speed-regulating technologies, research on belt speed measurement is crucial, as it allows for more effective monitoring of conveyor operation. Belt speed detection can detect faults such as slippage, belt breakage, and overspeed. Some regulations mandate belt speed detection, and belt speed protection is one of the eight protections in a comprehensive belt conveyor protection system. Currently, belt speed detection methods mainly utilize sensors such as linear speed sensors, photoelectric encoders, and Hall effect speed sensors, with linear speed sensors being a commonly used method.

[0015] With the continuous evolution of image processing technology, it has become possible to detect the belt speed of conveyors using machine vision technology. Based on this, a conveyor belt speed detection method based on sequential images is proposed. This method uses a camera to acquire images of the upper surface of the conveyor belt. Given that adjacent images in the sequential image sequence have the same time interval, an image matching algorithm is used to calculate the relative pixel displacement of feature points in adjacent images. Simultaneously, the mapping relationship between the image coordinate system and the physical coordinate system is used to calculate the physical displacement and belt speed of the conveyor belt. However, due to the relatively simple scenario of conveyor belts transporting coal, the feature points in the conveyor belt movement are not obvious enough, resulting in poor accuracy and reliability of belt speed detection.

[0016] In recent years, optical flow algorithms, a method for estimating velocity using motion vectors, have been well applied in river velocity estimation research. Optical flow estimation velocimetry has low requirements for tracers and can generate pixel-level flow fields, making it suitable for river velocity detection. Belt conveyor speed detection is similar to river velocity estimation; both scenarios involve moving targets only within the region of interest and lack visible tracers. Therefore, using optical flow algorithms for belt conveyor speed detection should be feasible. However, due to limitations in the assumptions, the measurement accuracy of optical flow algorithm-estimated belt speed is relatively low in scenarios with light and shadow interference or large displacements.

[0017] This disclosure proposes a conveyor belt speed detection method that integrates optical flow and feature matching. Optical flow calculates the motion vectors of pixels in an image, roughly reflecting the motion of the conveyor belt surface. These motion vectors provide information about the overall movement trend of the conveyor belt, offering a preliminary reference for feature extraction and matching. The results of feature extraction and matching are used to verify the calculation results of the optical flow method. A Bayesian algorithm is used for decision fusion, assigning weights based on information such as image brightness and contrast. Under good lighting conditions and with clear conveyor belt surface features, the accuracy of feature extraction and matching results is high, and these are given higher weights. However, under conditions of significant lighting variations or noise interference, the results of the optical flow method are given higher weights while being corrected in conjunction with the results of feature extraction and matching. This fusion detection method fully utilizes the advantages of optical flow and feature extraction and matching, improving the accuracy and reliability of conveyor belt speed detection.

[0018] As mentioned above, how to avoid inaccurate conveyor belt speed due to relying on a single algorithm to determine the conveyor belt speed has become an important research problem.

[0019] Based on the above description, such as Figure 1 As shown in this embodiment, the conveyor belt speed detection method includes: Step 101: Acquire multiple frames of conveyor belt images at preset time intervals, preprocess the conveyor belt images to obtain target images, and determine the mapping relationship between image pixel displacement and actual physical displacement.

[0020] In practice, a high-definition industrial camera is used to capture multiple frames of conveyor belt images at preset time intervals. The high-definition industrial camera can have a resolution of 1920×1080 or higher and a frame rate of 30fps or higher. The high-definition industrial camera can be fixed directly above or slightly above the conveyor belt to capture continuous multi-frame images of the conveyor belt during its operation. The installation height of the high-definition industrial camera can be 1.5m to 3m above the conveyor belt surface, and the angle between the camera lens axis and the conveyor belt surface can be less than or equal to 30°.

[0021] The camera's triggering mode can be set to continuous triggering, with a preset time interval between adjacent frame images. It is fixed; the preset time interval is determined based on the camera's frame rate. For example, when the camera's frame rate is 30fps, the preset time interval... Specifically, multiple frames of conveyor belt images are acquired at preset time intervals, and the acquisition timestamps are recorded simultaneously.

[0022] The target image is obtained by preprocessing the conveyor belt image, wherein the preprocessing includes at least one of the following: grayscale processing, noise reduction processing and brightness normalization processing, and the target image is the preprocessed conveyor belt image.

[0023] The mapping relationship is the correspondence between image pixel displacement and actual physical displacement. The mapping relationship between image pixel displacement and actual physical displacement is calibrated based on camera parameters. Thus, after determining the average pixel displacement based on the target image, the actual displacement corresponding to the average pixel displacement can be determined based on the mapping relationship, thereby enabling a more accurate determination of the conveyor belt speed based on the actual displacement.

[0024] Figure 2 This is a flowchart of a conveyor belt speed detection method according to an embodiment of this disclosure. Figure 2 As shown, the conveyor belt speed detection method includes: acquiring images with an industrial camera; determining the mapping relationship between image pixel displacement and actual physical displacement through camera calibration; determining the first conveyor belt speed using the RAFT-SEnet optical flow algorithm; determining the second conveyor belt speed using the Harris-BRIEF-RANSAC feature matching algorithm; and obtaining the target conveyor belt speed by weighting the first and second conveyor belt speeds using a Bayesian algorithm.

[0025] Step 102: Use a pre-trained deep learning model to perform image processing on the target image to obtain the pixel optical flow field, and determine the first conveyor belt speed based on the pixel optical flow field, the mapping relationship, and the preset time interval.

[0026] In practice, a pre-trained deep learning model is used to process the target image to obtain the pixel optical flow field. The deep learning model can be a RAFT-SEnet optical flow network model. The RAFT algorithm includes a feature extractor, a visual similarity calculator, and an updater.

[0027] Figure 3 This is a schematic diagram of the structure of a RAFT network according to an embodiment of this disclosure. Figure 3 As shown, a Recurrent All-pairs Field Transforms (RAFT) network is used to estimate the optical flow from two consecutive frames of target images. A feature extractor is used to extract features from the target images, and an attention mechanism using Squeeze-and-Excitation Networks (SENet) is added after the feature extractor. A four-layer pyramid feature structure is constructed in the visual similarity calculator. The updater updates the initial optical flow field through a Gated Recurrent Unit (GRU) to obtain the pixel optical flow field.

[0028] Step 103: Determine a set of feature points based on the pixels in the conveyor belt region of the target image, determine a set of target matching point pairs from the set of feature points using a feature matching algorithm, and determine the second conveyor belt speed according to the set of target matching point pairs, the mapping relationship, and the preset time interval.

[0029] In practical implementation, to address the shortcomings of conveyor belt images such as insignificant features and unclear overlapping areas, a Harris-BRIEF-RANSAC feature matching algorithm suitable for conveyor belt images is proposed. This algorithm employs a corner extraction algorithm (Harris operator) to extract feature points from the conveyor belt image, utilizes the Binary Robust Independent Elementary Features (BRIEF) algorithm for descriptor extraction and matching to achieve a uniform distribution of feature points, and uses the Random Sampling Consensus (RANSAC) algorithm to filter matching points. Furthermore, defining the conveyor belt region of the target image accelerates the algorithm's execution speed.

[0030] Figure 4 This is a flowchart of the feature matching algorithm according to an embodiment of this disclosure. Figure 4 As shown, the feature matching algorithm consists of three parts: Harris algorithm for extracting scale space feature points, BRIEF algorithm for extracting descriptors, and RANSAC algorithm for filtering matching points.

[0031] Step 104: The first conveyor belt speed and the second conveyor belt speed are weighted to obtain the target conveyor belt speed.

[0032] In practice, a first weight for the speed of the first conveyor belt and a second weight for the speed of the second conveyor belt are determined. Based on the first and second weights, the first and second conveyor belt speeds are weighted to obtain the target conveyor belt speed.

[0033] Through the above embodiments, multiple frames of conveyor belt images are acquired at preset time intervals. The conveyor belt images are preprocessed to obtain a target image, and the mapping relationship between image pixel displacement and actual physical displacement is determined. A pre-trained deep learning model is used to process the target image to obtain the pixel optical flow field. Based on the pixel optical flow field, the mapping relationship, and the preset time interval, the first conveyor belt speed is determined. Thus, the optical flow algorithm can accurately determine the first conveyor belt speed based on the target image. A feature point set is determined based on the pixels in the conveyor belt region of the target image. A target matching point pair set is determined from the feature point set using a feature matching algorithm. Based on the target matching point pair set, the mapping relationship, and the preset time interval, the second conveyor belt speed is determined. Thus, the feature matching algorithm can accurately determine the second conveyor belt speed based on the target image. The first and second conveyor belt speeds are weighted to obtain the target conveyor belt speed. This approach combines the advantages of both the optical flow algorithm and the feature matching algorithm, effectively improving the accuracy and reliability of conveyor belt speed detection and reducing errors that may arise from calculating conveyor belt speed using a single algorithm.

[0034] In some embodiments, step 101 includes: Step 1011: The conveyor belt image is converted to grayscale to obtain a grayscale image. The grayscale image is then denoised using Gaussian filtering to obtain a denoised image. Finally, the denoised image is normalized to obtain the target image.

[0035] In practice, the acquired conveyor belt images are red, green, and blue (RGB) images, which are color images synthesized based on the three basic color channels of red, green, and blue. Specifically, a weighted average method is used to convert the RGB conveyor belt images to grayscale to obtain single-channel grayscale images, which reduces computational complexity. .

[0036] Specifically, the kernel size is set to 5×5, and the standard deviation is... The grayscale image is denoised by Gaussian filtering to suppress environmental noise and image sensor noise.

[0037] Specifically, the pixel values ​​of the denoised image are mapped to intervals. The target image is obtained by performing brightness normalization on the denoised image, which reduces the impact of light intensity fluctuations on subsequent optical flow estimation.

[0038] In addition, preprocessing can also include region identification. Specifically, the conveyor belt region is identified from the target image. The conveyor belt region is extracted through threshold segmentation and morphological operations (erosion + dilation) to eliminate background interference. The conveyor belt region can be set as a rectangle, and the region boundary is determined by the conveyor belt edge detection results (using the Canny operator, high threshold 200, low threshold 100) to reduce invalid calculation areas.

[0039] Step 1012: Collect multiple calibration board images of the preset calibration board on the conveyor belt, and determine the camera's intrinsic parameter matrix, distortion coefficients, and extrinsic parameters based on the multiple calibration board images.

[0040] In practice, the preset calibration board is a board that is pre-set on the conveyor belt to calibrate the relationship between image pixel displacement and actual physical displacement. The preset calibration board can be a checkerboard calibration board. For example, the size of the checkerboard in the preset calibration board can be 20mm×20mm, the number of rows of the checkerboard is greater than or equal to 8, and the number of columns of the checkerboard is greater than or equal to 6.

[0041] Multiple calibration board images are images of the calibration board that are captured when the preset calibration board is set at different positions and angles on the conveyor belt. For example, multiple calibration board images can be at least 10 calibration board images.

[0042] The camera's intrinsic parameter matrix K, distortion coefficients D, and extrinsic parameters are determined using a calibration algorithm based on multiple calibration board images. The distortion coefficients D include: radial distortion coefficients. and tangential distortion coefficient The extrinsic parameters include: rotation matrix R and translation vector T.

[0043] Step 1013: Determine the pixel physical size based on the intrinsic parameter matrix, the distortion coefficient, and the extrinsic parameter, and generate a mapping relationship between image pixel displacement and actual physical displacement based on the pixel physical size.

[0044] In practice, the camera's intrinsic parameter matrix, distortion coefficients, and extrinsic parameters are used as calibration results. Based on these calibration results, the physical size of each pixel is determined, and a mapping relationship between image pixel displacement and actual physical displacement is constructed according to the pixel physical size. ,in, This represents the actual physical displacement (unit: m). Image pixel displacement (unit: pixel). Physical pixel size (unit: m / pixel).

[0045] The above scheme involves converting the conveyor belt image to grayscale to obtain a grayscale image, then using Gaussian filtering to denoise the grayscale image to obtain a denoised image, and finally normalizing the brightness of the denoised image to obtain the target image. This pre-processed target image allows for accurate detection of the conveyor belt speed. Multiple calibration plate images of a pre-set calibration plate on the conveyor belt are acquired, and the camera's intrinsic parameter matrix, distortion coefficients, and extrinsic parameters are determined based on these images. The pixel physical size is then determined based on the intrinsic parameter matrix, distortion coefficients, and extrinsic parameters, and a mapping relationship between image pixel displacement and actual physical displacement is generated based on this pixel physical size. This allows for accurate calibration of the mapping relationship between image pixel displacement and actual physical displacement based on the camera parameters, and enables the determination of the actual displacement corresponding to the average pixel displacement, thus allowing for more accurate determination of the conveyor belt speed based on the actual displacement.

[0046] In some embodiments, the pre-trained deep learning model includes: a feature encoder, a context encoder, a similarity calculator, and an updater; step 102 includes: Step 1021: Use the feature encoder to perform feature extraction processing on the target image to obtain the first feature vector of the current frame image and the second feature vector of the adjacent frame image, and perform inner product operation on the first feature vector and the second feature vector to obtain the feature similarity matrix.

[0047] In practice, deep learning has been widely applied in the field of computer vision in recent years, achieving remarkable results. To address the problems of traditional optical flow estimation methods, deep learning-based optical flow estimation methods have become mainstream. This disclosure optimizes the RAFT optical flow estimation algorithm using it as the baseline network architecture. The RAFT algorithm mainly consists of a feature extractor, a visual similarity calculator, and an updater.

[0048] The core function of the feature extractor is to extract distinctive motion features from consecutive frames of target images, providing a feature basis for subsequent pixel matching. Simultaneously, it uses the SENet attention mechanism to filter key information and suppress interference. The feature extractor consists of a feature encoder and a context encoder, both of which are built upon a Convolutional Neural Network (CNN) and optimized with the SENet attention module.

[0049] SENet is an attention mechanism for deep convolutional neural networks (CNNs) designed to enhance the representation capabilities of CNNs on feature channels. It learns importance weights for each channel and then reweights the feature maps using these importance weights, thereby enhancing the representation of useful information and suppressing irrelevant information.

[0050] The RAFT-SEnet optical flow network model incorporates the SENet attention module, enabling the motion feature extraction network to focus on information-rich regions (e.g., pixels with particles or ripples) and reduce the influence of irrelevant regions (e.g., pixels with shadows and background).

[0051] The feature encoder consists of two CNN sub-networks sharing weights. Each sub-network contains four convolutional layers (3×3 kernels, stride 1, padding 1), a BatchNorm layer, and a ReLU activation function. Two consecutive frames of the target image (the current frame in frame t and the adjacent frame in frame (t+1)) are input into the feature encoder. The feature encoder performs feature extraction on the two consecutive frames to obtain a 256-dimensional high-dimensional feature vector. Specifically, the feature encoder extracts features from the current frame in frame t to obtain a first feature vector, and extracts features from the adjacent frame in frame (t+1) to obtain a second feature vector. The first and second feature vectors are then multiplied to obtain a feature similarity matrix.

[0052] After the feature similarity matrix is ​​output, a SENet attention mechanism is added. Through global average pooling → channel weight learning → feature weighting, the importance of each feature channel is adaptively adjusted to highlight effective motion features such as particles and ripples on the conveyor belt surface, while suppressing interfering features from irrelevant regions such as shadows and background. The importance of each feature channel is represented as follows: , The importance of each feature channel is represented. It is the Sigmoid activation function. As the second weight, It is the ReLU activation function. As the first weight, This is the feature similarity matrix.

[0053] Step 1022: Use the context encoder to perform feature extraction processing on the target image to obtain the first feature vector of the current frame image, and determine global semantic information based on the first feature vector.

[0054] In practice, the context encoder consists of a single-path CNN sub-network, each containing four convolutional layers (3×3 kernels, stride 1, padding 1), a BatchNorm layer, and a ReLU activation function. Two consecutive frames of the target image are input into the context encoder, which performs feature extraction to obtain a 256-dimensional high-dimensional feature vector. Specifically, the context encoder extracts the first feature vector of the current frame image and captures global semantic information from the target image. This global semantic information can include the overall outline and motion trend of the conveyor belt.

[0055] Step 1023: Use the similarity calculator to convert the feature similarity matrix into a multi-scale feature representation, and determine the semantic similarity based on the multi-scale feature representation.

[0056] In practice, the similarity calculator can be a visual similarity calculator. The core function of the similarity calculator is to determine the matching relationship of pixels in two consecutive target images based on the feature similarity matrix output by the feature encoder, so as to provide initial matching clues for optical flow estimation.

[0057] Specifically, the feature similarity matrix output by the feature encoder is input into a similarity calculator. The similarity calculator is used to construct a four-layer pyramid using scaling ratios of 1 / 1, 1 / 2, 1 / 4, and 1 / 8 to achieve multi-scale feature representation. At each level of the pyramid, a global search is performed on the feature vectors to calculate the semantic similarity of pixels at different locations, finding the potential corresponding positions of pixels in the current frame (frame t) and their neighboring frames (frame t+1). Here, the semantic similarity can be the semantic similarity feature vector after multi-scale matching, containing initial pixel-level correspondence information.

[0058] Step 1024: The updater updates the initial optical flow value based on the semantic similarity and the global semantic information to obtain an updated optical flow value. The updated optical flow value is then smoothed to obtain a smoothed optical flow value. The smoothed optical flow value with the number of iterations reaching a preset threshold is used as the pixel optical flow field.

[0059] In practice, the core function of the updater is to iteratively calculate and correct the optical flow value based on the initial matching clues, and finally obtain the pixel-level accurate optical flow field (i.e., the motion vector of each pixel).

[0060] The updater includes a gated recurrent unit (GRU) and a convolutional layer iterative update module. Initial optical flow values ​​are obtained based on feature matching results, and these initial optical flow values, semantic similarity, and global semantic information are input into the updater. The GRU fuses the initial optical flow values, semantic similarity, and global semantic information, dynamically adjusting the update direction of the initial optical flow values ​​to obtain updated optical flow values, thus resolving matching deviations caused by large displacements and lighting interference. Two 3×3 convolutional layers smooth the updated optical flow values ​​to obtain smoothed optical flow values, ensuring the continuity of the optical flow field (conforming to the physical laws of the conveyor belt's overall motion). Starting from the initial optical flow value, iterative updates are performed 10 times (adjustable according to operating conditions), gradually correcting errors, and finally outputting a converged and accurate pixel optical flow field.

[0061] The pixel optical flow field can be represented as ,in, For pixels In the direction of conveyor belt movement ( Pixel displacement (axis) Perpendicular to the direction of conveyor belt movement ( The pixel displacement of the axis. Only the pixel displacement of the axis is needed for subsequent conveyor belt speed calculations. Pixel displacement of axis .

[0062] In addition, the sample images are conveyor belt operation images under different working conditions (conveyor belt speed 0.5-4.5m / s, sufficient lighting / insufficient lighting, conveyor belt empty / conveyor belt full load). Pixel displacement is marked on the continuous frame target images (based on the physical displacement after calibration) to construct a belt speed detection dataset, which is divided into training set and validation set in a 7:3 ratio.

[0063] The deep learning model is trained using a training set and validated using a validation set. Specifically, when training the deep learning model, the optimizer used is Adam, with an initial learning rate of 0.0001 and a weight decay factor of 1×10⁻⁶. ﹣4 (i.e., 1e-4), momentum is 0.9, batch size is set to 4, and training rounds are 500. The loss function is a weighted sum of photometric loss (weight 1.0) and smoothing loss (weight 0.01). The photometric loss is calculated based on the assumption of constant brightness, and the smoothing loss constrains the continuity of the optical flow field.

[0064] The above scheme utilizes a feature encoder to extract features from the target image, obtaining the first feature vector of the current frame and the second feature vectors of adjacent frames. An inner product operation is performed on the first and second feature vectors to obtain a feature similarity matrix, thus determining the feature similarity matrix between two adjacent target images. A context encoder is used to extract features from the target image, obtaining the first feature vector of the current frame. Global semantic information is determined based on this first feature vector, enabling the determination of the global semantic information for each frame of the target image. A similarity calculator is used to convert the feature similarity matrix into a multi-scale feature representation. Semantic similarity is determined based on this multi-scale feature representation, providing initial matching clues for optical flow estimation. An updater is used to update the initial optical flow value based on semantic similarity and global semantic information, obtaining an updated optical flow value. This updated optical flow value is then smoothed to obtain a smoothed optical flow value. The smoothed optical flow value, after reaching a preset threshold of iterations, is used as the pixel optical flow field, making the obtained pixel optical flow field more accurate. This allows the determination of the first pixel displacement in the conveyor belt's motion direction based on the pixel optical flow field.

[0065] In some embodiments, step 102 includes: Step 1025: Determine the first pixel displacement in the direction of the conveyor belt movement from the pixel optical flow field, and perform averaging on multiple first pixel displacements in the conveyor belt area to obtain the first average pixel displacement.

[0066] In practice, the direction of conveyor belt movement (pixel) is determined from the pixel optical flow field. The first pixel displacement of the axis) .

[0067] Background regions in the target image are eliminated through image segmentation, retaining only the optical flow vector of the conveyor belt region. This is done for all pixels in the conveyor belt region within the pixel optical flow field. First pixel displacement of axis The first average pixel displacement is obtained by averaging. ,in, The first average pixel displacement, This represents the total number of pixels in the conveyor belt area. For the first in the conveyor belt area The first pixel displacement of 1 pixel.

[0068] Step 1026: Determine the first actual displacement corresponding to the first average pixel displacement based on the mapping relationship.

[0069] In practice, the first actual displacement corresponding to the first average pixel displacement is determined based on the mapping relationship between image pixel displacement and actual physical displacement. ,in, This is the first actual displacement. The first average pixel displacement, This refers to the physical size of a pixel.

[0070] Step 1027: The ratio of the first actual displacement to the preset time interval is taken as the first conveyor belt speed.

[0071] In practice, the ratio of the first actual displacement to the preset time interval is used as the first conveyor belt speed. ,in, The first conveyor belt speed, This is the first actual displacement. This is the preset time interval.

[0072] The above scheme determines the first pixel displacement in the direction of the conveyor belt movement from the pixel optical flow field. Multiple first pixel displacements in the conveyor belt region are averaged to obtain a first average pixel displacement. The first actual displacement corresponding to the first average pixel displacement is determined based on a mapping relationship. The ratio of the first actual displacement to a preset time interval is used as the first conveyor belt speed. In this way, the pixel optical flow field determined by the optical flow algorithm can accurately determine the first conveyor belt speed.

[0073] In some embodiments, step 103 includes: Step 1031: Determine the reference frame image and the frame image to be matched from the target image.

[0074] In practice, the reference frame image can be the t-th frame image, and the frame image to be matched can be the (t+1)-th frame image.

[0075] Step 1032: Determine the first directional gradient and the second directional gradient based on the conveyor belt region in the target image, and determine the gradient covariance matrix based on the first directional gradient and the second directional gradient.

[0076] In practice, the Sobel operator is used to determine the first and second directional gradients based on the conveyor belt region in the target image. The Sobel operator is an operator used in image processing to calculate image differences. Combined with Gaussian smoothing and differentiation operations, it can detect edge features while reducing noise interference. The first directional gradient can be the gradient along the x-axis. The second directional gradient can be the gradient along the y-axis. .

[0077] Use a Gaussian window (window size 7×7, standard deviation). Weighted smoothing is performed to obtain the gradient covariance matrix. ,in, This represents the Gaussian convolution operation.

[0078] Step 1033: Determine the corner response value based on the gradient covariance matrix, and construct a feature point set based on the pixels whose corner response values ​​are greater than a preset response threshold; wherein, the feature point set includes a reference frame feature point set and a frame feature point set to be matched.

[0079] In practice, the corner response values ​​are determined based on the gradient covariance matrix. ,in, Corner response value, The gradient covariance matrix The determinant, The gradient covariance matrix traces, It is an empirical constant. The value range is from 0.04 to 0.06. In this embodiment of the disclosure... The value can be 0.05.

[0080] A feature point set is constructed based on pixels whose corner response values ​​are greater than a preset response threshold. The preset response threshold is... Preset response threshold It can be 10% of the maximum corner response value among multiple pixels in the conveyor belt area, retaining corner response values ​​greater than a preset response threshold. The pixels are selected. Non-Maximum Suppression (NMS) is used to preserve the corner response values ​​within a 3×3 neighborhood. The pixel with the maximum value is used to obtain the feature point set.

[0081] The feature point set includes the reference frame feature point set. and the feature point set of the frame to be matched .

[0082] Step 1034: Generate reference frame pixels and matching frame pixels based on the feature point set according to the Gaussian distribution. Determine the descriptor of the reference frame pixels by comparing the gray values ​​of the reference frame pixels and the matching frame pixels.

[0083] In practice, a 31×31 pixel neighborhood window is constructed for each feature point in the feature point set, and 256 pairs of pixels are randomly generated based on a Gaussian distribution. Each pixel pair includes pixels from the reference frame. and the pixels of the frame to be matched , .

[0084] The descriptor of the reference frame pixels is determined by comparing the grayscale values ​​of pixels in the reference frame with those in the frame to be matched. Specifically, for each pair of pixels, the grayscale values ​​at corresponding positions within the neighborhood window are compared. Then determine the first The bit descriptor is 1. When Then determine the first The bit descriptor is 0. A 256-bit binary descriptor (Binary Robust Independent Elementary Features, or BRIEF for short) is generated by comparing 256 pairs of pixels sequentially.

[0085] Step 1035: Determine the Hamming distance between the reference frame pixel and the frame pixel to be matched based on the descriptor, and construct an initial set of matching point pairs based on the reference frame pixel and the frame pixel to be matched whose Hamming distance is less than a preset distance threshold.

[0086] In practice, Hamming distance is used to measure the similarity of descriptors, and a distance threshold is set. For the feature points of the reference frame The descriptor is matched with the descriptors of all feature points in the frame to be matched, retaining those with a Hamming distance less than the distance threshold. Feature point pairs are used to obtain the initial set of matching point pairs. ,in, For the initial set of matching point pairs, For the matching point pairs in the initial set of matching point pairs, This indicates that the Hamming distance between matching point pairs in the initial set of matching point pairs is less than the distance threshold.

[0087] Step 1036: Determine a preset number of matching point pairs from the initial set of matching point pairs, determine the translation vector of the matching point pairs, and determine the target set of matching point pairs based on the deviation between the matching point pairs and the translation vector.

[0088] In practice, it is assumed that the motion model of the feature point pairs is a translation model: ,in, It is a global translation vector (i.e., pixel displacement caused by the movement of the conveyor belt). For the frame pixels to be matched in the initial set of matching point pairs, The reference frame pixels for matching point pairs in the initial set of matching point pairs. is a coefficient.

[0089] Randomly select from the initial set of matching points Select 4 matching point pairs and solve for the translation vector. Calculate the initial set of matching point pairs. All matching point pairs and translation vectors Deviation between ,in, The deviation between the matching point pairs in the initial set of matching point pairs and the translation vector. For the first One frame pixel to be matched Coordinate values No. A reference frame pixel Coordinate values For translation vectors Coordinate values No. One frame pixel to be matched Coordinate values No. A reference frame pixel Coordinate values For translation vectors Coordinate values. Set deviation threshold. Statistical satisfaction The number of interior points.

[0090] Repeat the iteration 1000 times, and retain the translation vector with the most interior points as the optimal solution. The corresponding set of interior points is the set of target matching point pairs. Additionally, when the target matching point pair set If the number of matching point pairs is less than 20, adjust the preset response threshold. Repeat steps 1031 to 1036 to ensure the matching is valid.

[0091] The above scheme determines a reference frame image and a frame image to be matched from the target image. A first-direction gradient and a second-direction gradient are determined based on the conveyor belt region in the target image, and a gradient covariance matrix is ​​determined based on these gradients. Corner response values ​​are determined according to the gradient covariance matrix, and a feature point set is constructed based on pixels with corner response values ​​greater than a preset response threshold, enabling accurate extraction of feature points in the conveyor belt. Reference frame pixels and frame images to be matched are generated based on the feature point set using a Gaussian distribution. A descriptor for the reference frame pixels is determined by comparing the grayscale values ​​of the reference frame pixels and the frame images to be matched, achieving a uniform distribution of feature points. The Hamming distance between the reference frame pixels and the frame images to be matched is determined based on the descriptor, and an initial set of matching point pairs is constructed based on reference frame pixels and frame images to be matched whose Hamming distance is less than a preset distance threshold. A preset number of matching point pairs are determined from the initial set, and a translation vector for each matching point pair is determined. The target set of matching point pairs is determined based on the deviation between the matching point pairs and the translation vector, making the obtained target set of matching point pairs more accurate.

[0092] In some embodiments, step 103 includes: Step 1037: Determine the second average pixel displacement based on each pair of matching points in the target matching point pair set.

[0093] In practice, it is based on the target matching point pair set. Matching point pairs in the data, calculating the direction of conveyor belt movement (pixels) The second average pixel displacement (axis) ,in, This is the second average pixel displacement. For the target matching point pair set The number of matching point pairs in the data. For the target matching point pair set Matching point pairs in the middle, For the first One frame pixel to be matched Coordinate values No. A reference frame pixel Coordinate values.

[0094] Step 1038: Determine the second actual displacement corresponding to the second average pixel displacement based on the mapping relationship.

[0095] In practice, the second actual displacement corresponding to the second average pixel displacement is determined based on the mapping relationship between image pixel displacement and actual physical displacement. ,in, This is the second actual displacement. This is the second average pixel displacement. This refers to the physical size of a pixel.

[0096] Step 1039: The ratio of the second actual displacement to the preset time interval is taken as the second conveyor belt speed.

[0097] In practice, the ratio of the first actual displacement to the preset time interval is used as the first conveyor belt speed. ,in, For the second conveyor belt speed, This is the second actual displacement. This is the preset time interval.

[0098] The above scheme determines the second average pixel displacement for each pair of matching points in the target matching point pair set. The second actual displacement corresponding to the second average pixel displacement is then determined based on the mapping relationship. The ratio of the second actual displacement to a preset time interval is used as the second conveyor belt speed. In this way, the target matching point pair set determined by the feature matching algorithm can accurately determine the second conveyor belt speed.

[0099] In some embodiments, step 104 includes: Step 1041: Obtain the brightness and contrast of the target image, and determine the first initial weight and the second initial weight based on the brightness and the contrast.

[0100] In practical implementation, due to the harsh working environment of the conveyor belt, weights are allocated based on the brightness and contrast of the target image to achieve belt speed detection under different lighting and interference conditions. A Bayesian algorithm is used to fuse optical flow and feature matching algorithms for decision-making to achieve belt speed detection. A first initial weight and a second initial weight are determined based on the brightness and contrast. The first initial weight corresponds to the initial weight of the optical flow algorithm, and the second initial weight corresponds to the initial weight of the feature matching algorithm.

[0101] The first initial weights are determined based on the brightness and contrast of the target image. ,in, As the first initial weight, This is the first brightness adjustment parameter. The brightness of the target image. Adjust the parameters for the first contrast ratio. The contrast of the target image. First brightness adjustment parameter. and the first contrast adjustment parameter Adjustments can be made based on the actual situation on site.

[0102] The second initial weights are determined based on the brightness and contrast of the target image. ,in, As the second initial weight, This is the second brightness adjustment parameter. The brightness of the target image. Adjust the parameters for the second contrast ratio. The contrast of the target image. Second brightness adjustment parameter. Second contrast adjustment parameters Adjustments can be made based on the actual situation on site.

[0103] Step 1042: Normalize the first initial weight to obtain the first target weight, and normalize the second initial weight to obtain the second target weight.

[0104] In practice, to satisfy the constraint that the sum of the weight coefficients is 1, the first initial weight coefficients are normalized to obtain the first target weight, and the second initial weights are normalized to obtain the second target weight. The first target weight is the target weight corresponding to the optical flow algorithm, and the second target weight is the target weight corresponding to the feature matching algorithm. The sum of the first target weight and the second target weight is 1.

[0105] Specifically, the first initial weight coefficients are normalized to obtain the first target weights. ,in, As the first objective weight, As the first initial weight, This is the second initial weight.

[0106] Specifically, the first target weight is obtained by normalizing the second initial weight coefficient. ,in, As the weight of the second objective, As the first initial weight, This is the second initial weight.

[0107] Step 1043: Based on the first target weight and the second target weight, the first conveyor belt speed and the second conveyor belt speed are weighted to obtain the target conveyor belt speed.

[0108] In practice, based on the Bayesian algorithm, the first conveyor belt speed and the second conveyor belt speed are weighted and processed according to the first target weight and the second target weight to obtain the target conveyor belt speed. , in, For the target conveyor belt speed, As the first objective weight, The first conveyor belt speed, As the weight of the second objective, This is the second conveyor belt speed.

[0109] The above scheme obtains the brightness and contrast of the target image, and determines the first and second initial weights based on these values. This ensures that the first and second initial weights are adapted to the brightness and contrast of the target image, making them more accurate. Normalization is applied to the first initial weight to obtain the first target weight, and the second initial weight is also normalized to obtain the second target weight, ensuring that the sum of the weight coefficients is 1. Based on the first and second target weights, the first and second conveyor belt speeds are weighted to obtain the target conveyor belt speed. This approach combines the advantages of optical flow and feature matching algorithms, enabling more accurate determination of the target conveyor belt speed.

[0110] Through the above embodiments, multiple frames of conveyor belt images are acquired at preset time intervals. The conveyor belt images are preprocessed to obtain a target image, and the mapping relationship between image pixel displacement and actual physical displacement is determined. A pre-trained deep learning model is used to process the target image to obtain the pixel optical flow field. Based on the pixel optical flow field, the mapping relationship, and the preset time interval, the first conveyor belt speed is determined. Thus, the optical flow algorithm can accurately determine the first conveyor belt speed based on the target image. A feature point set is determined based on the pixels in the conveyor belt region of the target image. A target matching point pair set is determined from the feature point set using a feature matching algorithm. Based on the target matching point pair set, the mapping relationship, and the preset time interval, the second conveyor belt speed is determined. Thus, the feature matching algorithm can accurately determine the second conveyor belt speed based on the target image. The first and second conveyor belt speeds are weighted to obtain the target conveyor belt speed. This approach combines the advantages of both the optical flow algorithm and the feature matching algorithm, effectively improving the accuracy and reliability of conveyor belt speed detection and reducing errors that may arise from calculating conveyor belt speed using a single algorithm.

[0111] In some embodiments, the algorithm of this disclosure is verified and the results are analyzed through experiments. The specific experiments and results analysis are as follows: (1) Training of RAFT-SEnet model In order to acquire images of conveyor belts in operation under working conditions, this embodiment of the disclosure designs an image acquisition memory and installs it at the working sites of multiple belt conveyors to acquire image data. This results in obtaining a large amount of real-world conveyor belt operation image data. Pixel displacement marking is then applied to the image data to generate a conveyor belt speed detection dataset for RAFT model training.

[0112] During training, the Adam optimizer was used in the training parameter settings, with an initial learning rate of 0.0001, weight decay of 1e-4, momentum of 0.9, batch size of 4, training epochs of 500, photometric loss weight of 1.0, and smoothing loss weight of 0.01.

[0113] (2) Optical flow method test To evaluate the effectiveness of the methods in the embodiments of this disclosure, videos collected under different belt conveyor operating scenarios are used for verification. To more intuitively demonstrate the effectiveness and generalization ability of the optical flow detection method, optical flow image visualization is provided.

[0114] (3) Comparative test of conveyor belt speed detection To verify the advantages of the method in this embodiment, video data with different speeds and lighting conditions were used for verification. The conveyor belt speeds in the test videos were 0.5 m / s, 1 m / s, 1.5 m / s, 2 m / s, 3 m / s, 3.5 m / s, and 4.5 m / s. The fusion algorithm, the RAFT-SEnet optical flow algorithm, and the feature matching algorithm of this embodiment were used separately for belt speed detection. The results and errors were recorded, and the comparison results are shown in Table 1.

[0115] Table 1 Comparison Results

[0116] Table 1 shows the comparison results of different algorithms. Under sufficient lighting conditions, the maximum deviation of the optical flow method in detecting belt speed occurred at a set belt speed of 0.5 m / s, with a deviation of -8%. The maximum deviation of the feature matching algorithm was -10%, while the maximum deviation of the fusion algorithm was only -4%, an improvement of 6 percentage points. Under insufficient lighting conditions, the maximum deviation of the optical flow method in detecting belt speed occurred at a set belt speed of 2 m / s, with a deviation of 19%. The maximum deviation of the feature matching algorithm was 32%, while the maximum deviation of the fusion algorithm was only 8%, an improvement of 24 percentage points. Similarly, the mean square error, as the main parameter for detection stability, was also significantly improved in the fusion algorithm. In summary, the fusion algorithm can provide relatively accurate and stable belt speed detection results under both sufficient and insufficient lighting conditions, showing a clear advantage over using the optical flow method or the feature matching algorithm alone. The optical flow method performs well under sufficient lighting conditions but is somewhat affected under insufficient lighting conditions. The feature matching algorithm has large errors and poor stability under both lighting conditions. Therefore, the fusion algorithm proposed in this embodiment can effectively improve the accuracy and reliability of conveyor belt speed detection.

[0117] The conveyor belt speed detection method based on optical flow and feature matching fusion proposed in this disclosure was tested in a laboratory setting using a belt conveyor system. The results from a traditional speed sensor were used as comparative data.

[0118] Figure 5 This is a schematic diagram illustrating the comparative experimental results of an embodiment of this disclosure. For example... Figure 5 As shown in the experiment, the maximum deviation of the speed sensor is 18.8% under different belt speeds, while the algorithm of this disclosure yields a maximum deviation of 13.8% for belt speed. The results indicate that the speed measurement method of this disclosure is essentially similar to the sensor-based speed measurement method in detecting belt speed errors, both exhibiting good stability and reliability. Therefore, this method can effectively replace traditional speed sensors for belt speed detection, achieving non-contact measurement of conveyor belt speed, effectively improving the sensor's lifespan and stability, and overcoming the shortcomings of contact-based sensor speed measurement which is easily affected by the external environment.

[0119] The conveyor belt speed detection method based on optical flow and feature matching fusion proposed in this disclosure introduces an improved RAFT optical flow algorithm based on an attention mechanism and a Harris-BRIEF-RANSAC feature matching algorithm to calculate the belt speed separately, and then uses a Bayesian algorithm for decision fusion. This fully utilizes the advantages of both methods, allowing them to complement and fuse each other, effectively improving the accuracy and reliability of belt speed detection. The method of this disclosure provides strong support for conveyor belt operation monitoring and fault diagnosis, helping to improve production efficiency, reduce energy consumption, and decrease the risk of equipment damage. The detection method of this disclosure can be continuously optimized to adapt to the conveyor belt speed detection needs under different operating conditions, further optimizing the algorithm and improving computational efficiency.

[0120] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.

[0121] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0122] Based on the same inventive concept, corresponding to any of the above embodiments, this disclosure also provides a belt speed detection device for a conveyor belt.

[0123] refer to Figure 6 The conveyor belt speed detection device includes: The preprocessing module 301 is configured to acquire multiple frames of conveyor belt images at preset time intervals, preprocess the conveyor belt images to obtain a target image, and determine the mapping relationship between image pixel displacement and actual physical displacement. The first conveyor belt speed determination module 302 is configured to use a pre-trained deep learning model to perform image processing on the target image to obtain a pixel optical flow field, and determine the first conveyor belt speed according to the pixel optical flow field, the mapping relationship and the preset time interval. The second conveyor belt speed determination module 303 is configured to determine a feature point set based on the pixels of the conveyor belt region in the target image, determine a target matching point pair set from the feature point set through a feature matching algorithm, and determine the second conveyor belt speed according to the target matching point pair set, the mapping relationship and the preset time interval. The target conveyor belt speed determination module 304 is configured to perform weighted processing on the first conveyor belt speed and the second conveyor belt speed to obtain the target conveyor belt speed.

[0124] In some embodiments, the preprocessing module 301 includes: The preprocessing unit is configured to perform grayscale processing on the conveyor belt image to obtain a grayscale image, perform denoising processing on the grayscale image through Gaussian filtering to obtain a denoised image, and perform brightness normalization processing on the denoised image to obtain a target image. The camera parameter determination unit is configured to acquire multiple calibration plate images of a preset calibration plate on a conveyor belt, and determine the camera's intrinsic parameter matrix, distortion coefficients, and extrinsic parameters based on the multiple calibration plate images. The mapping relationship generation unit is configured to determine the pixel physical size based on the intrinsic parameter matrix, the distortion coefficient and the extrinsic parameter, and generate a mapping relationship between the image pixel displacement and the actual physical displacement according to the pixel physical size.

[0125] In some embodiments, the pre-trained deep learning model includes: a feature encoder, a context encoder, a similarity calculator, and an updater; The first conveyor belt speed determination module 302 includes: The feature similarity matrix determination unit is configured to use the feature encoder to perform feature extraction processing on the target image to obtain a first feature vector of the current frame image and a second feature vector of the adjacent frame image, and to perform an inner product operation on the first feature vector and the second feature vector to obtain a feature similarity matrix; The global semantic information determination unit is configured to use the context encoder to perform feature extraction processing on the target image to obtain a first feature vector of the current frame image, and determine global semantic information based on the first feature vector. The semantic similarity determination unit is configured to use the similarity calculator to convert the feature similarity matrix into a multi-scale feature representation, and determine semantic similarity based on the multi-scale feature representation; The pixel optical flow field determination unit is configured to use the updater to update the initial optical flow value based on the semantic similarity and the global semantic information to obtain an updated optical flow value, to smooth the updated optical flow value to obtain a smoothed optical flow value, and to use the smoothed optical flow value with the number of iterations reaching a preset threshold as the pixel optical flow field.

[0126] In some embodiments, the first conveyor belt speed determining module 302 includes: The first average pixel displacement determination unit is configured to determine the first pixel displacement in the direction of conveyor belt movement from the pixel optical flow field, and to perform averaging processing on multiple first pixel displacements in the conveyor belt area to obtain the first average pixel displacement. The first actual displacement determination unit is configured to determine the first actual displacement corresponding to the first average pixel displacement based on the mapping relationship. The first conveyor belt speed determination unit is configured to use the ratio of the first actual displacement to the preset time interval as the first conveyor belt speed.

[0127] In some embodiments, the second conveyor belt speed determining module 303 includes: The image determination unit is configured to determine a reference frame image and a frame image to be matched from the target image; The gradient covariance matrix determination unit is configured to determine a first directional gradient and a second directional gradient based on the conveyor belt region in the target image, and to determine the gradient covariance matrix based on the first directional gradient and the second directional gradient. The feature point set construction unit is configured to determine the corner response value based on the gradient covariance matrix, and construct a feature point set based on the pixels whose corner response values ​​are greater than a preset response threshold; wherein, the feature point set includes a reference frame feature point set and a frame feature point set to be matched; The descriptor determination unit is configured to generate reference frame pixels and match frame pixels based on the feature point set according to a Gaussian distribution, and determine the descriptor of the reference frame pixels by comparing the gray values ​​of the reference frame pixels and the gray values ​​of the match frame pixels. The initial matching point pair set construction unit is configured to determine the Hamming distance between the reference frame pixel and the frame pixel to be matched based on the descriptor, and construct an initial matching point pair set based on the reference frame pixel and the frame pixel to be matched whose Hamming distance is less than a preset distance threshold; The target matching point pair set determination unit is configured to determine a preset number of matching point pairs from the initial matching point pair set, determine the translation vector of the matching point pairs, and determine the target matching point pair set based on the deviation between the matching point pairs and the translation vector.

[0128] In some embodiments, the second conveyor belt speed determining module 303 includes: The second average pixel displacement determination unit is configured to determine the second average pixel displacement based on each pair of matching points in the target matching point pair set; The second actual displacement determination unit is configured to determine the second actual displacement corresponding to the second average pixel displacement based on the mapping relationship. The second conveyor belt speed determination unit is configured to use the ratio of the second actual displacement to the preset time interval as the second conveyor belt speed.

[0129] In some embodiments, the target conveyor belt speed determination module 304 includes: An initial weight determination unit is configured to acquire the brightness and contrast of the target image, and determine a first initial weight and a second initial weight based on the brightness and the contrast. The target weight determination unit is configured to normalize the first initial weight to obtain a first target weight, and to normalize the second initial weight to obtain a second target weight. The target conveyor belt speed determination unit is configured to perform weighted processing on the first conveyor belt speed and the second conveyor belt speed based on the first target weight and the second target weight to obtain the target conveyor belt speed.

[0130] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this disclosure, the functions of each module can be implemented in one or more software and / or hardware.

[0131] The apparatus described above is used to implement the belt speed detection method of the corresponding conveyor belt in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0132] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the belt speed detection method of any of the above embodiments.

[0133] Figure 7 This embodiment illustrates a more specific hardware structure of an electronic device. The device may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0134] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0135] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0136] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0137] The communication interface 1040 is used to connect the communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB (Universal Serial Bus), network cable, etc.) or wireless means (such as mobile network, WIFI (Wireless Fidelity), Bluetooth, etc.).

[0138] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0139] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0140] The electronic devices described above are used to implement the belt speed detection method of the corresponding conveyor belt in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0141] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the belt speed detection method of the conveyor belt as described in any of the above embodiments.

[0142] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0143] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the belt speed detection method of the conveyor belt as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0144] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides a computer program product, including computer program instructions. When the computer program instructions are run on a computer, the computer causes the computer to execute the conveyor belt speed detection method as described in any of the above embodiments, which has the beneficial effects of the corresponding method embodiments, and will not be repeated here.

[0145] It is understood that before using the technical solutions of the various embodiments in this disclosure, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0146] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations of this disclosed technical solution.

[0147] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0148] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0149] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure is limited to these examples; within the framework of this disclosure, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this disclosure as described above, which are not provided in detail for the sake of brevity.

[0150] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this disclosure, the provided drawings may or may not show well-known power / ground connections to integrated circuit (IC) chips and other components. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this disclosure, and this also takes into account the fact that the details of implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this disclosure will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this disclosure, it will be apparent to those skilled in the art that the embodiments of this disclosure can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0151] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0152] This disclosure is intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this disclosure. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A method for detecting the belt speed of a conveyor belt, characterized in that, The method includes: Multiple frames of conveyor belt images are acquired at preset time intervals, the conveyor belt images are preprocessed to obtain target images, and the mapping relationship between image pixel displacement and actual physical displacement is determined. The target image is processed using a pre-trained deep learning model to obtain a pixel optical flow field, and the first conveyor belt speed is determined based on the pixel optical flow field, the mapping relationship, and the preset time interval. A feature point set is determined based on the pixels in the conveyor belt region of the target image. A target matching point pair set is determined from the feature point set using a feature matching algorithm. The second conveyor belt speed is determined based on the target matching point pair set, the mapping relationship, and the preset time interval. The target conveyor belt speed is obtained by weighting the speeds of the first and second conveyor belts.

2. The method according to claim 1, characterized in that, The step of preprocessing the conveyor belt image to obtain the target image and determining the mapping relationship between image pixel displacement and actual physical displacement includes: The conveyor belt image is converted to grayscale to obtain a grayscale image. The grayscale image is then denoised using Gaussian filtering to obtain a denoised image. Finally, the denoised image is normalized to obtain the target image. Multiple calibration board images of a preset calibration board are acquired on the conveyor belt, and the camera's intrinsic parameter matrix, distortion coefficients, and extrinsic parameters are determined based on the multiple calibration board images; The pixel physical size is determined based on the intrinsic parameter matrix, the distortion coefficient, and the extrinsic parameter, and a mapping relationship between image pixel displacement and actual physical displacement is generated based on the pixel physical size.

3. The method according to claim 1, characterized in that, The pre-trained deep learning model includes: a feature encoder, a context encoder, a similarity calculator, and an updater; The step of using a pre-trained deep learning model to process the target image to obtain the pixel optical flow field includes: The feature encoder is used to perform feature extraction processing on the target image to obtain the first feature vector of the current frame image and the second feature vector of the adjacent frame image. The first feature vector and the second feature vector are then processed by inner product operation to obtain the feature similarity matrix. The context encoder is used to perform feature extraction processing on the target image to obtain the first feature vector of the current frame image, and global semantic information is determined based on the first feature vector. The similarity calculator is used to convert the feature similarity matrix into a multi-scale feature representation, and semantic similarity is determined based on the multi-scale feature representation; The updater updates the initial optical flow value based on the semantic similarity and the global semantic information to obtain the updated optical flow value. The updated optical flow value is then smoothed to obtain the smoothed optical flow value. The smoothed optical flow value with the number of iterations reaching a preset threshold is used as the pixel optical flow field.

4. The method according to claim 1, characterized in that, Determining the first conveyor belt speed based on the pixel optical flow field, the mapping relationship, and the preset time interval includes: The first pixel displacement in the direction of conveyor belt movement is determined from the pixel optical flow field, and the first average pixel displacement is obtained by averaging multiple first pixel displacements in the conveyor belt area. The first actual displacement corresponding to the first average pixel displacement is determined based on the mapping relationship. The ratio of the first actual displacement to the preset time interval is used as the first conveyor belt speed.

5. The method according to claim 1, characterized in that, The step of determining a feature point set based on pixels in the conveyor belt region of the target image, and determining a target matching point pair set from the feature point set using a feature matching algorithm, includes: Determine the reference frame image and the frame image to be matched from the target image; A first directional gradient and a second directional gradient are determined based on the conveyor belt region in the target image, and a gradient covariance matrix is ​​determined based on the first directional gradient and the second directional gradient. The corner response value is determined based on the gradient covariance matrix, and a feature point set is constructed based on the pixels whose corner response values ​​are greater than a preset response threshold; wherein, the feature point set includes a reference frame feature point set and a frame feature point set to be matched; Based on the Gaussian distribution, a reference frame pixel and a matching frame pixel are generated according to the feature point set. The descriptor of the reference frame pixel is determined by comparing the gray values ​​of the reference frame pixel and the matching frame pixel. The Hamming distance between the reference frame pixel and the frame pixel to be matched is determined according to the descriptor, and an initial set of matching point pairs is constructed based on the reference frame pixel and the frame pixel to be matched whose Hamming distance is less than a preset distance threshold. A preset number of matching point pairs are determined from the initial set of matching point pairs, the translation vector of the matching point pairs is determined, and the target set of matching point pairs is determined based on the deviation between the matching point pairs and the translation vector.

6. The method according to claim 1, characterized in that, The step of determining the second conveyor belt speed based on the target matching point pair set, the mapping relationship, and the preset time interval includes: A second average pixel displacement is determined for each pair of matching points in the target matching point set; The second actual displacement corresponding to the second average pixel displacement is determined based on the mapping relationship. The ratio of the second actual displacement to the preset time interval is used as the second conveyor belt speed.

7. The method according to claim 1, characterized in that, The step of weighting the first conveyor belt speed and the second conveyor belt speed to obtain the target conveyor belt speed includes: The brightness and contrast of the target image are obtained, and a first initial weight and a second initial weight are determined based on the brightness and contrast. The first initial weight is normalized to obtain the first target weight, and the second initial weight is normalized to obtain the second target weight; The target conveyor speed is obtained by weighting the first conveyor belt speed and the second conveyor belt speed based on the first target weight and the second target weight.

8. A belt speed detection device for a conveyor belt, characterized in that, include: The preprocessing module is configured to acquire multiple frames of conveyor belt images at preset time intervals, preprocess the conveyor belt images to obtain a target image, and determine the mapping relationship between image pixel displacement and actual physical displacement. The first conveyor belt speed determination module is configured to use a pre-trained deep learning model to perform image processing on the target image to obtain a pixel optical flow field, and determine the first conveyor belt speed based on the pixel optical flow field, the mapping relationship and the preset time interval. The second conveyor belt speed determination module is configured to determine a set of feature points based on the pixels of the conveyor belt region in the target image, determine a set of target matching point pairs from the set of feature points through a feature matching algorithm, and determine the second conveyor belt speed according to the set of target matching point pairs, the mapping relationship and the preset time interval. The target conveyor belt speed determination module is configured to perform weighted processing on the first conveyor belt speed and the second conveyor belt speed to obtain the target conveyor belt speed.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor, when executing the program, implements the method as claimed in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions for causing a computer to perform the method according to any one of claims 1 to 7.

Images