Method, apparatus and storage medium for detecting misalignment of a cabinet conveyor

Abstract

The application provides a method and device for detecting offset of a storage conveyor belt and a storage medium. The method comprises: collecting original image data of the storage conveyor belt; preprocessing the original image data to obtain candidate image data; reconstructing the candidate image data to obtain target image data, with edge enhancement as the reconstruction target; detecting sub-pixels in the target image data; fitting a circular arc where the edge of the storage conveyor belt is located according to the sub-pixels; and detecting the offset of the storage conveyor belt according to the circular arc. The embodiment has high reliability in a complex industrial environment, is suitable for monitoring the offset of a conveyor belt in the tobacco industry, and effectively improves the accuracy of detecting the edge of the storage conveyor belt.

Description

Technical Field

[0001] This invention belongs to the technical field of computer vision, and in particular relates to a method, device and storage medium for detecting the offset of a storage tank conveyor belt. Background Technology

[0002] In cigarette manufacturing plants, the storage conveyor belt transports tobacco along a specific route. Due to factors such as installation position deviations and cigarette distribution deviations, the conveyor belt may deviate to a certain extent. Therefore, it is necessary to regularly check the deviation of the storage conveyor belt to ensure production continuity and product quality.

[0003] Currently, vision-based solutions are commonly used to detect the offset of storage tank conveyor belts. Industrial cameras are used to acquire image data of the storage tank conveyor belt, and operators such as Canny and Sobel are used to extract the edge contour of the storage tank conveyor belt from the image data. This contour is then compared with a reference contour (when the conveyor belt is centered) to detect the offset of the storage tank conveyor belt.

[0004] However, operators such as Canny and Sobel have low accuracy in edge detection of storage tank conveyor belts, resulting in low accuracy in detecting the offset of the storage tank conveyor belts. Summary of the Invention

[0005] In view of this, the present invention provides a method, device and storage medium for detecting the offset of a storage tank conveyor belt, so as to improve the accuracy of detecting the offset of the storage tank conveyor belt.

[0006] A first aspect of the present invention provides a method for detecting the offset of a storage tank conveyor belt, comprising: Acquire raw image data of the storage tank conveyor belt; The original image data is preprocessed to obtain candidate image data; Image reconstruction is performed on the candidate image data with edge enhancement as the reconstruction target to obtain the target image data; Detecting subpixels in the target image data; The arc containing the edge of the storage tank conveyor belt is fitted based on the subpixel fit; The offset of the storage tank conveyor belt is detected based on the arc.

[0007] Optionally, the step of reconstructing the candidate image data with edge enhancement as the reconstruction target to obtain the target image data includes: A model for edge enhancement is determined; the image reconstruction includes a first encoder, a second encoder, a third encoder, a fourth encoder, a bridging layer, a first decoder, a second decoder, a third decoder, and a convolutional layer; The candidate image data is input into the first encoder and encoded into a first image feature; The first image feature is input into the second encoder and encoded into the second image feature; The second image feature is input into the third encoder and encoded into a third image feature; The third image feature is input into the fourth encoder and encoded as the fourth image feature; The fourth image feature is input into the bridging layer for processing to obtain the fifth image feature; The fifth image feature and the third image feature are input into the first decoder and decoded into the sixth image feature; The sixth image feature and the second image feature are input into the second decoder and decoded into the seventh image feature; The seventh image feature and the first image feature are input into the third decoder and decoded into the eighth image feature; The eighth image feature is input into the convolutional layer to extract the ninth image feature; The ninth image feature is activated as target image data.

[0008] Optionally, the first encoder sequentially includes a convolutional layer, a batch normalization layer, a linear rectifier unit, and an inverse residual module; The second encoder, the third encoder, and the fourth encoder each include an inverse residual module and an attention module based on compression and excitation, respectively. The first decoder, the second decoder, and the third decoder each include an upsampling layer, a splicing layer, and a convolutional layer in sequence.

[0009] Optionally, it also includes: Based on the first sub-loss value of the target image data in edge classification; Based on the second sub-loss value of the target image data in edge segmentation; The first sub-loss value and the second sub-loss value are combined into a total loss value; The image reconstruction model is updated based on the total loss value.

[0010] Optionally, the first sub-loss value is represented as: ; w p =N n / (N p +N n ); w n =N p / (N p +N n ); Among them, L WN is the first sub-loss value. p N represents the number of pixels belonging to the edge. n w represents the number of pixels belonging to the background, N represents the total number of pixels in the target image data, and w p For positive sample weights, w n For negative sample weights, p i Let y be the probability that pixel i is predicted as an edge. i The label for pixel i.

[0011] Optionally, the second sub-loss value is expressed as: ; Among them, L C The second sub-loss value is given by N, where N is the total number of pixels in the target image data, and p... i Let y be the probability that pixel i is predicted as an edge. i Let ε be the label of pixel i, and ε be a constant.

[0012] Optionally, the step of fusing the first sub-loss value and the second sub-loss value into a total loss value includes: The total loss value is obtained by adding the product of the first sub-loss value and the preset first loss weight, and the product of the second sub-loss value and the preset second loss weight.

[0013] Optionally, detecting sub-pixels in the target image data includes: In each row of the target image data, locate the pixel with the largest feature response value and use it as a candidate pixel; Add a window of a specified half-width centered on the candidate pixel; Within the window, the product between the column position of the pixel and the feature response value of the pixel is added to obtain a first reference value; In the window, the feature response values ​​of each pixel are summed to obtain a second reference value; The ratio between the first reference value and the second reference value is calculated to obtain the sub-pixel.

[0014] A second aspect of the present invention provides a deflection detection device for a storage tank conveyor belt, comprising: The image acquisition module is used to acquire raw image data of the storage tank conveyor belt; The preprocessing module is used to preprocess the original image data to obtain candidate image data; The edge enhancement module is used to reconstruct the candidate image data with edge enhancement as the reconstruction target to obtain the target image data; Subpixel detection module, used to detect subpixels in the target image data; An arc fitting module is used to fit the arc where the edge of the storage tank conveyor belt is located based on the sub-pixel; The offset detection module is used to detect the offset of the storage tank conveyor belt based on the arc.

[0015] Optionally, the edge enhancement module includes: A model determination module is used to determine an image reconstruction model for edge enhancement; the image reconstruction includes a first encoder, a second encoder, a third encoder, a fourth encoder, a bridging layer, a first decoder, a second decoder, a third decoder, and a convolutional layer; The first encoding module is used to input the candidate image data into the first encoder and encode it into a first image feature; The second encoding module is used to input the first image features into the second encoder and encode them into second image features; The third encoding module is used to input the second image feature into the third encoder and encode it into a third image feature; The fourth encoding module is used to input the third image feature into the fourth encoder and encode it into a fourth image feature; The bridging module is used to input the fourth image feature into the bridging layer for processing to obtain the fifth image feature; The first decoding module is used to input the fifth image feature and the third image feature into the first decoder to decode them into a sixth image feature; The second decoding module is used to input the sixth image feature and the second image feature into the second decoder to decode them into the seventh image feature; The third decoding module is used to input the seventh image feature and the first image feature into the third decoder to decode them into the eighth image feature; The convolution module is used to input the eighth image feature into the convolution layer to extract the ninth image feature; The activation module is used to activate the ninth image feature into target image data.

[0016] Optionally, the first encoder sequentially includes a convolutional layer, a batch normalization layer, a linear rectifier unit, and an inverse residual module; The second encoder, the third encoder, and the fourth encoder each include an inverse residual module and an attention module based on compression and excitation, respectively. The first decoder, the second decoder, and the third decoder each include an upsampling layer, a splicing layer, and a convolutional layer in sequence.

[0017] Optionally, it also includes: The first sub-loss value calculation module is used to calculate the first sub-loss value of the target image data in edge classification. The second sub-loss value calculation module is used to calculate the second sub-loss value of the target image data in edge segmentation. The total loss value fusion module is used to fuse the first sub-loss value and the second sub-loss value into a total loss value; The model update module is used to update the image reconstruction model based on the total loss value.

[0018] Optionally, the first sub-loss value is represented as: ; w p =N n / (N p +N n ); w n =N p / (N p +N n ); Among them, L W N is the first sub-loss value. p N represents the number of pixels belonging to the edge. n w represents the number of pixels belonging to the background, N represents the total number of pixels in the target image data, and w p For positive sample weights, w n For negative sample weights, p i Let y be the probability that pixel i is predicted as an edge. i The label for pixel i.

[0019] In one embodiment of the present invention, the second sub-loss value is expressed as: ; Among them, L C The second sub-loss value is given by N, where N is the total number of pixels in the target image data, and p... i Let y be the probability that pixel i is predicted as an edge. i Let ε be the label of pixel i, and ε be a constant.

[0020] Optionally, the total loss value fusion module includes: The weighted fusion module is used to add the product of the first sub-loss value and the preset first loss weight, and the product of the second sub-loss value and the preset second loss weight, to obtain the total loss value.

[0021] Optionally, the sub-pixel detection module 304 includes: The pixel localization module is used to locate the pixel with the largest feature response value in each row of the target image data as a candidate pixel; A window adding module is used to add a window with a specified half-width centered on the candidate pixel; The first reference value calculation module is used to add the product between the column position of the pixel and the feature response value of the pixel within the window to obtain the first reference value; The second reference value calculation module is used to add the feature response values ​​of each pixel in the window to obtain the second reference value. The sub-pixel positioning module is used to calculate the ratio between the first reference value and the second reference value to obtain the sub-pixel.

[0022] A third aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the offset detection method for a storage tank conveyor belt as described in the first aspect above.

[0023] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the offset detection method for the storage tank conveyor belt as described in the first aspect above.

[0024] A fifth aspect of the present invention provides a computer program product that, when run on a computer, causes the computer to perform the offset detection method for the storage tank conveyor belt as described in the first aspect above.

[0025] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: In this embodiment, raw image data of the storage tank conveyor belt is acquired; the raw image data is preprocessed to obtain candidate image data; image reconstruction is performed on the candidate image data with edge enhancement as the reconstruction target to obtain target image data; sub-pixels are detected in the target image data; the arc where the edge of the storage tank conveyor belt is located is fitted based on the sub-pixels; and the offset of the storage tank conveyor belt is detected based on the arc. This embodiment uses image reconstruction to perform edge enhancement on the image data of the storage tank conveyor belt, adaptively achieving robust edge detection based on the actual situation of the storage tank conveyor belt. It uses sub-pixel positioning and arc fitting to accurately describe the edge morphology of the storage tank conveyor belt, thereby calculating the offset of the storage tank conveyor belt. It maintains high reliability in complex industrial environments and is suitable for conveyor belt offset monitoring in industries such as tobacco, effectively improving the accuracy of edge detection of storage tank conveyor belts. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of a method for detecting the offset of a storage tank conveyor belt according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of an image reconstruction model provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a deflection detection device for a storage tank conveyor belt provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0028] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the present invention. However, those skilled in the art will recognize that the present application may be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted to avoid unnecessary detail that could obscure the description of the present application.

[0029] The technical solution of the present invention will be illustrated below through specific embodiments.

[0030] Reference Figure 1 The diagram illustrates a method for detecting the offset of a storage tank conveyor belt according to an embodiment of the present invention, which may specifically include the following steps: Step 101: Collect raw image data of the storage tank conveyor belt.

[0031] In this embodiment, an industrial camera is installed in the tobacco production environment. The field of view (FOV) of the industrial camera covers the industrial camera, and the industrial camera is calibrated. The industrial camera is periodically or irregularly called to collect raw image data on the storage tank conveyor belt.

[0032] Step 102: Preprocess the original image data to obtain candidate image data.

[0033] In this embodiment, the original image data can be preprocessed to obtain candidate image data, thereby improving image quality and facilitating subsequent offset detection tasks.

[0034] The preprocessing includes at least one of the following: Grayscale conversion, distortion correction based on camera calibration information, image contrast, and noise reduction.

[0035] Step 103: Reconstruct the candidate image data using edge enhancement as the reconstruction target to obtain the target image data.

[0036] In this embodiment, image reconstruction can be performed on candidate image data based on deep learning image reconstruction technology to obtain target image data. During image reconstruction, the main data of the storage tank conveyor belt is kept stable and the edge data of the storage tank conveyor belt is enhanced to realize edge enhancement detection of the storage tank conveyor belt and improve the detection accuracy.

[0037] In one embodiment of the present invention, step 103 may include the following steps: Step 1031: Determine the image reconstruction model for edge enhancement.

[0038] In this embodiment, an image reconstruction model can be constructed and trained based on deep learning technology, so that the image reconstruction model can be used to enhance the edge data in the image data when reconstructing the image data.

[0039] Among them, such as Figure 2 As shown, the image reconstruction model includes a first encoder, a second encoder, a third encoder, a fourth encoder, a bridging layer, a first decoder, a second decoder, a third decoder, and a convolutional layer.

[0040] The image reconstruction model has a simple structure and is suitable for rapid edge enhancement in industrial applications.

[0041] Step 103-12: Input the candidate image data into the first encoder and encode it into the first image feature.

[0042] In this embodiment, candidate image data is input into a first encoder, which encodes the candidate image data and outputs a first image feature.

[0043] For example, the first encoder sequentially includes a convolutional layer (Conv), a batch normalization layer (BatchNorm), a linear rectified unit (ReLU6), and an inverse residual module (MBConv6).

[0044] In this example, the convolutional layer extracts features from the candidate image data, the batch normalization layer performs normalization processing on the features output by the convolutional layer, the linear rectifier unit performs activation processing on the features output by the batch normalization layer, and the inverse residual module further extracts features from the features output by the linear rectifier unit to obtain the first image features.

[0045] Steps 103-13: Input the first image features into the second encoder and encode them into second image features.

[0046] In this embodiment, the first image feature is input into the second encoder, the second encoder encodes the first image feature, and outputs the second image feature.

[0047] For example, the second encoder includes, in sequence, an inverse residual module (MBConv6) and a compression and excitation-based attention module (SE Attention).

[0048] In this example, the inverse residual module extracts features from the first image features, and the attention module based on compression and excitation adaptively adjusts the channel weights of the features output by the inverse residual module to obtain the second image features.

[0049] Steps 103-14: Input the second image features into the third encoder and encode them into third image features.

[0050] In this embodiment, the second image features are input into the third encoder, the third encoder encodes the second image features, and outputs the third image features.

[0051] For example, the third encoder includes, in turn, an inverse residual module (MBConv6) and a compression-and-excitation-based attention module (SE Attention).

[0052] In this example, the inverse residual module extracts features from the second image features, and the attention module based on compression and excitation adaptively adjusts the channel weights of the features output by the inverse residual module to obtain the third image features.

[0053] Steps 103-15: Input the third image features into the fourth encoder and encode them into the fourth image features.

[0054] In this embodiment, the third image feature is input into the fourth encoder, the fourth encoder encodes the third image feature, and outputs the fourth image feature.

[0055] For example, the fourth encoder includes, in turn, an inverse residual module (MBConv6) and a compression-and-excitation-based attention module (SE Attention).

[0056] In this example, the inverse residual module extracts features from the third image features, and the attention module based on compression and excitation adaptively adjusts the channel weights of the features output by the inverse residual module to obtain the fourth image features.

[0057] In this embodiment, the use of inverse residual modules and compression- and excitation-based attention modules in multiple stages can effectively improve edge detection capabilities, suppress noise effects, and enhance the robustness of edge enhancement.

[0058] Steps 103-16: Input the fourth image feature into the bridging layer for processing to obtain the fifth image feature.

[0059] In this embodiment, the fourth image feature is input to the bridging layer and processed by linear transformation, dimension compression / expansion, feature flattening / reshaping, transpose, etc., to obtain the fifth image feature. This ensures that the output of the encoding module (first encoder, second encoder, third encoder, fourth encoder) is compatible with the input of the decoding module (first decoder, second decoder, third decoder), thereby achieving efficient feature transmission, adaptation and initialization.

[0060] Steps 103-17: Input the fifth image feature and the third image feature into the first decoder to decode them into the sixth image feature.

[0061] In this embodiment, the fifth image feature and the third image feature are simultaneously input into the first decoder. The first decoder decodes the fifth image feature and the third image feature to obtain the sixth image feature.

[0062] For example, the first decoder includes an upsampling layer, a concatenation layer, and a convolutional layer.

[0063] In this example, the upsampling layer upsamples the features of the fifth image, the concatenation layer uses functions such as Concatenate to concatenate the features output by the upsampling layer with the features of the third image, and the convolutional layer further extracts features from the features output by the concatenation layer to obtain the features of the sixth image.

[0064] The first decoder uses a skip connection to combine low-resolution and high-resolution information, providing more refined features for edge and background segmentation.

[0065] Steps 103-18: Input the sixth image feature and the second image feature into the second decoder to decode them into the seventh image feature.

[0066] In this embodiment, the sixth image feature and the second image feature are simultaneously input into the second decoder, and the second decoder decodes the sixth image feature and the second image feature to obtain the seventh image feature.

[0067] For example, the second decoder includes an upsampling layer, a concatenate layer, and a convolutional layer.

[0068] In this example, the upsampling layer upsamples the features of the sixth image, the concatenation layer uses functions such as Concatenate to concatenate the features output by the upsampling layer with the features of the second image, and the convolutional layer further extracts features from the features output by the concatenation layer to obtain the features of the seventh image.

[0069] The second decoder uses a skip connection to combine low-resolution and high-resolution information, providing more refined features for edge and background segmentation.

[0070] Steps 103-19: Input the seventh image feature and the first image feature into the third decoder to decode them into the eighth image feature.

[0071] In this embodiment, the seventh image feature and the first image feature are simultaneously input into the third decoder. The third decoder decodes the seventh image feature and the first image feature to obtain the eighth image feature.

[0072] For example, the third decoder includes an upsampling layer, a concatenate layer, and a convolutional layer.

[0073] In this example, the upsampling layer upsamples the features of the seventh image, the concatenation layer uses functions such as Concatenate to concatenate the features output by the upsampling layer with the features of the first image, and the convolutional layer further extracts features from the features output by the concatenation layer to obtain the features of the eighth image.

[0074] The third decoder uses a skip connection to combine low-resolution and high-resolution information, providing more refined features for edge and background segmentation.

[0075] Steps 103-20: Input the eighth image features into the convolutional layer to extract the ninth image features.

[0076] In this embodiment, the eighth image feature is input into the convolutional layer to extract features, resulting in the ninth image feature.

[0077] Step 103-21: Activate the ninth image feature as target image data.

[0078] In this embodiment, functions such as Sigmoid are used to activate the ninth image feature to obtain the target image data.

[0079] In one embodiment of the present invention, the image reconstruction model can be trained in the following manner: Step 103-31: Based on the first sub-loss value of the target image data in edge classification.

[0080] During the training phase of the image reconstruction model, candidate image data are labeled with labels, which include information such as edges and background.

[0081] For edge classification (i.e., distinguishing pixels as edges or background), the first sub-loss value can be calculated based on the difference between the target image data and the label.

[0082] For example, the first sub-loss value is represented as: ; w p =N n / (N p +N n ); w n =N p / (N p +N n ); Among them, L W N is the first sub-loss value. p N represents the number of pixels belonging to the edge. n w represents the number of pixels belonging to the background, N represents the total number of pixels in the target image data, and w p For positive sample weights, w n For negative sample weights, p i Let y be the probability that pixel i is predicted as an edge. i The label for pixel i.

[0083] Steps 103-32: Based on the second sub-loss value of the target image data in edge segmentation.

[0084] For edge segmentation (i.e. semantic segmentation of image data, dividing edge regions into background regions), a second sub-loss value can be calculated based on the difference between the labels of the target image data.

[0085] For example, the second sub-loss value is expressed as: ; Among them, L C The second sub-loss value is given by N, where N is the total number of pixels in the target image data, and p... i Let y be the probability that pixel i is predicted as an edge. i Let ε be the label of pixel i, and ε be a constant.

[0086] Steps 103-33: Combine the first sub-loss value and the second sub-loss value into a total loss value.

[0087] In this embodiment, the first sub-loss value and the second sub-loss value can be fused into a total loss value using a linear or non-linear method.

[0088] For example, the total loss value is obtained by adding the product of the first sub-loss value and the preset first loss weight, and the product of the second sub-loss value and the preset second loss weight.

[0089] In this example, the total loss value is expressed as: L T =αL W +βL C ; Among them, L T L represents the total loss value. W L is the first sub-loss value. C α is the second sub-loss value, α is the first loss weight, and β is the second loss weight.

[0090] Steps 103-34: Update the image reconstruction model based on the total loss value.

[0091] In this embodiment, the total loss value can be substituted into optimization algorithms such as SGD (stochastic gradient descent) and Adam (adaptive momentum) to calculate the update magnitude of the parameters in the image reconstruction model network, and the parameters in the image reconstruction model network are updated according to the update magnitude.

[0092] This embodiment trains the image reconstruction model using loss values ​​from multiple dimensions, including classification and semantic segmentation, which effectively improves the robustness of the image reconstruction model.

[0093] Step 104: Detect subpixels in the target image data.

[0094] A pixel is the smallest discrete sampling unit of image data. In the image coordinate system, the horizontal and vertical coordinates of a pixel are integers (such as (100,200)). It is the result of the camera discretizing the real continuous scene and has an inherent upper limit to the sampling accuracy.

[0095] Subpixel coordinates are sub-pixel level positions with higher precision than single pixel coordinates (e.g., (100.27, 200.53)). They are not physically existing pixels, but coordinates that are derived from the grayscale information of discrete pixels through mathematical methods such as grayscale moment calculation, function fitting, and interpolation. They are close to the coordinates of a real continuous scene and can usually achieve a positioning accuracy of 0.01 to 0.1 pixels.

[0096] In this embodiment, sub-pixels can be detected in the target image data based on information from the target image data itself.

[0097] In practical implementation, the accuracy of sub-pixel detection can be improved through the following two-stage detection: 1. Coarse positioning The algorithm iterates through each row of pixels in the target image data, locates the pixel with the largest feature response value in each row as a candidate pixel, and thus locks the core region of each row's edge, avoiding traversing the entire row during sub-pixel calculations and improving efficiency.

[0098] Coarse positioning can be represented as: x max (y)=argmax x F(x,y), where y is the row index and x is the column index, represents the feature response value of the pixel at coordinates (x,y). A larger feature response value indicates a higher probability that the pixel is a true edge. max (y) represents the candidate pixels in the y-th row.

[0099] 2. Precise positioning Add a window of a specified half-width centered on the candidate pixel.

[0100] On the one hand, within this window, the product between the column position of the pixel and the feature response value of the pixel is added together to obtain the first reference value.

[0101] On the other hand, in this window, the feature response values ​​of each pixel are summed to obtain a second reference value.

[0102] The ratio between the first reference value and the second reference value is calculated to obtain the sub-pixel.

[0103] Therefore, subpixels can be represented as: ; Where, x sub (y) represents the sub-pixel of the y-th row, x max Let x be the candidate pixel, w be the half-width of the window, k be the window offset ranging from [-w, w], and x be the offset. max +k represents the x-th node within the window. max +k columns, F(x) max +k,y) represents the x-th node in the window. max +k column, y-th row characteristic response value.

[0104] Step 105: Fit the arc of the edge of the storage tank conveyor belt based on subpixel fit.

[0105] The edges of the conveyor belt on the storage tank rollers often exhibit an arc shape under tension. Using circular arc fitting can more accurately describe its true shape than linear fitting.

[0106] In arc fitting, a real arc is a continuous and smooth curve, while discrete sampling of image data can cause jagged or stepped edges on the arc. If edge points of integer pixels are used for fitting, significant geometric deviations will occur, resulting in distortion of the calculated results of the center, radius, and curvature.

[0107] In this embodiment, the arc (including origin, radius, and other information) where the edge of the storage tank conveyor belt is located can be fitted using methods such as the least squares method based on subpixels, eliminating the accuracy loss of discrete sampling, obtaining high-precision edge points, and greatly improving the accuracy of arc fitting.

[0108] Step 106: Detect the offset of the storage tank conveyor belt based on the arc.

[0109] In this embodiment, the offset of the storage tank conveyor belt can be calculated using camera calibration information, the outermost point of the arc, the intersection of a specific straight line and the arc, and other data.

[0110] In this embodiment, raw image data of the storage tank conveyor belt is acquired; the raw image data is preprocessed to obtain candidate image data; image reconstruction is performed on the candidate image data with edge enhancement as the reconstruction target to obtain target image data; sub-pixels are detected in the target image data; the arc where the edge of the storage tank conveyor belt is located is fitted based on the sub-pixels; and the offset of the storage tank conveyor belt is detected based on the arc. This embodiment uses image reconstruction to perform edge enhancement on the image data of the storage tank conveyor belt, adaptively achieving robust edge detection based on the actual situation of the storage tank conveyor belt. It uses sub-pixel positioning and arc fitting to accurately describe the edge morphology of the storage tank conveyor belt, thereby calculating the offset of the storage tank conveyor belt. It maintains high reliability in complex industrial environments and is suitable for conveyor belt offset monitoring in industries such as tobacco, effectively improving the accuracy of edge detection of storage tank conveyor belts.

[0111] It should be noted that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0112] Reference Figure 3 The diagram shows a schematic of a deflection detection device for a storage tank conveyor belt according to an embodiment of the present invention, which may specifically include the following modules: Image acquisition module 301 is used to acquire raw image data of the storage tank conveyor belt; Preprocessing module 302 is used to preprocess the original image data to obtain candidate image data; Edge enhancement module 303 is used to reconstruct the candidate image data with edge enhancement as the reconstruction target to obtain target image data; Subpixel detection module 304 is used to detect subpixels in the target image data; The arc fitting module 305 is used to fit the arc where the edge of the storage tank conveyor belt is located based on the sub-pixel; The offset detection module 306 is used to detect the offset of the storage tank conveyor belt based on the arc.

[0113] In one embodiment of the present invention, the edge enhancement module 303 includes: A model determination module is used to determine an image reconstruction model for edge enhancement; the image reconstruction includes a first encoder, a second encoder, a third encoder, a fourth encoder, a bridging layer, a first decoder, a second decoder, a third decoder, and a convolutional layer; The first encoding module is used to input the candidate image data into the first encoder and encode it into a first image feature; The second encoding module is used to input the first image features into the second encoder and encode them into second image features; The third encoding module is used to input the second image feature into the third encoder and encode it into a third image feature; The fourth encoding module is used to input the third image feature into the fourth encoder and encode it into a fourth image feature; The bridging module is used to input the fourth image feature into the bridging layer for processing to obtain the fifth image feature; The first decoding module is used to input the fifth image feature and the third image feature into the first decoder to decode them into a sixth image feature; The second decoding module is used to input the sixth image feature and the second image feature into the second decoder to decode them into the seventh image feature; The third decoding module is used to input the seventh image feature and the first image feature into the third decoder to decode them into the eighth image feature; The convolution module is used to input the eighth image feature into the convolution layer to extract the ninth image feature; The activation module is used to activate the ninth image feature into target image data.

[0114] In one embodiment of the present invention, the first encoder sequentially includes a convolutional layer, a batch normalization layer, a linear rectifier unit, and an inverse residual module; The second encoder, the third encoder, and the fourth encoder each include an inverse residual module and an attention module based on compression and excitation, respectively. The first decoder, the second decoder, and the third decoder each include an upsampling layer, a splicing layer, and a convolutional layer in sequence.

[0115] In one embodiment of the present invention, it further includes: The first sub-loss value calculation module is used to calculate the first sub-loss value of the target image data in edge classification. The second sub-loss value calculation module is used to calculate the second sub-loss value of the target image data in edge segmentation. The total loss value fusion module is used to fuse the first sub-loss value and the second sub-loss value into a total loss value; The model update module is used to update the image reconstruction model based on the total loss value.

[0116] In one embodiment of the present invention, the first sub-loss value is expressed as: ; w p =N n / (N p +N n ); w n =N p / (N p +N n ); Among them, L W N is the first sub-loss value. p N represents the number of pixels belonging to the edge. n w represents the number of pixels belonging to the background, N represents the total number of pixels in the target image data, and w p For positive sample weights, w n For negative sample weights, p i Let y be the probability that pixel i is predicted as an edge. i The label for pixel i.

[0117] In one embodiment of the present invention, the second sub-loss value is expressed as: ; Among them, L C The second sub-loss value is given by N, where N is the total number of pixels in the target image data, and p... i Let y be the probability that pixel i is predicted as an edge. i Let ε be the label of pixel i, and ε be a constant.

[0118] In one embodiment of the present invention, the total loss value fusion module includes: The weighted fusion module is used to add the product of the first sub-loss value and the preset first loss weight, and the product of the second sub-loss value and the preset second loss weight, to obtain the total loss value.

[0119] In one embodiment of the present invention, the sub-pixel detection module 304 includes: The pixel localization module is used to locate the pixel with the largest feature response value in each row of the target image data as a candidate pixel; A window adding module is used to add a window with a specified half-width centered on the candidate pixel; The first reference value calculation module is used to add the product between the column position of the pixel and the feature response value of the pixel within the window to obtain the first reference value; The second reference value calculation module is used to add the feature response values ​​of each pixel in the window to obtain the second reference value. The sub-pixel positioning module is used to calculate the ratio between the first reference value and the second reference value to obtain the sub-pixel.

[0120] The present invention provides a deflection detection device for a storage tank conveyor belt. By using the deflection detection device for the storage tank conveyor belt, the various steps in the aforementioned deflection detection method embodiments for the storage tank conveyor belt can be implemented.

[0121] It should be noted that the module division in the various storage tank conveyor belt offset detection devices provided in the above embodiments is illustrative and only represents a logical functional division. In actual implementation, other division methods may also be used. Furthermore, the functional modules in the various embodiments of this invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0122] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the technical solution of the embodiments of the present invention can be embodied in the form of a computer program product, which is stored in a computer storage medium and includes several instructions to cause an electronic device or processor to execute all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned computer storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0123] Furthermore, the offset detection device for the storage tank conveyor belt provided in the above embodiments and the offset detection method for the storage tank conveyor belt belong to the same concept. For details of its implementation process, please refer to the method embodiments, which will not be repeated here.

[0124] Reference Figure 4 The diagram illustrates an electronic device according to an embodiment of the present invention. Figure 4As shown, the electronic device in this embodiment of the invention includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the above-described embodiment of the offset detection method for the storage tank conveyor belt. Alternatively, when the processor executes the computer program, it implements the functions of each module in the above-described embodiment of the offset detection device for the storage tank conveyor belt.

[0125] For example, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete this application. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which can be used to describe the execution process of the computer program in the electronic device.

[0126] The electronic device may be a desktop computer, a cloud server, or other computing device. The electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 4 This is merely one example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0127] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0128] The memory can be an internal storage unit of the electronic device, such as a hard drive or RAM. Alternatively, it can be an external storage device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory can include both internal and external storage units. The memory is used to store the computer program and other programs and data required by the electronic device. The memory can also be used to temporarily store data that has been output or will be output.

[0129] This invention also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the offset detection method for the storage tank conveyor belt as described in the foregoing embodiments.

[0130] This invention also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the offset detection method for the storage tank conveyor belt as described in the foregoing embodiments.

[0131] This invention also discloses a computer program product that, when run on a computer, causes the computer to execute the offset detection method for the storage tank conveyor belt described in the foregoing embodiments.

[0132] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for detecting the offset of a storage tank conveyor belt, characterized in that, include: Acquire raw image data of the storage tank conveyor belt; The original image data is preprocessed to obtain candidate image data; Image reconstruction is performed on the candidate image data with edge enhancement as the reconstruction target to obtain the target image data; Detecting subpixels in the target image data; The arc containing the edge of the storage tank conveyor belt is fitted based on the subpixel fit; The offset of the storage tank conveyor belt is detected based on the arc.

2. The method according to claim 1, characterized in that, The process of reconstructing the candidate image data with edge enhancement as the reconstruction target to obtain the target image data includes: A model for edge enhancement is determined; the image reconstruction includes a first encoder, a second encoder, a third encoder, a fourth encoder, a bridging layer, a first decoder, a second decoder, a third decoder, and a convolutional layer; The candidate image data is input into the first encoder and encoded into a first image feature; The first image feature is input into the second encoder and encoded into the second image feature; The second image feature is input into the third encoder and encoded into a third image feature; The third image feature is input into the fourth encoder and encoded as the fourth image feature; The fourth image feature is input into the bridging layer for processing to obtain the fifth image feature; The fifth image feature and the third image feature are input into the first decoder and decoded into the sixth image feature; The sixth image feature and the second image feature are input into the second decoder and decoded into the seventh image feature; The seventh image feature and the first image feature are input into the third decoder and decoded into the eighth image feature; The eighth image feature is input into the convolutional layer to extract the ninth image feature; The ninth image feature is activated as target image data.

3. The method according to claim 2, characterized in that, The first encoder sequentially includes a convolutional layer, a batch normalization layer, a linear rectifier unit, and an inverse residual module; The second encoder, the third encoder, and the fourth encoder each include an inverse residual module and an attention module based on compression and excitation, respectively. The first decoder, the second decoder, and the third decoder each include an upsampling layer, a splicing layer, and a convolutional layer in sequence.

4. The method according to claim 2, characterized in that, Also includes: Based on the first sub-loss value of the target image data in edge classification; Based on the second sub-loss value of the target image data in edge segmentation; The first sub-loss value and the second sub-loss value are combined into a total loss value; The image reconstruction model is updated based on the total loss value.

5. The method according to claim 4, characterized in that, The first sub-loss value is expressed as: ； w p =N n / (N p +N n ); w n =N p / (N p +N n ); Among them, L W N is the first sub-loss value. p N represents the number of pixels belonging to the edge. n w represents the number of pixels belonging to the background, N represents the total number of pixels in the target image data, and w p For positive sample weights, w n For negative sample weights, p i Let y be the probability that pixel i is predicted as an edge. i The label for pixel i.

6. The method according to claim 4, characterized in that, The second sub-loss value is expressed as: ； Among them, L C The second sub-loss value is given by N, where N is the total number of pixels in the target image data, and p... i Let y be the probability that pixel i is predicted as an edge. i Let ε be the label of pixel i, and ε be a constant.

7. The method according to claim 4, characterized in that, The step of fusing the first sub-loss value and the second sub-loss value into a total loss value includes: The total loss value is obtained by adding the product of the first sub-loss value and the preset first loss weight, and the product of the second sub-loss value and the preset second loss weight.

8. The method according to any one of claims 1-7, characterized in that, The detection of sub-pixels in the target image data includes: In each row of the target image data, locate the pixel with the largest feature response value and use it as a candidate pixel; Add a window of a specified half-width centered on the candidate pixel; Within the window, the product between the column position of the pixel and the feature response value of the pixel is added to obtain a first reference value; In the window, the feature response values ​​of each pixel are summed to obtain a second reference value; The ratio between the first reference value and the second reference value is calculated to obtain the sub-pixel.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the offset detection method for the storage tank conveyor belt as described in any one of claims 1-8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the offset detection method for the storage tank conveyor belt as described in any one of claims 1-8.

Images