A deep learning-based tubular belt deviation grading detection method and system

Abstract

The present application relates to the technical field of tubular belt state monitoring, and proposes a tubular belt deviation grading detection method and system based on deep learning, which comprises: acquiring first, second and third collection images of the tubular belt and performing enhancement preprocessing on the images to obtain preprocessed three-view images; inputting the three-view images into a deep learning key point detection model to obtain key point information of the tubular belt joint; using a situation analysis algorithm to analyze the key point quantity information and the key point position information, detecting the current working state of the tubular belt, dividing to obtain a mild, moderate and severe deviation state grade, triggering corresponding early warning or intervention measures according to the deviation state grade; and synchronously transmitting the detection result to a visual detection system to realize cloud-edge collaborative management. The present application solves the problems of existing contact detection methods, such as installation difficulty, inconvenience of maintenance, limited detection precision and high cost.

Description

Technical Field

[0001] This invention relates to the field of tubular belt condition monitoring technology, and in particular to a deep learning-based method and system for classifying and detecting tubular belt misalignment. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] As a new type of conveying equipment, tubular belt conveyors have advantages such as good dustproof performance, high transportation efficiency, and the ability to achieve large-angle conveying, and have been widely used in the field of bulk material conveying.

[0004] However, due to the structural characteristics of tubular belt conveyors, belt misalignment is prone to occur during operation. When this happens, the edges of the tubular belt easily rub against the idlers, leading to wear and even damage to the belt edges. In severe cases, localized friction can generate heat, potentially causing fires and other safety accidents. Therefore, while ensuring continuous production, timely detection of tubular belt misalignment and appropriate corrective measures based on the degree of misalignment are crucial for ensuring the safe and stable operation of the equipment.

[0005] In the existing technology, contact sensors are usually used to monitor the misalignment of tubular belts. However, due to the special closed tubular structure of tubular belts and the complex on-site working conditions, contact detection methods have problems such as difficult installation, inconvenient maintenance, limited detection accuracy, and high cost, making it difficult to accurately detect the misalignment level of tubular belts. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention provides a deep learning-based method and system for classifying and detecting misalignment of tubular belts, which solves the problems of difficult installation, inconvenient maintenance, limited detection accuracy, and high cost of existing contact detection methods.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of this invention provides a deep learning-based method for classifying and detecting misalignment of tubular belts, comprising: The first, second, and third images of the tubular belt were acquired and enhanced by preprocessing to obtain the preprocessed three-view images. The three-view images are input into a deep learning key point detection model to obtain key point information of the tubular belt joint. The situation analysis algorithm is used to analyze the number and location information of key points in the key point information, detect the current working status of the tubular belt, and divide it into three levels of deviation status: mild, moderate and severe. Corresponding early warning or intervention measures are triggered according to the deviation status level. The detection results are simultaneously transmitted to the big data management and control platform, and remote real-time monitoring, maintenance and operation parameter adjustment are performed through the remote computing server. Abnormal images are stored in the cloud to achieve cloud-edge collaborative management and control.

[0008] As a further technical solution, a first image acquisition unit positioned directly above the tubular belt, a second image acquisition unit positioned to the left of the tubular belt, and a third image acquisition unit positioned to the right of the tubular belt are used to capture images of the tubular belt by means of adjacent side overlapping shooting to obtain the first acquired image, the second acquired image, and the third acquired image.

[0009] As a further technical solution, the deep learning key point detection model includes a backbone network, a feature fusion network, and a detection head; Multi-scale feature extraction is performed through the backbone network, and feature fusion network is used to stitch and fuse feature maps of different scales; the key point pixel coordinates of the overlapping edge of the tubular belt seam are regressed through the detection head, and the number of key points identified in a single image is output.

[0010] As a further technical solution, the situation analysis algorithm analyzes the number and location information of key points in the key point information, including: Obtain the number of seam key points output in a single image, and determine the branch status of the working state based on the number of key points. If the number of key points reaches the first threshold, the working state of the tubular belt is determined to be that the tubular belt has undergone severe twisting and deformation. If the number of key points meets the second threshold, it is determined that there is no tortuous deformation of the tubular belt. The acquired key points are connected or fitted in sequence to generate a joint identification line segment to characterize the direction of the tubular belt joint. The current deviation status level of the tubular belt is determined based on the joint identification line segment.

[0011] As a further technical solution, the current misalignment level of the tubular belt is determined based on the seam marking line segment, specifically as follows: Construct a reference coordinate system with the standard circle center of the tubular belt cross-section as the origin, and within this coordinate system, define the deviation threshold limits corresponding to different deviation status levels along different radial angles based on the set angular tolerance. The deflection position and intersection relationship of the seam marking line segment relative to the deviation threshold are calculated, and the relative position information is compared with a preset standard to determine the current deviation status level of the tubular belt.

[0012] As a further technical solution, the deviation status level is at least divided into mild deviation, moderate deviation and severe deviation; among which, torsional deformation belongs to severe deviation. When the deviation is determined to be mild or moderate, the vision detection system generates a warning signal of the corresponding level and prompts manual intervention; when the deviation is determined to be severe, the vision detection system generates a shutdown alarm signal.

[0013] As a further technical solution, the visual inspection system includes a big data management and control platform, a remote computing server, and a cloud; the collected images are transmitted to the remote computing server via the network for algorithm inference, and the algorithm deployed in the remote computing server analyzes whether the tubular belt has deviated, stores the status information of the tubular belt deviating in the big data management and control platform, and stores the abnormal images of the tubular belt in the cloud.

[0014] A second aspect of the present invention provides a deep learning-based system for classifying and detecting misalignment of tubular belts, comprising: The image acquisition module is configured to acquire a first, second, and third image of the tubular belt and perform enhancement preprocessing on the images to obtain a preprocessed three-view image. The key point detection module is configured to input the three-view images into a deep learning key point detection model to obtain key point information of the tubular belt joint; The situation analysis module is configured to: use a situation analysis algorithm to analyze the number and location information of key points in the key point information, detect the current working status of the tubular belt, divide it into three levels of deviation status: mild, moderate and severe, and trigger corresponding early warning or intervention measures according to the deviation status level. The collaborative management module is configured to: synchronously transmit the detection results to the big data management platform, remotely monitor, maintain and adjust the operating parameters in real time through the remote computing server, and store abnormal images in the cloud to achieve cloud-edge collaborative management.

[0015] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of a deep learning-based method for classifying and detecting misalignment of a tubular belt as described in the first aspect of the present invention.

[0016] A fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a deep learning-based tubular belt misalignment classification detection method as described in the first aspect of the present invention.

[0017] The above one or more technical solutions have the following beneficial effects: This invention addresses the unique enclosed tubular structure of tubular belt conveyors by employing a method of overlapping adjacent image capture with three image acquisition units positioned at the top, left, and right sides, eliminating potential blind spots in single or dual-view perspectives. By combining deep learning-based keypoint detection with spatial geometric analysis, it can not only accurately extract joint locations but also effectively identify extreme conditions such as severe distortions, significantly improving the accuracy and comprehensiveness of tubular belt operation status detection. An end-to-end full-vision detection scheme is adopted, inputting simultaneous frame images of the tubular belt captured by three vision sensors. Compared to the difficulties in installation, easy wear, and high maintenance costs of traditional contact sensors, the use of non-contact vision sensors simplifies equipment placement, does not interfere with normal belt operation, and is easy to install and maintain, significantly reducing detection and maintenance costs. It achieves refined graded control of belt misalignment, classifying misalignment into mild, moderate, and severe states through multi-dimensional situational analysis. The system can automatically output corresponding early warning signals or shutdown alarm signals according to different levels, thereby guiding manual intervention or linkage with the correction mechanism in a timely manner, effectively avoiding serious safety accidents such as belt tearing or local friction heat generation causing fire, and ensuring the continuity and safety of production.

[0018] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0019] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0020] Figure 1 This is a flowchart of the tubular belt misalignment classification detection method of the first embodiment; Figure 2 This is a schematic diagram of the hardware installation for the first embodiment; Figure 3 This is a schematic diagram of the twisted joint of the tubular belt in the first embodiment. Figure 4 This is a schematic diagram illustrating the deviation threshold boundary division in the first embodiment; Figure 5 This is a schematic diagram illustrating the entire process of real-time display of the system detection screen in the first embodiment.

[0021] In the diagram, 1. First visual sensor; 2. Second visual sensor; 3. Third visual sensor; 4. Remote computing server; 5. Tubular belt seam; 6. Key point of the seam; 7. Tubular belt twist; 8. First-level left deviation limit; 9. Second-level left deviation limit; 10. Third-level left deviation limit; 11. First-level right deviation limit; 12. Second-level right deviation limit; 13. Third-level right deviation limit; 14. First acquired image; 15. Second acquired image; 16. Third acquired image. Detailed Implementation

[0022] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0023] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0024] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0025] Example 1 like Figure 1 As shown, this embodiment discloses a deep learning-based method for classifying and detecting misalignment of tubular belts, including: Step 1: Acquire the first acquisition image 14, the second acquisition image 15, and the third acquisition image 16 of the tubular belt and perform enhancement preprocessing on the images to obtain the preprocessed three-view images.

[0026] Specifically, the images of the tubular belt are acquired by an image acquisition unit. The image acquisition unit includes a first image acquisition unit positioned directly above the tubular belt, a second image acquisition unit positioned to the left of the tubular belt, and a third image acquisition unit positioned to the right of the tubular belt. The first, second, and third image acquisition units capture images of the tubular belt using an adjacent-side overlapping imaging method, obtaining a first acquired image 14, a second acquired image 15, and a third acquired image 16, respectively.

[0027] like Figure 2 As shown, in this embodiment, the first image acquisition unit includes a first visual sensor 1, the second image acquisition unit includes a second visual sensor 2, and the third image acquisition unit includes a third visual sensor 3. The first visual sensor 1 is positioned directly above the tubular belt; the second visual sensor 2 is positioned directly to the left of the tubular belt; and the third visual sensor 3 is positioned directly to the right of the tubular belt. The target of both visual sensors is the tubular belt.

[0028] The visual sensors positioned directly above, to the left, and to the right are adjusted to ensure that the captured images are free from reflections, overexposure, glare, and halo effects, thus guaranteeing image clarity. The acquired raw images undergo image enhancement and grayscale processing to suppress environmental interference from complex industrial conditions such as high dust levels and low illumination. Feature filtering and redundant information removal are performed to obtain a high-contrast grayscale image of the tubular belt, thereby improving the contrast of the tubular belt's surface and seam features.

[0029] In this embodiment, each vision sensor must have a unique ID code to determine the time of belt misalignment and belt number, so as to facilitate accurate positioning of the belt misalignment.

[0030] In the obtained grayscale image of the tubular belt, the state characteristics of the tubular belt joint 5 are obvious. The first visual sensor 1 at the top is used to determine the working characteristics of the tubular belt joint 5, the second visual sensor 2 at the left is used to determine the degree of left deviation of the tubular belt, and the third visual sensor 3 at the right is used to determine the degree of right deviation of the tubular belt.

[0031] Further, the image acquisition unit transmits the acquired images to the visual detection system. The visual detection system receives the first acquired image 14, the second acquired image 15, and the third acquired image 16, and transmits them to a deep learning key point detection model for feature extraction. Based on the key point features extracted by the model, the position information of the tubular belt joint 5 is obtained. The working state of the tubular belt is detected through the position information and situation analysis algorithm, and its working state is judged to be related to the preset deviation threshold position. If the deviation threshold position is exceeded, an early warning is issued. The tubular belt deviation state detection and early warning specifically includes steps 2-3: Step 2: Construct and train a deep learning keypoint detection model; input the preprocessed three-view images into the pre-trained deep learning keypoint detection model to obtain keypoint information of the tubular belt joint 5.

[0032] In this embodiment, the deep learning key point detection model preferably adopts the YOLOv11-pose network model, and the purpose of detecting the seam key points 6 is to comprehensively judge the working status of the tubular belt.

[0033] Step 2.1: The training process of the deep learning keypoint detection model is as follows: Image data containing the joint area of ​​the tubular belt were collected, and key point annotation information was constructed at the beginning, middle and end of the joint to form a key point sample dataset of the tubular belt joint. The image dataset with the key point information annotated was divided into training set and test set.

[0034] The training set was fed into the deep learning keypoint detection model YOLOv11-pose for model training, where the key training parameters were set as follows: learning rate 0.012, training epochs 500, training image input size 640*640, and batch size 64.

[0035] Specifically, during training, the input image is first preprocessed and augmented, including random scaling, random translation, random flipping, and brightness perturbation, to improve the model's generalization ability. The processed image is then input into the YOLOv11-pose network, whose structure includes a backbone network, a feature fusion network, and a keypoint detection head.

[0036] Multi-level feature extraction is performed on images containing the tubular belt seam area through a backbone network to obtain feature maps at different scales. A feature fusion network is used to fuse features at different scales to enhance the model's ability to express local detail features and global structural features of the tubular belt seam. The fused features are then input into a keypoint detection head to predict the position coordinates and confidence information of the keypoints at the beginning, middle, and end of the seam.

[0037] Based on the error between the prediction results and the key point annotation information, a key point localization loss function is constructed. The gradient of the network parameters is calculated through the backpropagation algorithm, and the model parameters are iteratively updated using the gradient descent optimization algorithm to continuously reduce the key point localization error.

[0038] When the preset training rounds or loss function convergence conditions are met, a well-trained deep learning keypoint detection model is obtained. The trained model is then validated using a test set to evaluate the keypoint localization accuracy and generalization performance. The optimal model parameters are then saved as the final detection model.

[0039] The keypoint detection head outputs the x-coordinate, y-coordinate, and corresponding confidence score of each keypoint. The network parameters are jointly optimized by keypoint regression loss and target confidence loss. A multi-scale training strategy is adopted to improve the model's ability to detect tubular belt seam areas under different shooting distances and lighting conditions.

[0040] Step 2.2: In actual testing, the specific steps for obtaining key point information of the tubular belt joint are as follows: Step 2.2.1: After obtaining the grayscale image of the tubular belt, input it into the trained deep learning key point detection model based on a single-stage architecture, use its backbone network to extract multi-scale feature information in the image, and use the feature fusion network to stitch and fuse features of different scales.

[0041] Specifically, feature analysis, feature fusion, and feature detection are performed on the extracted feature maps: deep global structural features are extracted through image convolutional layers and spatial pyramid pooling layers (SPPF), and multiple feature maps of different scales are output, which are denoted as high-level semantic feature maps, mid-level feature maps, and low-level detail feature maps, respectively.

[0042] The output feature map is input into a feature fusion network, which includes an upsampling layer, a feature concatenation layer, a convolutional fusion layer, and a downsampling convolutional layer. First, the upsampling layer scales the high-level semantic feature map to make its spatial resolution comparable to that of features at adjacent scales. Figure 1 First, the upsampled feature map is concatenated with the corresponding shallow feature map at the same scale using a feature concatenation layer (Concat layer). The concatenated feature map is then input into a convolutional fusion layer, where convolution operations are used to reorganize features and fuse information, enhancing the ability to represent edge, texture, and spatial information in the seam region. The fused feature map is then passed to the next scale via a downsampling convolutional layer and fused again with features from adjacent scales, forming a feature pyramid structure containing multi-scale contextual information. After processing by the feature fusion network, fused feature maps at multiple detection scales can be obtained, improving the detection capability of tubular belt seam targets of different sizes.

[0043] Step 2.2.2: Regress the pixel coordinates of key points on the overlapping edge of the tubular belt seam using the key point detection head of the deep learning key point detection model, and output the number of key points identified in a single image.

[0044] Specifically, the detection head includes a convolutional prediction layer, a keypoint regression branch, and a target confidence branch: the fused multi-scale feature maps are input into the detection head; the detection head first further compresses and reconstructs the features through several convolutional prediction layers; then it is divided into multiple prediction branches.

[0045] (1) Keypoint Regression Branch The keypoint regression branch consists of convolutional layers and linear regression layers; it is used to predict the position coordinates of keypoints at the lap joint of tubular belt seams. ; (2) Key point confidence branch The keypoint confidence branch consists of a convolutional layer and a sigmoid activation layer; it is used to output the probability of the existence of the corresponding keypoint. ,when If the key point is valid, T is the pre-set confidence threshold of the key point detection algorithm; otherwise, it is considered an invalid key point.

[0046] The model's detection head is based on an anchor-free mechanism to simultaneously regress the bounding box information of the tubular belt seam region and the two-dimensional pixel coordinates and visibility scores of the key point 6 of the tubular belt seam. The non-maximum suppression (NMS) operator is used to remove redundant detection targets based on the confidence of the bounding box and the intersection-union ratio. Finally, the key point position information corresponding to the optimal detection target is extracted and output.

[0047] In this embodiment, the YOLOv11-pose model detection process is as follows: The acquired images of the tubular conveyor belt in operation are input into the trained deep learning keypoint detection model. First, the images are input into the backbone network in RGB three-channel format. Through multi-layer convolution operations and feature extraction modules, texture, edge, contour, and seam structure features in the images are extracted to generate feature maps containing information at different semantic levels.

[0048] Subsequently, feature maps at different scales are fused using a feature pyramid structure to achieve joint expression of shallow detail features and deep semantic features, thereby enhancing the model's ability to perceive features of the tubular belt joint area and improving the accuracy of key point detection at different scales and under different working conditions.

[0049] The fused feature information is input into the keypoint detection head to predict the spatial coordinates and corresponding confidence information of the key points at the beginning, middle, and end of the tubular belt joint. Based on the keypoint prediction results, the location and orientation of the joint area are determined, and the obtained keypoint coordinate information serves as the data basis for subsequent situational analysis algorithms.

[0050] Step 3: Perform situation analysis algorithm analysis based on key point information to determine the motion status of the tubular belt.

[0051] Step 3.1: Analyze the number and location information of key points using a situational analysis algorithm to detect the current working status of the tubular conveyor belt and determine if there is any abnormal belt misalignment. Specifically, this includes: Step 3.1.1: Obtain the number of seam key points 6 output in a single image, and determine the branch of the working state based on the number of key points; if the number of key points reaches the first threshold, it is determined that the tubular belt is twisted 7 and has undergone severe twisting deformation, and its current working state is directly classified as a severe deviation state; if the number of key points meets the second threshold, it is determined that there is no twisting deformation of the tubular belt, and it is classified as a non-severe deviation state. The obtained key points are connected or fitted in sequence to generate seam identification line segments used to characterize the direction of the tubular belt seam 5.

[0052] In this embodiment, the number of key points 6 of the detected tubular belt joint are statistically analyzed and their spatial relationships are analyzed.

[0053] When only one set of key seam points is detected, including the beginning key point, the middle key point and the end key point, a total of 3 key points, it is determined that there is no distortion. The 3 key points are connected in sequence using line segments to form the seam center line, and the degree of deviation of the tubular belt is judged based on the offset of the center line relative to the preset reference position.

[0054] When 4 or 5 key points are detected, it indicates that the previously hidden seam area is partially exposed, and the tubular belt is determined to be in a state of twisting development. Specifically, detecting 4 key points is considered a slight twisting state, and detecting 5 key points is considered a moderate twisting state. At this point, the center line of the possible seam is fitted, and the degree of twisting is evaluated by combining the key point spacing, center line offset, and angle change.

[0055] like Figure 3 As shown, when six key points are detected, two sets of complete joint key points are formed, indicating that the upper and lower joints of the tubular belt are exposed simultaneously, which is judged as a severe twisting state. The degree of twisting is determined based on the distance, angle, and positional relationship between the center lines of the two sets of joints, and corresponding alarm information and operational status evaluation results are output.

[0056] Step 3.1.2: Construct a reference coordinate system with the standard circle center of the tubular belt cross-section as the origin, and within this coordinate system, define the deviation threshold limits corresponding to different deviation levels along different radial angles based on the set angular tolerance.

[0057] like Figure 4 As shown, in this embodiment, if there is no twisting of the tubular belt joint, three levels of deviation limits are set for the left side: Level 1 left deviation limit 8, Level 2 left deviation limit 9, and Level 3 left deviation limit 10; and three levels of deviation limits are set for the right side: Level 1 right deviation limit 11, Level 2 right deviation limit 12, and Level 3 right deviation limit 13.

[0058] Preferably, the first-level left deviation limit 8 and the first-level right deviation limit 11 are set in the video image captured by the first vision sensor 1. The first vision sensor 1 is responsible for shooting the area directly above the tubular belt. Only when the deviation is slight and the tubular belt is still within the offset range of this shooting area, it belongs to the first-level deviation, corresponding to the first-level left and right deviation limit judgment.

[0059] The secondary left deviation limit 9 and the tertiary left deviation limit 10 are set in the video image captured by the second vision sensor 2; the secondary right deviation limit 12 and the tertiary right deviation limit 13 are set in the video image captured by the third vision sensor 3. That is, when the deviation of the tubular belt exceeds the primary limit and shifts to a larger area on the left, it enters the shooting area of ​​the second vision sensor 2, which identifies the secondary and tertiary left deviations accordingly. Similarly, when the tubular belt shifts to the right beyond the primary limit, the third vision sensor 3, which is responsible for shooting the right area, identifies the secondary and tertiary right deviations.

[0060] Step 3.1.3: Calculate the deflection position and intersection relationship of the seam segment relative to the misalignment threshold limit, and compare this relative position information with the preset standard to determine the current misalignment level of the tubular belt. The preset standard is set by on-site workers based on production site requirements.

[0061] In this embodiment, the top left corner of the acquired image is taken as the origin of the image pixel system, and the coordinates of all the center pixels of the tubular belt seam key point 6 are determined by a deep learning key point detection model. .

[0062] Determine the equation of the straight line on the image pixel coordinate system for the left deviation limit. Determine the equation of the straight line representing the right deviation limit in the image pixel coordinate system. ,in It's a deviation level. , It is the pixel coordinate number.

[0063] The center pixel coordinates of each tubular belt joint key point 6 are... of Input the equation of the line that limits the left deviation, and calculate it using the inverse function. , , Input the equation of the straight line that marks the right deviation limit, and calculate using the inverse function. , , .

[0064] when At that time, it was classified as a level one left deviation. At that time, it was classified as a level two left deviation. It was classified as a level three left deviation.

[0065] when At that time, it was classified as a level one right deviation. At that time, it was classified as a level two right deviation. It was classified as a level three right deviation.

[0066] when It doesn't mean it's off track.

[0067] Step 3.2: Trigger corresponding early warning or intervention measures based on the deviation status level. Specifically: The deviation status is classified into at least three levels: mild deviation, moderate deviation, and severe deviation. Specifically, Level 1 left and right deviation is considered mild, Level 2 left and right deviation is moderate, and Level 3 left and right deviation is severe. In this embodiment, once distortion occurs, i.e., the number of detected key points exceeds the second threshold of 3, it is considered a severe deviation.

[0068] When the deviation is determined to be mild or moderate, the vision inspection system generates a warning signal of the corresponding level and prompts manual intervention or triggers a minor correction mechanism. When the deviation is determined to be severe, the vision inspection system generates a shutdown alarm signal to prevent serious safety accidents such as tearing of the tubular belt or high temperature fire.

[0069] Step 4: Simultaneously transmit the detection results to the big data management and control platform, and remotely monitor, maintain and adjust the operating parameters in real time through the remote computing server 4. Abnormal images are stored in the cloud to realize cloud-edge collaborative management and control.

[0070] In this embodiment, the visual inspection system includes a big data management and control platform, a remote computing server 4, and a cloud. The visual sensor transmits the acquired images to the remote computing server 4 via the network for algorithm inference. The algorithm deployed in the remote computing server 4 analyzes whether the tubular belt has deviated. The status information of the tubular belt deviating is stored in the big data management and control platform, and the abnormal images of the tubular belt are stored in the cloud for subsequent status statistical analysis, hazard cause analysis, and historical data query.

[0071] In this embodiment, before the deep learning key point detection model is deployed to the remote computing server 4 of the visual inspection system, it undergoes model pruning and quantization processing, and is converted into intermediate representation formats such as ONNX or the inference engine format of a specific edge computing platform, in order to reduce the number of model parameters and improve the real-time inference speed of edge computing devices for multiple video streams.

[0072] like Figure 5 As shown, in this embodiment, the remote computing server 4 of the visual inspection system is also configured with a real-time video stream analysis and streaming module, which is used to realize hardware decoding, algorithm inference analysis, result rendering, and re-encoding and real-time presentation of video images from multiple video streams. The specific operation steps of this module include: 1. Multi-channel real-time video stream access and hardware decoding: The system accesses real-time video streams from various image acquisition units in the factory via the RTSP protocol. It then uses the FFmpeg framework to call the GPU hardware acceleration unit to efficiently decode the video streams, extracting tubular belt images frame by frame and converting them into NumPy matrix or tensor format to construct input data for real-time inference of deep learning models.

[0073] 2. Keypoint detection, feature rendering, and situational analysis: The decoded and converted image data is input into the key point detection algorithm model (preferably the YOLOv11-pose model) to obtain the two-dimensional pixel coordinate information of the key points at the five joints of the tubular belt; the obtained key points are connected in sequence to generate joint marking line segments, and combined with the situation analysis algorithm, the relative spatial relationship between the joint marking line segments and the system's preset deviation threshold boundary is calculated to automatically determine the current working status and deviation level of the tubular belt.

[0074] 3. Real-time video encoding and front-end streaming display: Image frames with superimposed detection results such as seam marker lines, deviation threshold limits, and deviation level status are re-encoded into RTSP video streams in H.264 or H.265 format using FFmpeg to call the GPU hardware acceleration unit. Then, a streaming media service program (such as MediaMTX) deployed on edge nodes or local servers converts the RTSP stream into a high real-time streaming media protocol format suitable for web front-ends (such as WebRTC, HTTP-FLV, or HLS), and distributes it to the front-end display platform for rendering and playback of the real-time monitoring interface.

[0075] Example 2 This embodiment discloses a deep learning-based system for classifying and detecting misalignment of tubular conveyor belts, including: The image acquisition module is configured to acquire a first, second, and third image of the tubular belt and perform enhancement preprocessing on the images to obtain a preprocessed three-view image. The key point detection module is configured to input three-view images into a deep learning key point detection model to obtain key point information of tubular belt joints; The situation analysis module is configured to: use situation analysis algorithms to analyze the number and location information of key points in the key point information, detect the current working status of the tubular conveyor belt, classify it into three levels of deviation status: mild, moderate and severe, and trigger corresponding early warning or intervention measures according to the deviation status level. The collaborative management module is configured to: synchronously transmit the detection results to the big data management platform, remotely monitor, maintain and adjust the operating parameters in real time through the remote computing server, and store abnormal images in the cloud to achieve cloud-edge collaborative management.

[0076] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0077] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a deep learning-based tubular belt misalignment classification detection method as described in Embodiment 1 of this disclosure.

[0078] Example 4 The purpose of this embodiment is to provide an electronic device.

[0079] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the deep learning-based tubular belt misalignment classification detection method as described in Embodiment 1 of this disclosure.

[0080] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0081] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0082] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A deep learning-based method for graded detection of tubular belt misalignment, characterized in that, include: The first, second, and third images of the tubular belt were acquired and enhanced by preprocessing to obtain the preprocessed three-view images. The three-view images are input into a deep learning key point detection model to obtain key point information of the tubular belt joint. The situation analysis algorithm is used to analyze the number and location information of key points in the key point information, detect the current working status of the tubular belt, and divide it into three levels of deviation status: mild, moderate and severe. Corresponding early warning or intervention measures are triggered according to the deviation status level. The detection results are simultaneously transmitted to the big data management and control platform, and remote real-time monitoring, maintenance and operation parameter adjustment are performed through the remote computing server. Abnormal images are stored in the cloud to achieve cloud-edge collaborative management and control.

2. The method for graded detection of tubular belt misalignment based on deep learning as described in claim 1, characterized in that, The first image acquisition unit, the second image acquisition unit, and the third image acquisition unit are located directly above the tubular belt, respectively. They capture images of the tubular belt by means of adjacent side overlapping shooting to obtain the first image, the second image, and the third image.

3. The deep learning-based method for graded detection of tubular belt misalignment as described in claim 1, characterized in that, The deep learning keypoint detection model includes a backbone network, a feature fusion network, and a detection head; Multi-scale feature extraction is performed through the backbone network, and feature fusion network is used to stitch and fuse feature maps of different scales; the key point pixel coordinates of the overlapping edge of the tubular belt seam are regressed through the detection head, and the number of key points identified in a single image is output.

4. The deep learning-based method for graded detection of tubular belt misalignment as described in claim 1, characterized in that, The situational analysis algorithm analyzes the number and location information of key points in the key point information, including: Obtain the number of seam key points output in a single image, and determine the branch status of the working state based on the number of key points. If the number of key points reaches the first threshold, the working state of the tubular belt is determined to be that the tubular belt has undergone severe twisting and deformation. If the number of key points meets the second threshold, it is determined that there is no tortuous deformation of the tubular belt. The acquired key points are connected or fitted in sequence to generate a joint identification line segment to characterize the direction of the tubular belt joint. The current deviation status level of the tubular belt is determined based on the joint identification line segment.

5. The deep learning-based method for graded detection of tubular belt misalignment as described in claim 4, characterized in that, The current misalignment level of the tubular belt is determined based on the seam marking line segment, specifically as follows: Construct a reference coordinate system with the standard circle center of the tubular belt cross-section as the origin, and within this coordinate system, define the deviation threshold limits corresponding to different deviation status levels along different radial angles based on the set angular tolerance. The deflection position and intersection relationship of the seam marking line segment relative to the deviation threshold are calculated, and the relative position information is compared with a preset standard to determine the current deviation status level of the tubular belt.

6. The deep learning-based method for graded detection of tubular belt misalignment as described in claim 4, characterized in that, The deviation status is classified into at least three levels: mild deviation, moderate deviation, and severe deviation; among which, torsional deformation belongs to the severe deviation status. When the deviation is determined to be mild or moderate, the vision detection system generates a warning signal of the corresponding level and prompts manual intervention; when the deviation is determined to be severe, the vision detection system generates a shutdown alarm signal.

7. The deep learning-based method for graded detection of tubular belt misalignment as described in claim 6, characterized in that, The visual inspection system includes a big data management and control platform, a remote computing server, and a cloud platform. The collected images are transmitted to the remote computing server via the network for algorithm inference. The algorithm deployed on the remote computing server analyzes whether the tubular belt has deviated. The status information of the tubular belt deviating is stored in the big data management and control platform, and the abnormal images of the tubular belt are stored in the cloud platform.

8. A deep learning-based system for classifying and detecting misalignment of tubular belts, characterized in that, include: The image acquisition module is configured to acquire a first, second, and third image of the tubular belt and perform enhancement preprocessing on the images to obtain a preprocessed three-view image. The key point detection module is configured to input the three-view images into a deep learning key point detection model to obtain key point information of the tubular belt joint; The situation analysis module is configured to: use a situation analysis algorithm to analyze the number and location information of key points in the key point information, detect the current working status of the tubular belt, divide it into three levels of deviation status: mild, moderate and severe, and trigger corresponding early warning or intervention measures according to the deviation status level. The collaborative management module is configured to: synchronously transmit the detection results to the big data management platform, remotely monitor, maintain and adjust the operating parameters in real time through the remote computing server, and store abnormal images in the cloud to achieve cloud-edge collaborative management.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by the processor, the program implements the steps in the deep learning-based tubular belt misalignment classification detection method as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the deep learning-based tubular belt misalignment classification detection method as described in any one of claims 1-7.

Images