Tubular belt of pipe belt machine positive and negative package abnormal monitoring method, device and storage medium

Abstract

The application discloses a tubular belt winding and unwinding abnormality monitoring method and device of a pipe belt machine and a storage medium, relates to the field of pipe belt machines, and comprises the following steps: acquiring original tubular belt images shot by at least three camera modules in a current period, performing image distortion correction, and obtaining tubular belt images; identifying the pipe winding belt in all the tubular belt images, calculating a target pipe twisting angle according to the position relationship between the pipe winding belt and the camera modules; inputting the tubular belt images into a winding and unwinding CNN prediction model, predicting the winding and unwinding state of the current tubular belt based on the winding and unwinding label state and the target pipe twisting angle, and outputting a prediction result; and triggering an alarm or shutdown of the pipe belt machine when the target pipe twisting angle exceeds a set threshold or the prediction output is an abnormal unwinding state. The method can realize real-time shooting of tubular belt images by the camera modules without modifying the equipment, calculate a pipe twisting angle, and predict a winding and unwinding state, thereby eliminating the need for manual inspection, reducing cost investment, and improving the response speed and detection accuracy of monitoring.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of pipe belt machines, in particular to a pipe belt positive and reverse wrapping abnormality monitoring method and device applied to a pipe belt machine and a storage medium. BACKGROUND

[0002] A pipe belt conveyor (referred to as a pipe belt machine) is developed on the basis of a belt conveyor. The pipe belt conveyor forces a conveying belt to be rolled into a pipe shape through six rollers.

[0003] The pipe belt machine is affected by factors such as mechanical part deformation or wear, insufficient manufacturing or installation precision of a polygonal roller and a support structure, deformation of a curved section conveying belt, and misalignment of the conveying belt during feeding, which causes the pipe conveying belt to be twisted to different degrees, and this situation is objective and cannot be avoided.

[0004] Once the pipe twisting angle is too large or the pipe is wrapped in reverse, the main hazards are as follows:

[0005] (1) It is easy to cause the conveying belt to be folded and wound into a drum at the discharge port.

[0006] (2) The edge of the conveying belt is inserted into the gap between the rollers, causing the belt to be torn or the structure to be damaged.

[0007] After the above hazards occur, a large amount of manpower and time is required to restore production. Traditional detection methods are to send people to patrol or use laser sensors, various angle measurement modules, communication modules, and other devices to modify the equipment, which is complicated, and can only calculate the pipe twisting angle, cannot predict the positive and reverse wrapping of the pipe, and the detection accuracy is not high. SUMMARY

[0008] Embodiments of the present application provide a pipe belt positive and reverse wrapping abnormality monitoring method and device for a pipe belt machine and a storage medium, which solve the problems of complicated detection of the pipe twisting angle and inability to predict the positive and reverse wrapping of the pipe.

[0009] In one aspect, the present application provides a pipe belt positive and reverse wrapping abnormality monitoring method for a pipe belt machine, which comprises:

[0010] S1, obtaining original pipe belt images captured by at least three camera modules in a current period and performing image distortion correction to obtain pipe belt images;

[0011] S2, identifying pipe wrapping belts in all the pipe belt images, and calculating a target pipe twisting angle according to the position relationship between the pipe wrapping belt and the camera module; at least one of the original pipe belt images captured by the camera modules in the same period displays a pipe wrapping edge image of the pipe belt;

[0012] S3, inputting the tubular tape image into a positive and negative packaging neural network CNN prediction model, predicting the positive and negative packaging state of the current tubular tape based on the model label state and the target pipe twisting angle, and outputting the prediction result; the positive and negative packaging state and the abnormal state of the tubular tape are determined based on the system and the pipe tape machine mechanical structure;

[0013] S4, when the target pipe twisting angle exceeds the set threshold value, or the prediction output is the negative packaging abnormal state, triggering the pipe tape machine alarm or shutdown.

[0014] Specifically, S1 includes:

[0015] Identifying the target detection object in the original tubular tape image, matching the calibrated matching points in the original image with the target detection object, and determining the matching points; the target detection object at least includes the tubular tape and the light source equipment above the left and right camera modules;

[0016] The original tubular tape image is corrected by the openCV calibration function and the matching points, and the tubular tape image is obtained.

[0017] Specifically, S2 includes:

[0018] S21, based on the target detection algorithm, identifying the pipe wrapping tape in each tubular tape image, displaying the pixel width X pixel of the tubular tape and the pixel width x pixel of the pipe wrapping tape; wherein the pipe wrapping tape is formed by wrapping and superimposing the strip edge, X pixel is the pixel width of the tubular tape center to the outer edge in the image, and x pixel is the pixel width of the tubular tape center to the pipe wrapping tape in the image;

[0019] S22, according to the number of each camera module, the radial width R of the tubular tape, and the position area of the pipe wrapping tape identified in the image, a pipe twisting model graph corresponding to the camera module view angle is constructed;

[0020] S23, based on the pipe twisting angle formula, the pipe twisting angle β under the corresponding camera number is calculated; the formula is as follows:

[0021]

[0022] Wherein, x represents the actual observation height of the pipe wrapping tape of the camera module, X represents the actual observation height of the tubular tape; the connecting line between the calibrated camera and the tubular tape center is the height reference line, and β represents the complementary angle between the connecting line of the center and the pipe wrapping tape position and the height reference line;

[0023] S24, the pipe twisting angles calculated under the view angle of each camera module are averaged to obtain the target pipe twisting angle.

[0024] Specifically, S24 includes:

[0025] Calculate the width of the pipe belt from the edge of the tubular belt in each of the tubular belt images respectively; when the distance width exceeds the monitoring range, the corresponding pipe twisting angle is filtered; wherein when the pipe belt is not identified in the tubular belt image, the pipe twisting angle calculation is not performed;

[0026] The pipe twisting angles that meet the distance width are calculated by averaging to obtain the target pipe twisting angle.

[0027] Specifically, S3 comprises:

[0028] S31, input the image after preprocessing of the tubular belt image into the positive and negative wrapping CNN prediction model, and pre-classify based on the pipe belt position and model classification label; the model classification label includes tubular belt positive wrapping label, tubular belt negative wrapping label and state unknown label;

[0029] S32, based on the identified pipe belt position, pre-classification result and fusion correction strategy, correct the state, and determine the corrected state as the positive and negative wrapping state of the tubular belt under the current view angle.

[0030] Specifically, the fusion correction strategy is provided with a misjudgment classification result on the left side of the image and a misjudgment classification result on the right side of the image; when the prediction classification result of the pipe belt position and the fusion correction strategy do not match, it indicates that the model prediction is wrong, and the state is corrected.

[0031] Specifically, when the left wrapping right is set as the positive wrapping state and the right wrapping left is set as the negative wrapping state, the fusion correction strategy comprises:

[0032] When the pipe belt is located on the left side of the image and the pre-classification result is positive wrapping, it is determined as a misjudgment prediction; when the pipe belt is located on the right side of the image and the pre-classification result is negative wrapping, it is determined as a misjudgment prediction, and the misjudgment prediction is corrected to an unknown state according to the fusion correction strategy;

[0033] Or,

[0034] When the left wrapping right is set as the negative wrapping state and the right wrapping left is set as the positive wrapping state, the fusion correction strategy comprises:

[0035] When the pipe belt is located on the left side of the image and the pre-classification result is negative wrapping, it is determined as a misjudgment prediction; when the pipe belt is located on the right side of the image and the pre-classification result is positive wrapping, it is determined as a misjudgment prediction, and the misjudgment prediction is corrected to an unknown state according to the fusion correction strategy.

[0036] Specifically, the preprocessing of the tubular belt image comprises:

[0037] Position the tubular belt in the position area of the tubular belt image, and intercept an RGB image of 128x128 size without left-right image flipping;

[0038] The forward and reverse wrapping CNN prediction model comprises at least 4 convolution layers and corresponding 4 pooling layers, 3 fully connected layers and a softmax layer, the convolution layers are all 3*3 convolution kernels with a step of 1*1, the pooling layers are based on maximum pooling with a convolution kernel size of 2*2 and a step of 2*2.

[0039] In another aspect, the application provides a tubular belt forward and reverse wrapping abnormality monitoring device of a pipe belt machine, which is used for the tubular belt forward and reverse wrapping abnormality monitoring method of the pipe belt machine in the above aspect, and the device comprises:

[0040] An image acquisition module is configured to acquire original tubular belt images captured by at least three groups of camera modules in a current period and perform image distortion correction to obtain tubular belt images.

[0041] A pipe twisting angle calculation module is configured to identify a wrapped pipe belt in all the tubular belt images, and calculate a target pipe twisting angle according to a position relationship between the wrapped pipe belt and the camera modules; the original tubular belt images captured by at least one group of camera modules in the same period display a pipe wrapping edge image of the tubular belt.

[0042] A state prediction module is configured to input the tubular belt images into a forward and reverse wrapping neural network (CNN) prediction model, predict a forward and reverse wrapping state of the current tubular belt based on a model label state and the target pipe twisting angle, and output a prediction result; the forward and reverse wrapping state and the abnormal state are determined based on a system and a pipe belt machine mechanical structure.

[0043] An execution module is configured to trigger an alarm or shutdown of the pipe belt machine when the target pipe twisting angle exceeds a set threshold or the prediction output is an abnormal reverse wrapping state.

[0044] In yet another aspect, the application provides a computer readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by a processor to implement the tubular belt forward and reverse wrapping abnormality monitoring method of the pipe belt machine in the above aspects.

[0045] The technical scheme provided by the embodiments of the present application has at least the following beneficial effects: at least three camera modules are arranged at multiple directions of the pipe belt machine to capture original pipe belt images of the pipe belt in real time, and distortion correction is performed on the original pipe belt images, thereby improving the accuracy of subsequent calculation of the pipe twisting angle; in the pipe twisting angle calculation stage, a target detection and recognition algorithm is used to recognize the pipe belt and the position in the image, a pipe twisting model image under each perspective view is constructed based on the pixel width ratio, and the pipe twisting angle is calculated; in the forward and reverse wrapping state prediction stage, a forward and reverse wrapping CNN prediction model is constructed to pre-classify the forward and reverse wrapping state, and then the pre-classified result is modified based on a fusion correction strategy and the pipe belt, so as to determine the forward and reverse wrapping state under the corresponding perspective view; finally, the running state of the pipe belt in the period is output according to the prediction result under each camera perspective view.

[0046] The scheme does not need to install various sensors and communication modules on the pipe belt machine, and does not need to modify the mechanical structure, and the cost investment is smaller. After the camera module captures the image, the computer device performs image recognition to calculate the pipe twisting angle and state prediction. Compared with the traditional manual patrol method, the monitoring real-time performance is higher, the forward and reverse wrapping state can be predicted in real time, and the detection accuracy is also greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a structure diagram of abnormal monitoring of the pipe belt machine in the running process

[0048] Figure 2 is a flowchart of the pipe belt machine pipe belt forward and reverse wrapping abnormal monitoring method provided by the embodiments of the present application;

[0049] Figure 3 is a schematic diagram before and after distortion correction of the pipe belt image;

[0050] Figure 4 is a schematic diagram of two pipe wrapping types of left wrapping right and right wrapping left of the pipe belt;

[0051] Figure 5 shows the picture of the pipe belt under each camera module perspective view;

[0052] Figure 6 is a schematic diagram of labeling the pixel width of the pipe belt and the pixel width of the pipe belt;

[0053] Figure 7 is a pipe twisting model image constructed under the camera2 perspective view;

[0054] Figure 8 is a structure diagram of the forward and reverse wrapping CNN prediction model;

[0055] Figure 9 is an example image of pre-classification and recognition of the forward and reverse wrapping CNN prediction model;

[0056] Figure 10 The state results predicted by the positive and negative package CNN prediction model under various camera perspectives are shown.

[0057] Figure 11 is an algorithm flowchart of the tubular belt positive and negative package anomaly monitoring method.

[0058] Figure 12 A monitoring interface schematic diagram displayed by a computer device is shown.

[0059] Figure 13 is a structural schematic diagram of the tubular belt positive and negative package anomaly monitoring device of the pipe belt machine. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0061] In this paper, "multiple" refers to two or more. "And / or" describes the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. The character " / " generally represents an "or" relationship between the associated objects before and after it.

[0062] Figure 1 is a structural schematic diagram of the anomaly monitoring during the operation of the pipe belt machine. The pipe belt machine winds the conveying belt into a circular tube shape through the polygonal roller and the support structure. The camera module is symmetrically arranged on the left and right sides of the circular tube, and the camera module is located on the reflector. A light source is installed above the reflector to provide light in the working state, thereby improving the definition and contrast of the camera image. Optionally, the reflector and the camera module can also be arranged in multiple directions such as directly above and / or directly below the pipe belt machine. The symmetrically arranged camera modules and the center of the pipe belt machine cross section are on the same straight line, all the camera modules are located on the same plane, and are parallel to the circular tube cross section. The camera module simultaneously shoots the original tubular belt image, which is processed and displayed by the computer device to show the current state of the pipe belt machine.

[0063] Figure 2 is a flowchart of the tubular belt positive and negative package anomaly monitoring method of the pipe belt machine provided by the embodiments of the present application, which is suitable for the pipe belt machine equipment in Figure 1 The method comprises the following steps:

[0064] S1, obtaining at least three groups of original tubular belt images shot by the camera module in the current period and performing image distortion correction to obtain the tubular belt image.

[0065] At least three groups of camera modules are located at different positions of the tubular belt, and the observation angle can meet the equipment monitoring accuracy. For example, the left and right sides are symmetrically arranged, the upper side is installed, or the left and right sides and the lower side are installed, and the installation can also be at any angle. The specific installation number can be determined according to the camera installation distance and the observation angle range. In theory, the more camera modules are installed, the larger the monitoring angle is.

[0066] At least three groups of camera modules are obtained to ensure detection accuracy. The present scheme takes the installation on the left and right sides and the upper side as an example to illustrate that the monitoring range exceeds 270 degrees (there is a part of the monitoring blind area at the bottom area).

[0067] The computer device periodically acquires the original tubular belt image photographed by all camera modules, and processes the image for distortion correction. Most of the images photographed by industrial cameras are distorted images. Figure 3 is a schematic diagram of the tubular belt image before and after distortion correction. If the distortion is not processed, the pipe belt cannot be accurately identified and the pipe angle cannot be calculated. Therefore, the distortion processing step can also include the following steps:

[0068] a. Identify the target detection object in the original tubular belt image, match the calibrated matching points in the original image with the target detection object, and determine the matching points.

[0069] The original image is a normal image photographed in advance at a normal angle. Some contour edge key points are calibrated as a target for subsequent identification. This step will identify the target detection object in the distorted image through a target detection algorithm, and then determine the matching points in the target detection object. For example, in the present scheme, the target detection object includes the light source (side light pipe) and the tubular belt. Figure 3

[0070] b. Perform radial distortion correction on the original tubular belt image through the openCV calibration function and the matching points to obtain the tubular belt image.

[0071] The openCV calibration function can one-to-one correspond the identified key points, establish an image transformation matrix, and perform matrix transformation to convert the original tubular belt image. The radial distortion curve is converted into a normal curve contour. The corrected image is shown in Figure 3 , and the image frame contour curve is restored to normal display.

[0072] S2, identify the pipe belt in all tubular belt images, and calculate the target pipe angle according to the position of the pipe belt and the position relationship of the camera module.

[0073] ​The pipe wrapping belt is a region displayed in the image by the edge of the pipe wrapping of the conveying belt after being pre-crimped. According to the pipe wrapping type, it can be divided into left wrapping right type and right wrapping left type, that is, the pipe wrapping is crimped clockwise or counterclockwise. The pipe wrapping type needs to be determined according to the installation structure of the mechanical equipment and the actual demand, and the positive wrapping and the reverse wrapping are determined by the system setting, for example, defining the left wrapping right as the positive wrapping, and the right wrapping left as the reverse wrapping, for reference Figure 4 The schematic diagram of the two pipe wrapping types is shown in the

[0074] It should be noted that in the same period, at least one set of original tubular belt images displayed in the pipe wrapping edge image in the camera module shooting, otherwise the positive and reverse wrapping states cannot be judged and the pipe twisting angle cannot be calculated, or the system selects the unknown state. Figure 5 The picture of the tubular belt under the view angle of each camera module is shown, and the boxed labels in camera2 and camera3 are the pipe wrapping belt (the white line in the frame). For the process of image recognition of the pipe wrapping belt and calculation of the target pipe twisting angle, the following steps are included:

[0075] S21, based on the target detection algorithm, identifying the pipe wrapping belt in each tubular belt image, displaying the pixel width X pixel of the tubular belt and the pixel width x pixel of the pipe wrapping belt.

[0076] The mainstream target detection algorithm includes Yolo and SSD. In this embodiment, the target detection model adopts the Yolov5s structure as an example for illustration. This structure is simpler than other Yolov5m, Yolov5l, and Yolov5x structures, and has a fast running speed, which can meet the accuracy and speed requirements of this embodiment.

[0077] In the model construction process, first, the labeling software is used to label the samples and make a sample set; then the learning framework such as pytorch and tensorflow is used to train the model; and then the model format is converted, such as converting the pt format to the onnx format, and converting to the C++ format, and the trained onnx format model can be called by using the opencv dnn module.

[0078] As shown in Figure 6 , the pipe wrapping belt in the image is first identified and labeled by the target detection algorithm. After the labeling is completed, the pixel width X pixel of the tubular belt and the pixel width x pixel of the pipe wrapping belt are calculated according to the image size. Because the pipe wrapping belt is formed by the wrapping and superposition of the belt edge, it will appear as a white strip under the action of the camera and the light, so this scheme can be identified according to the pixels. X pixelx is the pixel width of the tubular belt center to the outer edge (edge line is determined according to the camera module) in the image pixel x is the pixel width of the tubular belt center to the outer edge (edge line is determined according to the camera module) in the image

[0079] S22, according to the number of each camera module, the radial width R of the tubular belt, and the identified position area of the wrapping belt in the image, a twisted pipe model diagram corresponding to the camera module view angle is constructed.

[0080] As shown in Figure 7 , it is a twisted pipe model diagram constructed by camera2 view angle. The radial width R of the tubular belt is the radius of the cross-section circle, and the wrapping belt position reflected on the cross-section circle is the intersection point (cutting point) of the tangent line of camera2 and the radius R, that is, the maximum monitoring view angle range. Then the image X pixel corresponding to the actual observation of the wrapping belt height X, x pixel corresponding to the wrapping belt height x.

[0081] S23, the twisted pipe angle β under the corresponding camera number is calculated based on the twisted pipe angle formula.

[0082] The calculation formula is as follows:

[0083]

[0084] Wherein, x represents the actual observation of the wrapping belt height by the camera module, X represents the actual observation of the tubular belt height; the connecting line of the calibration camera and the tubular belt center is the height reference line, and β represents the complementary angle of the angle α between the connecting line of the center and the wrapping belt position and the height reference line. x pixel / X pixel is the pixel ratio of the image shot by the camera, which is used to calculate the actual wrapping belt height.

[0085] S24, the twisted pipe angles calculated under the view angle of each camera module are averaged to obtain the target twisted pipe angle.

[0086] For example, Figure 7 , by the same reasoning, camera3 can also monitor the wrapping belt position, and the twisted pipe model diagram corresponding to the camera view angle can also be constructed, and the x and β values under the corresponding view angle can also be calculated. The monitoring view angle of camera1 is on the opposite side of the wrapping belt, which is in the visual blind area, and the β value cannot be calculated (the camera module arranged below also cannot be monitored).

[0087] All β values of the wrapping belt monitored and the twisted pipe angle calculated are averaged to obtain the target twisted pipe angle.

[0088] But it needs to be explained that from Figure 7It can be seen that when the distance between the pipe belt and the edge of the tubular belt is greater, the result calculated based on the pixel ratio may have greater error, affecting the calculation result. Therefore, in a possible implementation, S24 can further include the following steps:

[0089] a. Calculate the width of the distance between the pipe belt and the edge of the tubular belt in each tubular belt image, respectively; when the distance width exceeds the monitoring range, the corresponding pipe bending angle is filtered;

[0090] This step is mainly to set the monitoring range. For those cases where the pipe belt is close to the edge of the tubular belt, the calculation result will introduce too large error, and at this time it should be filtered. The pipe belt position monitored by the threshold adjacent camera module must satisfy the distance monitoring range.

[0091] b. The pipe bending angle satisfying the distance width is calculated by averaging to obtain the target pipe bending angle.

[0092] S3, input the tubular belt image into the forward and reverse wrapping neural network CNN prediction model, predict the forward and reverse wrapping state of the current tubular belt based on the model label state and the target pipe bending angle, and output the prediction result.

[0093] The judgment of the pipe state forward and reverse wrapping is mainly based on the CNN (Convolutional Neural Network) classification model. The forward and reverse wrapping CNN structure designed in this embodiment is as shown in Figure 8 The model complexity can be increased or reduced according to the test effect. Four convolution layers, four pooling layers, three fully connected layers and a softmax layer forward and reverse wrapping are adopted. The input of the CNN prediction model is the region intercepted from the pipe belt position identified by the target detection model, which is converted into a 128x128 size RGB image. The convolution kernel adopts a small convolution kernel with a size of 3x3 and a step of 1x1; the pooling layer adopts maximum pooling with a size of 2x2 and a step of 2x2. Specifically, the four convolution layers are Cov3-32, Cov3-64, Cov3-128 and Cov3-256 in turn. The three fully connected layers are FC-1014, FC-1014 and FC-512 in turn, and finally output through the softmax layer.

[0094] The forward and reverse wrapping state and the abnormal state of the tubular belt are determined based on the system and the pipe belt machine mechanical structure, which can be specifically analyzed in actual operation. The prediction process of the forward and reverse wrapping CNN prediction model specifically includes the following steps:

[0095] S31, input the tubular belt image after pre-processing into the forward and reverse wrapping CNN prediction model, and pre-classify based on the pipe belt position and the model classification label.

[0096] The model classification labels include a positive wrapping label, a negative wrapping label, and an unknown state label. In one possible implementation, the positive wrapping label is set as 1, the negative wrapping label is set as 2, and the unknown state label is set as 0.

[0097] The image needs to be positioned in the position area of the tubular belt in the image first, and then the image is cropped to an RGB image with a size of 128x128, without left-right image flipping. Because the image flipping process can cause the left-right picture to be flipped, the position of the wrapping belt is changed, and thus a judgment error is caused. For the same reason, the left-right flipping process cannot be performed on the image in the positive-negative wrapping CNN prediction model training process.

[0098] Taking the right wrapping left label positive wrapping as an example, the image identified by the pre-classification is shown in FIG. 6. Figure 9 In the left image, the wrapping belt is located near the center of the tubular belt, and the prediction result is 0 (unknown state, and the pixel-level identification almost does not occur); in the middle image, the wrapping belt is located near the right side of the tubular belt, and the prediction result is 1 (positive wrapping); and in the right image, the wrapping belt is located near the left side of the tubular belt, and the prediction result is 2 (negative wrapping). Because the positive-negative wrapping CNN prediction is based on pixel-level identification, when the wrapping belt is near the middle part of the tubular belt (camera perspective), only the white line is displayed in the image, and the left-right side thickness difference cannot be identified, so the positive-negative wrapping cannot be predicted, and even if the positive-negative wrapping is identified, the identification accuracy cannot be guaranteed. In fact, the prediction of the positive-negative wrapping is mainly based on the pixel difference when the center part is deviated, and the side observation can see the edges of the stacked belts, and there is an obvious pixel difference (see FIG. 7). Figure 4

[0099] In S32, the state is corrected based on the identified wrapping belt position, the pre-classification result, and the fusion correction strategy, and the corrected state is determined as the positive-negative wrapping state of the tubular belt under the current perspective.

[0100] Based on the foregoing, to avoid the misjudgment of the positive-negative wrapping CNN prediction model, the embodiment further provides a fusion correction strategy to balance the error caused by the image prediction. All possible misjudgment situations are set in the fusion correction strategy, and specifically include a left image misjudgment classification result and a right image misjudgment classification result; when the prediction classification result of the wrapping belt position and the fusion correction strategy do not match, it is indicated that the model prediction is wrong, and the state is corrected.

[0101] When the right wrapping left is set as the positive wrapping state, and the left wrapping right is set as the negative wrapping state, the fusion correction strategy includes:

[0102] When the wrapping belt is located on the left side of the image, and the pre-classification result is the positive wrapping, it is determined that the prediction is misjudged; when the wrapping belt is located on the right side of the image, and the pre-classification result is the negative wrapping, it is determined that the prediction is misjudged.

[0103] ​Both of the two cases will be corrected to unknown state according to the fusion correction strategy. Figure 10 The state results predicted by the positive and negative wrapping CNN prediction model under various camera perspectives are shown, including Figures A-F. Table 1 lists the correction according to the fusion strategy.

[0104] Table 1 Fusion strategy correction table

[0105]

[0106] Based on the above settings, it can be determined that Figure 10 -B and Figure 10 -F belong to the misjudgment case (in B, the wrapping belt is located on the left side but the recognition result is positive wrapping, and in F, the wrapping belt is located on the left side but the recognition result is negative wrapping), and A and D belong to the position state that cannot be identified. Therefore, according to the fusion strategy, B and F are corrected to the unknown state. The remaining prediction results match the fusion strategy and are not corrected. In fact Figure 10 -B and Figure 10 -F are corrected to the unknown state. The remaining prediction results match the fusion strategy and are not corrected. In fact Figure 10 the predictions of “unknown” state in the above-mentioned cases all belong to the case that the wrapping belt is close to the edge of the pipe state and cannot be accurately judged in step S23.

[0107] In another possible implementation, it can also be set that right wrapping left is negative wrapping state, and left wrapping right is positive wrapping state, and the fusion correction strategy includes:

[0108] When the wrapping belt is located on the left side of the image and the pre-classification result is negative wrapping, it is determined as a misjudgment prediction; when the wrapping belt is located on the right side of the image and the pre-classification result is positive wrapping, it is determined as a misjudgment prediction, and according to the fusion correction strategy, the misjudgment prediction is corrected to the unknown state.

[0109] The judgment of this case can be analogized according to the prediction result and the fusion correction strategy, and this embodiment will not be described in detail.

[0110] Based on the above-mentioned prediction method, the computer device will count all the positive and negative wrapping states under the camera module perspective in the current period, classify and count according to the state type, and determine the positive and negative wrapping state of the tubular belt in the period according to the type statistical quantity and the monitoring angle of the camera module. Taking three camera modules Figure 7 ) as an example, when at least one position state (camera1) appears in the measurement result, and the other two (camera2 and camera3) are positive wrapping, it is determined that the output is positive wrapping; when camera2 and camera3 are negative wrapping, it is determined that the output is negative wrapping.

[0111] In an embodiment, when the prediction results of the three camera modules are all position states, it indicates that the pipe belt is located in the center of the camera view (almost no pixel-level recognition occurs), and the remaining cameras are in the view blind area and cannot determine the current state. At this time, the computer reads the forward and reverse pipe states of the previous period, and outputs the state of the previous period as the result state. In the case of high camera module acquisition frequency, the next detection period can accurately identify the forward and reverse pipe states, and does not affect the maintenance of the equipment.

[0112] S4, when the target pipe angle exceeds the set threshold or the prediction output is a reverse pipe abnormal state, triggering the pipe belt machine alarm or shutdown.

[0113] As shown in Figure 11 , it is an algorithm flowchart of the pipe belt forward and reverse wrapping abnormality monitoring method. The input raw pipe belt image is first corrected for image distortion, and then the pipe belt wrapping position is detected. When the pipe belt wrapping position is detected (otherwise, it is directly ended), the pipe angle is calculated, and the state prediction is performed through the pipe belt wrapping position to determine the pipe belt forward and reverse wrapping state output.

[0114] Figure 12 A monitoring interface diagram displayed by a computer device is shown, which displays the corrected pipe belt images obtained by each camera module, the calculated target pipe angle, the forward and reverse wrapping state, the running state, and the simulated forward and reverse wrapping state diagram.

[0115] Optionally, the system also sets a monitoring threshold for the target pipe angle, and when the calculated target pipe angle exceeds the set threshold or when the pipe belt machine is detected to be in a reverse wrapping state, the pipe belt machine alarm or shutdown is triggered.

[0116] In summary, the present scheme sets at least three camera modules at multiple positions of the pipe belt machine to capture raw pipe belt images of the pipe belt in real time, and corrects the distortion to improve the accuracy of subsequent pipe angle calculation. In the pipe angle calculation stage, the target detection recognition algorithm is used to identify the pipe belt and its position in the image, and a pipe model diagram under each perspective view is constructed based on the pixel width ratio, so as to calculate the pipe angle. In the forward and reverse wrapping state prediction stage, the forward and reverse wrapping CNN prediction model is used to pre-classify the forward and reverse wrapping state, and then the pre-classification result is modified based on the fusion correction strategy and the pipe belt to determine the forward and reverse wrapping state under the corresponding perspective view. Finally, the running state of the pipe belt in the period is output according to the prediction results under each camera perspective view.

[0117] The scheme does not need to install various sensors and communication modules on the pipe belt machine, and does not need to modify the mechanical structure, and the cost investment is smaller. After the camera module shoots the image, the computer device performs image recognition to calculate the pipe twisting angle and state prediction. Compared with the traditional manual patrol method, the monitoring real-time is higher, the positive and negative packaging state can be predicted in real time, and the detection accuracy is also greatly improved.

[0118] As shown in Figure 13 The embodiment of the application also provides a tubular belt positive and negative packaging abnormality monitoring device of a pipe belt machine. The device is used for the tubular belt positive and negative packaging abnormality monitoring method of the pipe belt machine, and the device comprises:

[0119] An image acquisition module is configured to acquire original tubular belt images shot by at least three camera modules in a current period and perform image distortion correction to obtain tubular belt images.

[0120] A pipe twisting angle calculation module is configured to identify a pipe wrapping belt in all the tubular belt images, and calculate a target pipe twisting angle according to the position of the pipe wrapping belt and the position of the camera module. The original tubular belt image shot by at least one camera module in the same period displays a tubular belt wrapping edge image.

[0121] A state prediction module is configured to input the tubular belt image into a positive and negative packaging neural network CNN prediction model, predict the positive and negative packaging state of the current tubular belt based on the model label state and the target pipe twisting angle, and output a prediction result. The positive and negative packaging state and the abnormal state of the tubular belt are determined based on the system and the mechanical structure of the pipe belt machine.

[0122] An execution module is configured to trigger an alarm or shutdown of the pipe belt machine when the target pipe twisting angle exceeds a set threshold or the prediction output is an abnormal negative packaging state.

[0123] The embodiment of the application also provides a computer readable storage medium. The computer readable storage medium stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the tubular belt positive and negative packaging abnormality monitoring method of the pipe belt machine as described in the above aspects.

[0124] The preferred embodiments of the present application have been described above; it needs to be understood that the present application is not limited to the above specific embodiments, wherein the devices and structures not described in detail should be understood as being implemented in the ordinary way in the art; any person skilled in the art, without departing from the technical solutions of the present application, can make many possible changes and modifications, or modify equivalent embodiments of equivalent changes, which do not affect the essential content of the present application; therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application, without departing from the technical solutions of the present application, still belongs to the protection scope of the technical solutions of the present application.

Claims

1. A method for monitoring anomalies in the forward and reverse wrapping of tubular tape in a tubular tape conveyor, characterized in that, The method includes: S1, acquire raw tubular zone images captured by at least three camera modules within the current period and perform image distortion correction to obtain tubular zone images; S2, identify the tubular bands in all the tubular band images, and calculate the target torsion angle based on the position of the tubular bands and the positional relationship between the camera module and the tubular bands; within the same period, at least one set of original tubular band images captured by the camera module displays the tubular band edge image; specifically, S21, identify the tubular bands and the displayed tubular band pixel width in each of the tubular band images based on the target detection algorithm. and the width of the protection band pixels The tubing is formed by overlapping and wrapping strips of edge. The width in pixels is the distance from the center of the tubular band to its outer edge in the image. The pixel width from the center of the tubular band to the encapsulation band in the image; S22, based on the number of each camera module, the radial width R of the tubular strip, and the identified location area of ​​the tubular strip in the image, construct the torsion tube model diagram corresponding to the viewpoint of the camera module. S23, Calculate the torsion tube angle for the corresponding camera number based on the torsion tube angle formula. The formula is as follows: in, This indicates the actual height of the tubular strip observed by the camera module; X represents the actual observed height of the tubular strip. The line connecting the center of the calibration camera and the center of the tubular strip is the height reference line. It represents the complementary angle between the line connecting the center of the circle to the position of the tubing and the height reference line; S24, calculate the average of the torsion tube angles calculated from the perspectives of each camera module to obtain the target torsion tube angle; S3, input the tubular strip image into the forward and reverse wrapping neural network (CNN) prediction model, predict the current forward and reverse wrapping state of the tubular strip based on the model label state and the target twist angle, and output the prediction result; the forward and reverse wrapping state and abnormal state of the tubular strip are determined based on the system and the mechanical structure of the tubular strip machine; S4. When the target twisting angle exceeds the set threshold, or the predicted output is an abnormal reverse wrapping state, trigger the tube conveyor alarm or stop the machine.

2. The method for monitoring abnormalities in the forward and reverse wrapping of tubular belts in a tubular conveyor according to claim 1, characterized in that, S1 includes: Identify the target detection object in the original tubular strip image, match the marked matching points in the original tubular strip image with the target detection object, and determine the matching point; the target detection object includes at least the tubular strip and the light source device above the left and right camera modules; The original tubular band image is corrected for radial distortion using OpenCV calibration functions and matching points to obtain the tubular band image.

3. The method for monitoring abnormalities in the forward and reverse wrapping of tubular belts in a tubular conveyor according to claim 1, characterized in that, S24 includes: Calculate the width of the tubular band from the edge of the tubular band in each of the tubular band images; when the width exceeds the monitoring range, filter out the corresponding torsion angle; wherein, if the tubular band is not identified in the tubular band image, the torsion angle calculation is not performed; The target twist angle is obtained by averaging the twist angles that meet the distance width requirements.

4. The method for monitoring abnormalities in the forward and reverse wrapping of tubular belts in a tubular belt conveyor according to claim 1, characterized in that, S3 include: S31, the preprocessed image of the tubular band is input into the positive and negative envelope neural network (CNN) prediction model, and pre-classification is performed based on the position of the tubular band and the model classification label; the model classification label includes the positive envelope label of the tubular band, the negative envelope label of the tubular band, and the unknown state label; S32, based on the identified tubular band position, pre-classification results and fusion correction strategy, the state is corrected, and the corrected state is determined as the positive and negative wrapping state of the tubular band under the current view.

5. The method for monitoring abnormalities in the forward and reverse wrapping of tubular belts in a tubular conveyor according to claim 4, characterized in that, The fusion correction strategy includes misclassification results for the left and right sides of the image. When the predicted classification result of the envelope position does not match the fusion correction strategy, the model is indicated to have made a prediction error, and state correction is performed.

6. The method for monitoring abnormalities in the forward and reverse wrapping of tubular belts in a tubular conveyor according to claim 1, characterized in that, When the left-to-right envelope is defined as the positive envelope state and the right-to-left envelope is defined as the negative envelope state, the fusion correction strategy includes: If the envelope is located on the left side of the image and the pre-classification result is a positive envelope, it is determined to be a misclassified prediction; if the envelope is located on the right side of the image and the pre-classification result is a negative envelope, it is determined to be a misclassified prediction, and the misclassified prediction is corrected to an unknown state according to the fusion correction strategy. or, When the left-to-right envelope is defined as the inverted envelope state and the right-to-left envelope is defined as the positive envelope state, the fusion correction strategy includes: If the envelope is located on the left side of the image and the pre-classification result is a negative envelope, it is determined to be a misclassified prediction; if the envelope is located on the right side of the image and the pre-classification result is a positive envelope, it is determined to be a misclassified prediction, and the misclassified prediction is corrected to an unknown state according to the fusion correction strategy.

7. The method for monitoring abnormalities in the forward and reverse wrapping of tubular belts in a tubular conveyor according to claim 4, characterized in that, The tubular band image preprocessing includes: The location of the tubular strip in the image will be cropped into... An RGB image of the specified size, without horizontal image flipping; The proposed neural network (CNN) prediction model includes at least four convolutional layers and four corresponding pooling layers, three fully connected layers, and one softmax layer. All convolutional layers are... Convolution kernel, stride is The pooling layer is based on max pooling, and the kernel size is [missing value]. Step size is .

8. A device for monitoring abnormalities in the forward and reverse wrapping of tubular tape in a tubular tape conveyor, characterized in that, The device is used in the method for monitoring abnormalities in the forward and reverse wrapping of tubular tape in any one of claims 1-7, and the device comprises: The image acquisition module is used to acquire raw tubular zone images captured by at least three camera modules within the current period and perform image distortion correction to obtain tubular zone images; The torsion angle calculation module is used to identify the tubular band in all the tubular band images and calculate the target torsion angle based on the position of the tubular band and the positional relationship between the position of the tubular band and the camera module; within the same period, at least one set of the original tubular band images captured by the camera module shows the tubular band edge image; The state prediction module is used to input the tubular strip image into the forward and reverse wrapping neural network (CNN) prediction model, predict the current forward and reverse wrapping state of the tubular strip based on the model label state and the target twist angle, and output the prediction result; the forward and reverse wrapping state and abnormal state of the tubular strip are determined based on the system and the mechanical structure of the tubular strip machine. The execution module is used to trigger an alarm or shutdown of the tube conveyor when the target torsion angle exceeds a set threshold or when the predicted output is an abnormal reverse wrapping state.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the tubular tape forward and reverse wrapping anomaly monitoring method of the tubular tape machine as described in any one of claims 1 to 7.

Images