A conveying monitoring method and system for a box folding machine

Abstract

The present application provides a conveying monitoring method and system for a box folding machine, which collects a depth video stream of the conveyor belt during the conveying process; fits the displacement paths of various locations on the conveyor belt based on the optical flow information in the depth video stream, and divides all displacement paths into nonlinear trajectories in the roller contact area and linear trajectories in the non-roller contact area; determines the wear of the rollers in the conveyor belt based on the curvature radius of each nonlinear trajectory and the inclination angle of the rollers; determines the sag deviation of the midpoint of the span between each two linear trajectories, and determines the tension fluctuation index of the conveyor belt during the paper box conveying process based on all the sag deviations; combines the wear and tension fluctuation index to determine the probability of the conveyor belt running off during the paper box conveying process, and sends an alarm signal to the monitoring center when the probability of running off is greater than a preset running off threshold. The solution of the present application can provide early warning of conveyor belt deviation abnormalities based on the aging of the conveying equipment.

Description

Technical Field

[0001] The present application relates to the field of industrial production technology, and more specifically, to a conveying monitoring method and system for a box folding machine. Background Art

[0002] Industrial production conveying refers to the efficient movement of raw materials, semi-finished products, and finished goods through the use of conveying equipment across various manufacturing, processing, logistics, and mining industries. Conveyor systems are a crucial component of modern industrial automation and are widely used in coal mining, power generation, steel, chemicals, food processing, pharmaceuticals, and port logistics. The conveyor system of a box folding machine is primarily used to move folded cartons from one stage of the production line to another.

[0003] The conveying system of a box folding machine is usually composed of a conveyor belt. Common problems of the conveyor belt during the conveying process include deviation and tearing. Among them, tearing is mainly caused by scratches from hard objects and serious deviation. Therefore, during the conveying process of the box folding machine, it is extremely important to monitor the deviation of the conveyor belt. In the existing technology, the deviation of the conveyor belt is usually monitored by a deviation sensor. When the conveyor belt deviates, the conveyor belt drives the deviation sensor to operate, thereby alarming. However, this deviation monitoring method can only alarm when the conveyor belt deviates, and since the deviation sensor is usually installed on both sides of the conveyor belt, if the conveyor belt structure is deformed due to aging of the equipment, the abnormality cannot be detected. Therefore, how to provide early warning of conveyor belt deviation abnormalities according to the aging of the conveying equipment has become a difficult problem faced by the industry. Summary of the Invention

[0004] The present application provides a conveying monitoring method and system for a box folding machine, which can provide early warning of abnormal conveyor belt deviation according to the aging condition of the conveying equipment.

[0005] In a first aspect, the present application provides a conveying monitoring method for a box folding machine, comprising:

[0006] Two visual sensors are used to capture two videos from different perspectives during the carton conveying process;

[0007] Performing stereo matching on the two videos to obtain a depth video stream of the conveyor belt;

[0008] Extracting multiple optical flow information from the depth video stream, fitting the displacement paths of various locations on the conveyor belt based on each optical flow information, and dividing all displacement paths into nonlinear trajectories in roller contact areas and linear trajectories in non-roller contact areas;

[0009] Determining the wear amount of the rollers in the conveyor belt according to the curvature radius of each nonlinear trajectory and the forward inclination angle of the rollers;

[0010] Determining the sag deviation of the midpoint of the span between each two linear tracks, and determining the tension fluctuation index of the conveyor belt during the carton conveying process based on all the sag deviations;

[0011] The deviation probability of the conveyor belt during the paper box conveying process is determined in combination with the wear amount and the tension fluctuation index, and an alarm signal is sent to a monitoring center when the deviation probability is greater than a preset deviation threshold.

[0012] In some embodiments, performing stereo matching on the two videos to obtain a depth video stream of the conveyor belt specifically includes:

[0013] Performing frame synchronization on the two channels of video to obtain an image group for each timestamp;

[0014] Extracting regions of interest from all images in each image group to obtain a conveyor belt image group of the conveyor belt area;

[0015] Perform stereo matching on the conveyor belt image group at each time stamp to obtain the depth map of the conveyor belt at each time stamp;

[0016] Reconstruct all depth maps into a conveyor belt depth video stream.

[0017] In some embodiments, fitting the displacement path of each location on the conveyor belt according to each optical flow information specifically includes:

[0018] Dividing a plurality of monitoring areas on the conveyor belt according to all depth maps in the depth video stream;

[0019] All optical flow information is filtered according to all monitoring areas to obtain the path optical flow set in each monitoring area;

[0020] Each path optical flow set is fitted separately to obtain the displacement path of each point on the conveyor belt.

[0021] In some embodiments, determining the wear amount of the rollers in the conveyor belt according to the curvature radius of each nonlinear trajectory and the inclination angle of the rollers specifically includes:

[0022] determining a plurality of curvature radii for each nonlinear trajectory;

[0023] According to the changing characteristics of all curvature radii on the conveyor belt surface, multiple local deformation features of the conveyor belt surface are identified;

[0024] Fit the elasticity of the conveyor belt according to all local deformation characteristics to obtain the elastic coefficient of the conveyor belt;

[0025] The wear amount of the roller in the conveyor belt is determined according to the elastic coefficient and the forward tilt angle of the roller.

[0026] In some embodiments, determining the sag deviation of the mid-span point between each two linear tracks specifically includes:

[0027] Determine the midpoint of the span between each two linear traces;

[0028] Determine the sag from the midpoint of each span to each linear track on either side of the midpoint of each span;

[0029] The sag deviation amount of the mid-span point between each two linear tracks is determined based on all sags at each mid-span point.

[0030] In some embodiments, determining the tension fluctuation index of the conveyor belt during the carton conveying process using all sag deviations specifically includes:

[0031] Determine a correlation sequence of sag deviation changes based on all sag deviations;

[0032] determining a cutoff point of the correlation sequence;

[0033] The tension fluctuation index of the conveyor belt during the paper box conveying process is determined according to the cutoff point.

[0034] In some embodiments, determining the probability of the conveyor belt running off during the carton conveying process by combining the wear amount and the tension fluctuation index specifically includes:

[0035] determining a deviation factor of the conveyor belt during the paper box conveying process according to the wear amount and the tension fluctuation index;

[0036] The probability of the conveyor belt running off course during the carton conveying process is determined according to the offset factor.

[0037] In a second aspect, the present application provides a conveying monitoring system for a box folding machine, comprising:

[0038] The acquisition module is used to collect two videos from different perspectives during the carton conveying process through two visual sensors;

[0039] A processing module, configured to perform stereo matching on the two videos to obtain a depth video stream of the conveyor belt;

[0040] The processing module is further configured to extract a plurality of optical flow information from the depth video stream, and for each optical flow information, fit the displacement path of each location on the conveyor belt according to the optical flow information, and divide all displacement paths into nonlinear trajectories in roller contact areas and linear trajectories in non-roller contact areas;

[0041] The processing module is further configured to determine the wear amount of the rollers in the conveyor belt according to the curvature radius of each nonlinear track and the inclination angle of the rollers;

[0042] The processing module is further configured to determine a sag deviation at a midpoint of a span between every two linear tracks, and determine a tension fluctuation index of the conveyor belt during the carton conveying process based on all sag deviations;

[0043] An execution module is used to determine the deviation probability of the conveyor belt during the paper box conveying process based on the wear amount and the tension fluctuation index, and send an alarm signal to a monitoring center when the deviation probability is greater than a preset deviation threshold.

[0044] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores codes, and the processor is configured to obtain the codes and execute the above-mentioned conveying monitoring method for a box folding machine.

[0045] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the above-mentioned conveying monitoring method for a box folding machine.

[0046] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0047] In the conveying monitoring method and system for a folding machine provided in the present application, first, two videos of different perspectives are collected by two visual sensors during the conveying of a carton; the two videos are stereo matched to obtain a depth video stream of the conveyor belt; multiple optical flow information in the depth video stream is extracted, and for each optical flow information, the displacement paths of various locations on the conveyor belt are fitted according to each optical flow information, and all displacement paths are divided into nonlinear trajectories in the roller contact area and linear trajectories in the non-roller contact area; the wear amount of the rollers in the conveyor belt is determined according to the curvature radius of each nonlinear trajectory and the inclination angle of the roller; the sag deviation amount of the midpoint of the span between each two linear trajectories is determined, and the tension fluctuation index of the conveyor belt during the conveying of the carton is determined by all the sag deviation amounts; the deviation probability of the conveyor belt during the conveying of the carton is determined in combination with the wear amount and the tension fluctuation index, and an alarm signal is sent to a monitoring center when the deviation probability is greater than a preset deviation threshold.

[0048] It can be seen that the present application targets the uneven surface of the conveyor belt and identifies the displacement paths of various locations on the conveyor belt through the depth map of the conveyor belt. It can more accurately extract the motion information of various locations on the conveyor belt, and divide the geometric shape of the displacement path into linear trajectories and nonlinear trajectories according to whether a large change occurs. Subsequently, the aging of the rollers (i.e., the amount of wear on the rollers) is measured through all nonlinear trajectories, and the aging of the conveyor belt (i.e., the tension fluctuation index of the conveyor belt) is measured based on all linear trajectories. The probability of the conveyor belt running off is then evaluated through the aging of the rollers and the aging of the conveyor belt. In summary, the present application can provide early warning of abnormal conveyor belt deviation based on the aging of the conveying equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is an exemplary flow chart of a conveying monitoring method for a box folding machine according to some embodiments of the present application;

[0050] Figure 2 is a schematic diagram of the principle of dividing the displacement path according to some embodiments of the present application;

[0051] Figure 3 is an exemplary flow chart for determining the amount of wear according to some embodiments of the present application;

[0052] Figure 4 is a structural diagram of a conveying monitoring system for a box folding machine according to some embodiments of the present application;

[0053] Figure 5 It is a structural diagram of a computer device for implementing a conveying monitoring method for a box folding machine according to some embodiments of the present application. DETAILED DESCRIPTION

[0054] In order to better understand the technical solution of the present application, the technical solution of the present application will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0055] refer to Figure 1 , which is an exemplary flow chart of a method for monitoring the conveyance of a box folding machine according to some embodiments of the present application. The method 100 for monitoring the conveyance of a box folding machine mainly includes the following steps:

[0056] In step 101, two visual sensors are used to collect two videos from different perspectives during the carton conveying process.

[0057] In specific implementation, the following method can be used to collect two videos of different perspectives during the carton conveying process through two visual sensors, namely: a visual sensor is installed on the left and right sides of the same height above the conveyor belt, and the video of the conveyor belt during the carton conveying process is collected by the two visual sensors. The videos collected by the two visual sensors are used as two videos of different perspectives during the carton conveying process. The frame rate of the video collected by the visual sensor can be preset according to demand. For example, the frame rate is set to 30FPS in this application.

[0058] In step 102, stereo matching is performed on the two videos to obtain a depth video stream of the conveyor belt.

[0059] In some embodiments, stereo matching of the two videos to obtain a depth video stream of the conveyor belt can be achieved by the following steps, namely:

[0060] Performing frame synchronization on the two channels of video to obtain an image group for each timestamp;

[0061] Extracting regions of interest from all images in each image group to obtain a conveyor belt image group of the conveyor belt area;

[0062] Perform stereo matching on the conveyor belt image group at each time stamp to obtain the depth map of the conveyor belt at each time stamp;

[0063] Reconstruct all depth maps into a conveyor belt depth video stream.

[0064] In specific implementation, the frame synchronization of the two videos can be performed to obtain the image group of each frame in the following manner: first, the timestamp of each frame image in the two videos is obtained, and then the images with the closest timestamps in the two videos are divided into the same set, and all the obtained sets are used as the image groups of each timestamp.

[0065] It should be noted that the image group in this application is a collection of images collected by two visual sensors at the same time.

[0066] In a specific implementation, regions of interest are extracted from all images in each image group to obtain a conveyor belt image group of the conveyor belt area. This can be achieved by: selecting an image group as a selected image group; first, performing edge detection on two images in the selected image group using the Canny algorithm in the prior art to obtain edge images of the two images; then, extracting all straight lines in each edge image using a Hough transform; then, extracting all straight lines in each edge image whose length is greater than a preset length threshold; selecting all two mutually parallel straight lines from the extracted straight lines; and calculating the span between each pair of mutually parallel straight lines; finally, taking the area between the two straight lines with the largest span in each image as the conveyor belt image of each image; and further determining the conveyor belt image groups of the remaining selected image groups. The length threshold can be preset based on the resolution of the visual sensor. For example, if the resolution of the visual sensor in this application is 1280x1024, then the length threshold is preset to half the width of the resolution, i.e., half of 1280, i.e., 640 pixels.

[0067] It should be noted that the conveyor belt image group in this application is a group of images with the background removed, which only includes the conveyor belt area.

[0068] In specific implementation, stereo matching is performed on the conveyor belt image group at each timestamp to obtain the depth map of the conveyor belt at each timestamp. This can be achieved in the following way: for the conveyor belt image group at each timestamp, stereo matching can be performed on the two conveyor belt images in each conveyor belt image group using the AD-Census stereo matching algorithm in the prior art, and all the results of the stereo matching are used as the depth map of each timestamp.

[0069] It should be noted that the depth map in this application is an image used to describe the concavity and convexity of the conveyor belt surface.

[0070] In a specific implementation, reconstructing all the depth maps into the depth video stream of the conveyor belt can be achieved in the following way, namely: arranging all the depth maps in the order of the timestamps of each depth map, using the arranged images as each frame image in the video, and then using the obtained video as the depth video stream of the conveyor belt.

[0071] It should be noted that the depth video stream in this application is a video composed of continuous depth maps, which is used to describe the concavity and convexity of the conveyor belt surface.

[0072] In step 103, multiple optical flow information is extracted from the depth video stream. For each optical flow information, the displacement path of each location on the conveyor belt is fitted according to each optical flow information, and all displacement paths are divided into nonlinear trajectories in the roller contact area and linear trajectories in the non-roller contact area.

[0073] In some embodiments, extracting multiple optical flow information from the depth video stream refers to performing optical flow extraction on the depth video stream through an optical flow method to obtain multiple optical flow fields, and using all the extracted optical flow fields as optical flow information.

[0074] In a specific implementation, the optical flow method is used to extract the optical flow of the depth video stream, and the extracted results are used as the optical flow information. This can be achieved in the following way, namely: the optical flow can be extracted from the depth video stream through the OpenCV sub-function CalcOpticalFlowFarneback.

[0075] It should be noted that the optical flow information in this application is a set of vectors that describe the motion trajectory of pixels in the depth video stream.

[0076] In some embodiments, fitting the displacement path of each location on the conveyor belt according to each optical flow information can be achieved by using the following steps, namely:

[0077] Dividing a plurality of monitoring areas on the conveyor belt according to all depth maps in the depth video stream;

[0078] All optical flow information is filtered according to all monitoring areas to obtain the path optical flow set in each monitoring area;

[0079] Each path optical flow set is fitted separately to obtain the displacement path of each point on the conveyor belt.

[0080] In specific implementation, dividing the conveyor belt into multiple monitoring areas according to all depth maps in the depth video stream can be achieved in the following manner, namely: selecting a depth map in the depth video stream as the selected depth map, first, determining the difference in depth value between each pixel point in the selected depth map and each pixel point in the surrounding 3x3 range, then connecting all pixels whose difference values ​​are less than a preset depth threshold, and then using all areas formed by the connected pixels as texture areas, and continuing to determine multiple texture areas of the remaining depth maps in the depth video stream, and finally, superimposing the texture areas of all depth maps in the same image, and using each superimposed area as the monitoring area on the conveyor belt, wherein the depth threshold is a value preset according to the texture depth of the conveyor belt surface. For example, in this application, the depth threshold is preset to 0.8 times the texture depth of the conveyor belt surface.

[0081] It should be noted that the monitoring area in this application refers to the area on the conveyor belt surface selected for path detection, and the monitoring area is an area on the conveyor belt surface with obvious texture features.

[0082] In specific implementation, all optical flow information is filtered according to all monitoring areas, and the path optical flow set in each monitoring area can be obtained in the following way, namely: all pixels in the monitoring area are filtered out, a monitoring area is selected as the selected monitoring area, and the optical flow vectors corresponding to the pixels in the selected monitoring area in all optical flow information are extracted. Then, the set of all extracted optical flow vectors is used as the path optical flow set in the selected monitoring area, and the path optical flow sets in the remaining monitoring areas are continued to be determined.

[0083] It should be noted that the path optical flow set in this application refers to the set of all optical flow vectors in the monitoring area.

[0084] In the specific implementation, each path optical flow set is fitted separately, and then the displacement path of each point on the conveyor belt can be obtained in the following way, namely: first, the optical flow accumulation calculation is performed on all the optical flow vectors in the path optical flow set, that is, the optical flow vectors at each pixel point in the path optical flow set are accumulated frame by frame. For example, any pixel point a is selected, and the optical flow vector of the pixel point a in the first frame is obtained. The optical flow vector of the first frame is added to the coordinates of the pixel point a, and the obtained vector is used as the position of the pixel point b in the next frame. Subsequently, the optical flow vector of the pixel point b in the second frame is continued to be obtained, and the optical flow vector of the pixel point b in the second frame is added. The vector is added to the coordinates of the pixel point b, and the obtained vector is used as the position of the pixel point c in the next frame, and so on, until the boundary of the depth video stream is reached. Finally, for each monitoring area, the coordinates of all pixel points obtained by the optical flow accumulation calculation of the path optical flow set of each monitoring area are arranged from small to large according to the size of the vertical coordinate, and the coordinates of all arranged pixel points are fitted into curves respectively, and all the obtained curves are used as the displacement path of each monitoring area. During fitting, the vertical coordinate of the pixel point is used as the independent variable and the horizontal coordinate is used as the dependent variable, and fitting is performed by the least squares algorithm in the prior art.

[0085] It should be noted that the displacement path in this application is a curve that describes the displacement conditions at various locations on the conveyor belt during the carton conveying process.

[0086] In some embodiments, dividing all displacement paths into nonlinear trajectories in roller contact areas and linear trajectories in non-roller contact areas can be achieved by:

[0087] Determine the geometric characteristics of each displacement path;

[0088] According to all geometric features, all displacement paths are divided into nonlinear trajectories in the roller contact area and linear trajectories in the non-roller contact area.

[0089] In specific implementation, the geometric characteristics of each displacement path can be determined in the following manner, namely: first, select a displacement path as the selected displacement path, calculate the first-order derivative curve and the second-order derivative curve of the selected displacement path, then multiply the first-order derivative curve by the second-order derivative curve, take the absolute value of the obtained curve and divide it by the cube of the selected displacement path, then perform a definite integral operation on the obtained curve, where the lower limit of the integral is 0 and the upper limit of the integral is the height of the depth map in this application, finally, use the result of the definite integral operation as the geometric characteristic of the selected displacement path, and continue to determine the geometric characteristics of the remaining displacement paths.

[0090] It should be noted that the geometric feature in this application is a parameter value that describes the geometric shape of the displacement path. The larger the geometric feature, the greater the difference between the geometric shape of the displacement path and the straight line. The smaller the geometric feature, the less the difference between the geometric shape of the displacement path and the straight line.

[0091] In specific implementation, all displacement paths are divided into nonlinear trajectories of the roller contact area and linear trajectories of the non-roller contact area according to all geometric features. This can be achieved in the following manner, namely: all geometric features are compared with a preset geometric threshold, and all displacement paths with geometric features greater than the geometric threshold are regarded as nonlinear trajectories of the roller contact area, and all displacement paths with geometric features less than or equal to the geometric threshold are regarded as linear trajectories of the non-roller contact area, wherein the geometric threshold is a value preset according to actual needs. For example, in this application, the geometric threshold is preset to 0.2.

[0092] It should be noted that in this application, the roller contact area refers to the area below the conveyor belt that is in direct contact with the roller, and the non-roller contact area refers to the area below the conveyor belt that is not in direct contact with the roller.

[0093] In addition, it should be noted that in the present application, a nonlinear trajectory refers to a displacement path whose geometric shape changes dynamically, and a linear trajectory refers to a displacement path whose geometric shape does not change.

[0094] In some embodiments, reference Figure 2 , this figure is a schematic diagram of the principle of dividing the displacement path shown in some embodiments of the present application, which is specifically explained as follows: at the contact point of the roller, due to the interaction force between the roller and the conveyor belt, the surface of the conveyor belt is deformed, which in turn causes the displacement path collected within the range around the contact point of the roller to be slightly bent, that is, the geometric shape of the nonlinear trajectory is constantly changing, while the surface of the conveyor belt in the range not in contact with the roller is smooth, and the collected displacement path does not fluctuate, that is, the geometric shape of the linear trajectory does not change.

[0095] In step 104, the wear amount of the rollers in the conveyor belt is determined according to the curvature radius of each nonlinear track and the inclination angle of the rollers.

[0096] In some embodiments, reference Figure 3 This figure is an exemplary flow chart for determining the amount of wear according to some embodiments of the present application. In the present application, the wear amount of the rollers in the conveyor belt can be determined based on the curvature radius of each nonlinear trajectory and the inclination angle of the rollers by using the following steps, namely:

[0097] In step 1041 , a plurality of curvature radii of each nonlinear trajectory are determined;

[0098] In step 1042, multiple local deformation features of the conveyor belt surface are identified based on the variation features of all curvature radii on the conveyor belt surface;

[0099] In step 1043, the elasticity of the conveyor belt is fitted according to all local deformation characteristics to obtain the elastic coefficient of the conveyor belt;

[0100] In step 1044, the wear amount of the roller in the conveyor belt is determined based on the elastic coefficient and the forward tilt angle of the roller.

[0101] In specific implementation, the determination of multiple curvature radii of each trajectory dynamic curve can be achieved in the following manner: first, discrete sampling is performed on each trajectory dynamic curve to obtain multiple sampling points, and then the curvature radius of each trajectory dynamic curve at each sampling point is calculated, wherein the sampling interval of the discrete sampling is one unit of the horizontal coordinate in the trajectory dynamic curve, that is, sampling is performed at positions where the horizontal coordinate is an integer.

[0102] In a specific implementation, multiple local deformation features of the conveyor belt surface can be identified based on the variation characteristics of all curvature radii on the conveyor belt surface. This can be achieved in the following manner: first, all curvature radii are arranged according to the positions of corresponding sampling points on the nonlinear trajectory, and the arranged sequence is divided into multiple subsequences according to different scales. A scale is selected as the selected scale, and the ratio of the mean value to the standard deviation of each subsequence obtained by the selected scale division is calculated. Then, the average value of all ratios is calculated, and the average value is used as the local deformation feature of the selected scale, and the local deformation features of the remaining scales are further determined. The different scales are multiple scales preset according to needs. For example, the present application presets 10 scales, namely 10, 15, 20, 25, 30, 35, 40, 45, 50, and 55. Dividing the arranged sequence into multiple subsequences according to different scales means dividing the arranged sequence into multiple subsequences of length n, wherein n is the scale. The mean value and standard deviation of the subsequences at different scales are the variation characteristics of all curvature radii on the conveyor belt surface.

[0103] It should be noted that the local deformation feature in this application is a parameter value used to measure the degree of local deformation of the conveyor belt. The larger the local deformation feature, the greater the degree of local deformation of the conveyor belt at a specific scale, and the smaller the local deformation feature, the smaller the degree of local deformation of the conveyor belt at a specific scale. Here, the scale refers to an area of ​​a specific length on the conveyor belt.

[0104] In specific implementation, the elasticity of the conveyor belt is fitted according to all local deformation features, and the elastic coefficient of the conveyor belt can be obtained in the following way, namely: first, the scale corresponding to each local deformation feature is obtained, and all scales and division features are fitted into a linear function curve through the least squares method in the existing technology, wherein the independent variable is the natural logarithm of the scale, and the vertical coordinate is the natural logarithm of the division feature. Finally, the slope of the linear function curve is used as the elastic coefficient of the conveyor belt.

[0105] It should be noted that the elastic coefficient in this application is a parameter value used to measure the elasticity of the conveyor belt. The larger the elastic coefficient, the greater the elasticity of the conveyor belt, and the smaller the elastic coefficient, the smaller the elasticity of the conveyor belt.

[0106] In specific implementation, the wear amount of the roller in the conveyor belt can be determined based on the elastic coefficient and the inclination angle of the roller. That is, first, the inclination angle of the roller set during the carton conveying process is obtained, then the tangent value of the inclination angle is calculated, and the elastic coefficient is divided by the tangent value. Finally, the quotient obtained is used as the wear amount of the roller in the conveyor belt.

[0107] It should be noted that the wear amount in this application is a parameter value used to measure the degree of wear of the rubber layer wrapping the roller. The greater the wear amount, the more serious the wear of the rubber layer wrapping the roller, and the smaller the wear amount, the lighter the wear of the rubber layer wrapping the roller.

[0108] In step 105, the sag deviation of the midpoint of the span between every two linear tracks is determined, and the tension fluctuation index of the conveyor belt during the carton conveying process is determined by all the sag deviations.

[0109] In some embodiments, determining the sag deviation of the midpoint of the span between each two linear tracks may be achieved by using the following steps, namely:

[0110] Determine the midpoint of the span between each two linear traces;

[0111] Determine the sag from the midpoint of each span to each linear track on either side of the midpoint of each span;

[0112] The sag deviation amount of the mid-span point between each two linear tracks is determined based on all sags at each mid-span point.

[0113] In specific implementation, the midpoint of the span between every two linear trajectories can be determined in the following manner: first, two linear trajectories are selected as selected linear trajectories, the average values ​​of all dependent variables and the average values ​​of all independent variables in the curves of the two selected linear trajectories are calculated, the average values ​​of all dependent variables are used as the ordinate, and the average values ​​of all independent variables are used as the abscissa, and then the point determined by the ordinate and the abscissa is used as the midpoint of the span between the two selected linear trajectories, and the midpoint of the span between every remaining two linear trajectories is determined.

[0114] It should be noted that, in this application, the midpoint of the span refers to the center point of the area surrounded by two linear tracks.

[0115] In a specific implementation, the sag from each span midpoint to each linear track on either side of each span midpoint can be determined in the following manner: first, in the coordinate axis of the linear track, a span midpoint is selected as the selected span midpoint, and a straight line perpendicular to the horizontal coordinate is drawn through the selected span midpoint. Then, the angle formed by the straight line and the two linear tracks at the selected span midpoint is used as the sag from the selected span midpoint to the two linear tracks on either side of the selected span midpoint, and the sag from each other span midpoint to each other linear track on either side of each other span midpoint is continuously determined.

[0116] It should be noted that the sag in this application refers to the angle formed by the linear trajectory and the straight line of the conveyor belt entrance and exit boundaries.

[0117] In a specific implementation, the sag deviation amount at the midpoint of the span between every two linear tracks can be determined based on all the sags at the midpoint of each span. This can be achieved by first selecting two linear tracks as selected linear tracks, obtaining two sags at the midpoints of the spans of the two selected linear tracks, taking the difference between the two sags and dividing it by 90°, and then using the obtained quotient as the sag deviation amount at the midpoint of the span between the two selected linear tracks. The sag deviation amount at the midpoint of the span between every two remaining linear tracks is then determined.

[0118] It should be noted that the sag deviation in this application is a parameter that measures the degree of vertical deviation between two linear tracks. The larger the sag deviation, the greater the vertical deviation between the two linear tracks, and the smaller the sag deviation, the smaller the vertical deviation between the two linear tracks.

[0119] In some embodiments, determining the tension fluctuation index of the conveyor belt during the carton conveying process by using all sag deviations can be achieved by the following steps, namely:

[0120] Determine a correlation sequence of sag deviation changes based on all sag deviations;

[0121] determining a cutoff point of the correlation sequence;

[0122] The tension fluctuation index of the conveyor belt during the paper box conveying process is determined according to the cutoff point.

[0123] In a specific implementation, determining the autocorrelation sequence of the sag deviation change based on all sag deviations can be achieved in the following manner: first, obtaining the ordinate of the midpoint of the span corresponding to each sag deviation, then arranging all the sag deviations into a sequence p in ascending order of the ordinate of the midpoint of the span, then arranging the ordinates of all the midpoints of the span into a sequence q in ascending order, then calculating the mutual correlation coefficients between sequence p and sequence q for all lags between lag 1 and lag N, and arranging all the mutual correlation coefficients in ascending order of the lag size, and finally, using the resulting sequence as the correlation sequence of the sag deviation change.

[0124] It should be noted that the correlation sequence in this application is a series that describes the degree of correlation between the change in the sag deviation and the change in the position on the conveyor belt.

[0125] In a specific implementation, determining the truncation point of the correlation sequence may be achieved in the following manner: comparing all the mutual correlation coefficients in the correlation sequence with a preset correlation threshold, and taking the position of the first mutual correlation coefficient smaller than the correlation threshold in the correlation sequence as the truncation point of the correlation sequence, wherein the correlation threshold may be preset to a smaller value greater than zero. For example, in the present application, the correlation threshold is preset to 0.1. In other embodiments, the correlation threshold may also be preset to other values, which are not limited here.

[0126] It should be noted that the truncation point in this application refers to the point where the mutual correlation coefficient in the correlation sequence drops sharply.

[0127] In specific implementation, the tension fluctuation index of the conveyor belt during carton conveying is determined according to the truncation point in the following manner, namely: first, the vertical coordinate of the midpoint of the span corresponding to each sag deviation is obtained, and then all the sag deviations are arranged into a sequence in ascending order of the vertical coordinate of the midpoint of the span, and then the arranged sequence is discretely sampled according to the interval t, and a plurality of sag deviations are obtained by discrete sampling. All the sag deviations obtained by discrete sampling are fitted to a cubic function curve by the least squares method in the prior art, wherein the independent variable is the vertical coordinate of the midpoint of the span corresponding to each sag deviation obtained by discrete sampling, and the dependent variable is all the sag deviations obtained by discrete sampling. Finally, the coefficient of the cubic term of the fitted curve is used as the tension fluctuation index of the conveyor belt during carton conveying, wherein t is the truncation point.

[0128] It should be noted that the tension fluctuation index in this application is a parameter value that measures the uniformity of tension distribution at various points on the conveyor belt during the carton conveying process. The larger the tension fluctuation index, the more uneven the tension distribution at various points on the conveyor belt during the carton conveying process. The smaller the tension fluctuation index, the more uniform the tension distribution at various points on the conveyor belt during the carton conveying process.

[0129] In step 106, the deviation probability of the conveyor belt during the carton conveying process is determined in combination with the wear amount and the tension fluctuation index, and an alarm signal is sent to a monitoring center when the deviation probability is greater than a preset deviation threshold.

[0130] In some embodiments, the following steps may be used to determine the probability of the conveyor belt deviating during the carton conveying process by combining the wear amount and the tension fluctuation index, namely:

[0131] determining a deviation factor of the conveyor belt during the paper box conveying process according to the wear amount and the tension fluctuation index;

[0132] The probability of the conveyor belt running off course during the carton conveying process is determined according to the offset factor.

[0133] In specific implementation, the offset factor of the conveyor belt during the carton conveying process can be determined based on the wear amount and the tension fluctuation index in the following manner, namely, the sum of the wear amount and the tension fluctuation index is used as the offset factor of the conveyor belt during the carton conveying process.

[0134] It should be noted that the offset factor in this application is a parameter value that measures the degree of influence of the wear of the rollers and the tension distribution of the conveyor belt at the current moment on the conveying process. The larger the offset factor, the greater the influence of the wear of the rollers and the tension distribution of the conveyor belt at the current moment on the conveying process. The smaller the offset factor, the smaller the influence of the wear of the rollers and the tension distribution of the conveyor belt at the current moment on the conveying process.

[0135] In specific implementation, the probability of the conveyor belt running off course during the carton conveying process can be determined based on the offset factor in the following manner, namely: the opposite of the offset factor is used as the exponent of a natural constant, and the obtained value is negated and then added to one, and finally the added value is used as the probability of the conveyor belt running off course during the carton conveying process.

[0136] It should be noted that the deviation probability in this application is a measure of the probability value of the conveyor belt deviating abnormally during the carton conveying process. The greater the deviation probability, the greater the probability of the conveyor belt deviating abnormally during the carton conveying process. The smaller the deviation probability, the smaller the probability of the conveyor belt deviating abnormally during the carton conveying process.

[0137] It should be noted that the deviation threshold in this application is a value preset according to actual needs. For example, if the wear amount is greater than or equal to the tension fluctuation index, the uneven distribution of surface tension of the conveyor belt is partly caused by the imbalance of the roller support. Therefore, the deviation threshold is preset to a larger value to reduce false alarms, that is, it is preset to 0.5. If the wear amount is less than the tension fluctuation index, the uneven distribution of surface tension of the conveyor belt is more caused by the conveyor belt itself (including deviation, aging, tearing, etc.), then the deviation threshold is preset to a smaller value to increase sensitivity, that is, it is preset to 0.3. In other embodiments, the deviation threshold can also be preset by other methods, which are not limited here.

[0138] In addition, in another aspect of the present application, in some embodiments, the present application provides a conveying monitoring system for a folding box machine, referring to Figure 4 , this figure is a schematic structural diagram of a conveying monitoring system for a box folding machine according to some embodiments of the present application. The conveying monitoring system 400 for a box folding machine includes: a collection module 401, a processing module 402 and an execution module 403, which are described as follows:

[0139] The acquisition module 401 in this application is mainly used to collect two videos of different perspectives during the carton conveying process through two visual sensors;

[0140] Processing module 402, in this application, the processing module 402 is mainly used to perform stereo matching on the two videos to obtain a depth video stream of the conveyor belt;

[0141] It should be noted that the processing module 402 in the present application is further configured to extract multiple optical flow information from the depth video stream, and for each optical flow information, fit the displacement path of each location on the conveyor belt according to the optical flow information, and divide all displacement paths into nonlinear trajectories in the roller contact area and linear trajectories in the non-roller contact area;

[0142] It should be noted that the processing module 402 in the present application is also used to determine the wear amount of the rollers in the conveyor belt according to the curvature radius of each nonlinear trajectory and the forward tilt angle of the rollers;

[0143] It should be noted that the processing module 402 in the present application is also used to determine the sag deviation of the midpoint of the span between each two linear tracks, and determine the tension fluctuation index of the conveyor belt during the carton conveying process through all the sag deviations;

[0144] Execution module 403, in this application, execution module 403 is mainly used to determine the probability of the conveyor belt deviating during the carton conveying process in combination with the wear amount and the tension fluctuation index, and send an alarm signal to the monitoring center when the deviation probability is greater than the preset deviation threshold.

[0145] In addition, the present application also provides a computer device, which includes a memory and a processor, wherein the memory stores codes, and the processor is configured to obtain the codes and execute the above-mentioned conveying monitoring method for a box folding machine.

[0146] In some embodiments, reference Figure 5 , which is a schematic diagram of the structure of a computer device for implementing a conveying monitoring method for a box folding machine according to some embodiments of the present application. The conveying monitoring method for a box folding machine in the above embodiment can be Figure 5 The computer device 500 shown in FIG. 5 is implemented as shown in FIG. 5 . The computer device 500 includes at least one processor 501 , a communication bus 502 , a memory 503 , and at least one communication interface 504 .

[0147] The processor 501 may be a general-purpose central processing unit (CPU) or an application specific integrated circuit (ASIC).

[0148] The communication bus 502 may be used to transmit information between the aforementioned components.

[0149] The memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD ROM) or other optical disc storage, an optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory 503 may be independent and connected to the processor 501 via the communication bus 502. The memory 503 may also be integrated with the processor 501.

[0150] Memory 503 is used to store program code for executing the present invention, and is controlled by processor 501. Processor 501 is used to execute the program code stored in memory 503. The program code may include one or more software modules. The conveying monitoring method for a folding machine in the above embodiment can be implemented by processor 501 and one or more software modules in the program code stored in memory 503.

[0151] The communication interface 504 uses any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0152] In a specific implementation, as an example, a computer device may include multiple processors, each of which may be a single-core (single CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0153] The aforementioned computer device can be a general-purpose computer device or a dedicated computer device. In a specific implementation, the computer device can be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of this application do not limit the type of computer device.

[0154] In addition, the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned conveying monitoring method for a box folding machine is implemented.

[0155] In summary, in the conveying monitoring method and system for a folding machine disclosed in the embodiment of the present application, first, two videos of different perspectives are collected by two visual sensors during the conveying process of the carton; the two videos are stereo matched to obtain a depth video stream of the conveyor belt; multiple optical flow information in the depth video stream is extracted, and for each optical flow information, the displacement paths of various locations on the conveyor belt are fitted according to each optical flow information, and all displacement paths are divided into nonlinear trajectories in the roller contact area and linear trajectories in the non-roller contact area; the wear amount of the rollers in the conveyor belt is determined according to the curvature radius of each nonlinear trajectory and the inclination angle of the roller; the sag deviation amount of the midpoint of the span between each two linear trajectories is determined, and the tension fluctuation index of the conveyor belt during the conveying of the carton is determined by all the sag deviation amounts; the deviation probability of the conveyor belt during the conveying of the carton is determined in combination with the wear amount and the tension fluctuation index, and an alarm signal is sent to the monitoring center when the deviation probability is greater than a preset deviation threshold.

[0156] It can be seen that the present application targets the uneven surface of the conveyor belt and identifies the displacement paths of various locations on the conveyor belt through the depth map of the conveyor belt. It can more accurately extract the motion information of various locations on the conveyor belt, and divide the geometric shape of the displacement path into linear trajectories and nonlinear trajectories according to whether a large change occurs. Subsequently, the aging of the rollers (i.e., the amount of wear on the rollers) is measured through all nonlinear trajectories, and the aging of the conveyor belt (i.e., the tension fluctuation index of the conveyor belt) is measured based on all linear trajectories. The probability of the conveyor belt running off is then evaluated through the aging of the rollers and the aging of the conveyor belt. In summary, the present application can provide early warning of abnormal conveyor belt deviation based on the aging of the conveying equipment.

[0157] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0158] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A conveying monitoring method for a box folding machine, characterized in that: include: Two visual sensors are used to capture two videos from different perspectives during the carton conveying process; Performing stereo matching on the two videos to obtain a depth video stream of the conveyor belt; Extracting multiple optical flow information from the depth video stream, fitting the displacement paths of various locations on the conveyor belt based on each optical flow information, and dividing all displacement paths into nonlinear trajectories in roller contact areas and linear trajectories in non-roller contact areas; Determining the wear amount of the rollers in the conveyor belt according to the curvature radius of each nonlinear trajectory and the forward inclination angle of the rollers; Determining the sag deviation of the midpoint of the span between each two linear tracks, and determining the tension fluctuation index of the conveyor belt during the carton conveying process based on all the sag deviations; Determining the probability of the conveyor belt deviating during the carton conveying process by combining the wear amount and the tension fluctuation index, and sending an alarm signal to a monitoring center when the deviation probability is greater than a preset deviation threshold; The process of performing stereo matching on the two videos to obtain a depth video stream of the conveyor belt specifically includes: Performing frame synchronization on the two channels of video to obtain an image group for each timestamp; Extracting regions of interest from all images in each image group to obtain a conveyor belt image group of the conveyor belt area; Perform stereo matching on the conveyor belt image group at each time stamp to obtain the depth map of the conveyor belt at each time stamp; Reconstruct all depth maps into a conveyor belt depth video stream; The step of fitting the displacement paths of various locations on the conveyor belt according to each optical flow information specifically includes: Dividing a plurality of monitoring areas on the conveyor belt according to all depth maps in the depth video stream; All optical flow information is filtered according to all monitoring areas to obtain the path optical flow set in each monitoring area; Fit each path optical flow set separately to obtain the displacement path of each point on the conveyor belt; The method of determining the wear amount of the rollers in the conveyor belt according to the curvature radius of each nonlinear trajectory and the inclination angle of the rollers specifically includes: determining a plurality of curvature radii for each nonlinear trajectory; According to the changing characteristics of all curvature radii on the conveyor belt surface, multiple local deformation features of the conveyor belt surface are identified; Fit the elasticity of the conveyor belt according to all local deformation characteristics to obtain the elastic coefficient of the conveyor belt; The wear amount of the roller in the conveyor belt is determined according to the elastic coefficient and the forward tilt angle of the roller.

2. The method according to claim 1, wherein Determining the sag deviation at the mid-span between each two linear tracks specifically includes: Determine the midpoint of the span between each two linear traces; Determine the sag from the midpoint of each span to each linear track on either side of the midpoint of each span; The sag deviation amount of the mid-span point between each two linear tracks is determined based on all sags at each mid-span point.

3. The method according to claim 1, wherein The tension fluctuation index of the conveyor belt during the carton conveying process is determined by all sag deviations, specifically including: Determine a correlation sequence of sag deviation changes based on all sag deviations; determining a cutoff point of the correlation sequence; The tension fluctuation index of the conveyor belt during the paper box conveying process is determined according to the cutoff point.

4. The method according to claim 1, wherein Determining the probability of the conveyor belt deviating during the carton conveying process by combining the wear amount and the tension fluctuation index specifically includes: determining a deviation factor of the conveyor belt during the paper box conveying process according to the wear amount and the tension fluctuation index; The probability of the conveyor belt running off course during the carton conveying process is determined according to the offset factor.

5. A conveying monitoring system for a box folding machine, which adopts the method according to any one of claims 1 to 4 for conveying monitoring, characterized in that: The conveying monitoring system for the box folding machine includes: The acquisition module is used to collect two videos from different perspectives during the carton conveying process through two visual sensors; A processing module, configured to perform stereo matching on the two videos to obtain a depth video stream of the conveyor belt; The processing module is further configured to extract a plurality of optical flow information from the depth video stream, and for each optical flow information, fit the displacement path of each location on the conveyor belt according to the optical flow information, and divide all displacement paths into nonlinear trajectories in roller contact areas and linear trajectories in non-roller contact areas; The processing module is further configured to determine the wear amount of the rollers in the conveyor belt according to the curvature radius of each nonlinear track and the inclination angle of the rollers; The processing module is further configured to determine a sag deviation at a midpoint of a span between every two linear tracks, and determine a tension fluctuation index of the conveyor belt during the carton conveying process based on all sag deviations; An execution module is used to determine the deviation probability of the conveyor belt during the paper box conveying process based on the wear amount and the tension fluctuation index, and send an alarm signal to a monitoring center when the deviation probability is greater than a preset deviation threshold.

6. A computer device, characterized in that: The computer device includes a memory and a processor, wherein the memory stores codes, and the processor is configured to obtain the codes and execute the conveying monitoring method for a box folding machine according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the conveying monitoring method for a box folding machine according to any one of claims 1 to 4 is implemented.

Images