Conveying monitoring method and system for box folding machine

Abstract

The invention provides a conveying monitoring method and system for a box folding machine. The method comprises the steps that a depth video stream of a conveying belt in the conveying process is collected; fitting displacement paths of all positions on the conveying belt according to optical flow information in the depth video flow, and dividing all the displacement paths into a nonlinear track of a carrier roller contact area and a linear track of a non-carrier roller contact area; the abrasion loss of the carrier roller in the conveying belt is determined according to the curvature radius of each non-linear track and the front rake angle of the carrier roller; the sag deviation value of the midpoint of the span between every two linear tracks is determined, and the tension fluctuation index of the conveying belt in the paper box conveying process is determined through all the sag deviation values; and the deviation probability of the conveying belt in the carton conveying process is determined by combining the abrasion loss and the tension fluctuation index, and an alarm signal is sent to a monitoring center when the deviation probability is larger than a preset deviation threshold value. By the adoption of the scheme, early warning can be conducted on deviation abnormity of the conveying belt according to the aging condition of the conveying equipment.

Description

Technical Field

[0001] The present application relates to the field of industrial production technology, and more specifically, to a conveying monitoring method and system for a box folding machine. Background Art

[0002] Industrial production conveying refers to the use of conveying equipment to achieve efficient transportation of raw materials, semi-finished products or finished products in various manufacturing, processing, logistics and mining industries. The conveying system is an important part of modern industrial automation and is widely used in coal mining, electricity, steel, chemical industry, food processing, pharmaceuticals, port logistics and other fields. Among them, the conveying system of the box folding machine is mainly used to transfer the folded cartons from one stage of the production line to another.

[0003] The conveying system of a box folding machine is usually composed of a conveyor belt. Common problems of the conveyor belt during the conveying process include deviation and tearing. Among them, tearing is mainly caused by scratches from hard objects and severe deviation. Therefore, during the conveying process of the box folding machine, it is extremely important to monitor the deviation of the conveyor belt. In the prior art, the deviation of the conveyor belt is usually monitored by a deviation sensor. When the conveyor belt deviates, the conveyor belt drives the deviation sensor to operate, thereby sounding an alarm. However, this deviation monitoring method can only sound an alarm when the conveyor belt deviates, and since the deviation sensor is usually installed on both sides of the conveyor belt, if the conveyor belt structure is deformed due to aging of the equipment, the abnormality cannot be detected. Therefore, how to give early warning of conveyor belt deviation abnormalities according to the aging of the conveying equipment has become a difficult problem faced by the industry. Summary of the invention

[0004] The present application provides a conveying monitoring method and system for a box folding machine, which can provide early warning of abnormal deviation of a conveyor belt according to the aging condition of the conveying equipment.

[0005] In a first aspect, the present application provides a conveying monitoring method for a box folding machine, comprising: Two visual sensors are used to collect two videos from different perspectives during the carton conveying process; Performing stereo matching on the two videos to obtain a depth video stream of the conveyor belt; Extracting multiple optical flow information from the depth video stream, fitting the displacement paths of various locations on the conveyor belt according to each optical flow information, and dividing all displacement paths into nonlinear tracks in roller contact areas and linear tracks in non-roller contact areas; Determining the wear amount of the rollers in the conveyor belt according to the curvature radius of each nonlinear trajectory and the forward inclination angle of the rollers; Determine the sag deviation of the midpoint of the span between every two linear tracks, and determine the tension fluctuation index of the conveyor belt during the paper box conveying process through all the sag deviations; The deviation probability of the conveyor belt during the paper box conveying process is determined in combination with the wear amount and the tension fluctuation index, and an alarm signal is sent to a monitoring center when the deviation probability is greater than a preset deviation threshold.

[0006] In some embodiments, performing stereo matching on the two videos to obtain a depth video stream of the conveyor belt specifically includes: Performing frame synchronization on the two channels of video to obtain an image group for each timestamp; Extracting the region of interest from all images in each image group to obtain a conveyor belt image group of the conveyor belt area; Perform stereo matching on the conveyor belt image group at each timestamp to obtain a depth map of the conveyor belt at each timestamp; Reconstruct all depth maps into a conveyor belt depth video stream.

[0007] In some embodiments, fitting the displacement path of each location on the conveyor belt according to each optical flow information specifically includes: Dividing a plurality of monitoring areas on the conveyor belt according to all depth images in the depth video stream; All optical flow information is filtered according to all monitoring areas to obtain the path optical flow set in each monitoring area; The optical flow set of each path is fitted separately to obtain the displacement path of each location on the conveyor belt.

[0008] In some embodiments, determining the wear amount of the rollers in the conveyor belt according to the curvature radius of each nonlinear trajectory and the forward inclination angle of the rollers specifically includes: determining a plurality of radii of curvature for each nonlinear trajectory; According to the changing characteristics of all the curvature radii on the conveyor belt surface, multiple local deformation features of the conveyor belt surface are identified; Fit the elasticity of the conveyor belt according to all local deformation characteristics to obtain the elastic coefficient of the conveyor belt; The wear amount of the roller in the conveyor belt is determined according to the elastic coefficient and the forward inclination angle of the roller.

[0009] In some embodiments, determining the sag deviation of the midpoint of the span between every two linear tracks specifically includes: Determine the midpoint of the span between every two linear tracks; Determine the sag from the midpoint of each span to each linear track on either side of the midpoint of each span; The sag deviation amount of the mid-span point between each two linear tracks is determined based on all sags at each mid-span point.

[0010] In some embodiments, determining the tension fluctuation index of the conveyor belt during the carton conveying process by all sag deviations specifically includes: Determine a correlation sequence of sag deviation changes based on all sag deviations; Determining a cutoff point of the correlation sequence; The tension fluctuation index of the conveyor belt during the paper box conveying process is determined according to the cutoff point.

[0011] In some embodiments, determining the deviation probability of the conveyor belt during the paper box conveying process in combination with the wear amount and the tension fluctuation index specifically includes: Determining a deviation factor of the conveyor belt during the paper box conveying process according to the wear amount and the tension fluctuation index; The probability of deviation of the conveyor belt during the paper box conveying process is determined according to the offset factor.

[0012] In a second aspect, the present application provides a conveying monitoring system for a box folding machine, comprising: The acquisition module is used to collect two videos from different perspectives during the carton conveying process through two visual sensors; A processing module, used for performing stereo matching on the two videos to obtain a depth video stream of the conveyor belt; The processing module is further used to extract multiple optical flow information from the depth video stream, and for each optical flow information, fit the displacement path of each location on the conveyor belt according to each optical flow information, and divide all displacement paths into nonlinear tracks in the roller contact area and linear tracks in the non-roller contact area; The processing module is also used to determine the wear amount of the rollers in the conveyor belt according to the curvature radius of each nonlinear track and the forward inclination angle of the rollers; The processing module is also used to determine the sag deviation of the midpoint of the span between every two linear tracks, and determine the tension fluctuation index of the conveyor belt during the paper box conveying process through all the sag deviations; The execution module is used to determine the deviation probability of the conveyor belt during the paper box conveying process by combining the wear amount and the tension fluctuation index, and send an alarm signal to a monitoring center when the deviation probability is greater than a preset deviation threshold.

[0013] In a third aspect, the present application provides a computer device, the computer device comprising a memory and a processor, the memory storing codes, the processor being configured to obtain the codes and execute the above-mentioned conveying monitoring method for a box folding machine.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned conveying monitoring method for a box folding machine is implemented.

[0015] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects: In the conveying monitoring method and system for a folding machine provided in the present application, first, two visual sensors are used to collect two videos of different perspectives during the conveying of a carton; the two videos are stereo matched to obtain a depth video stream of the conveyor belt; multiple optical flow information in the depth video stream is extracted, and for each optical flow information, the displacement paths at various locations on the conveyor belt are fitted according to each optical flow information, and all displacement paths are divided into nonlinear trajectories in the roller contact area and linear trajectories in the non-roller contact area; the wear amount of the rollers in the conveyor belt is determined according to the curvature radius of each nonlinear trajectory and the inclination angle of the roller; the sag deviation amount of the midpoint of the span between every two linear trajectories is determined, and the tension fluctuation index of the conveyor belt during the conveying of a carton is determined by all the sag deviation amounts; the deviation probability of the conveyor belt during the conveying of a carton is determined in combination with the wear amount and the tension fluctuation index, and an alarm signal is sent to a monitoring center when the deviation probability is greater than a preset deviation threshold.

[0016] It can be seen that the present application targets the uneven surface of the conveyor belt and identifies the displacement paths at various locations on the conveyor belt through the depth map of the conveyor belt. It can more accurately extract the motion information at various locations on the conveyor belt, and divide the geometric shape of the displacement path into linear trajectories and nonlinear trajectories according to whether a large change occurs. Subsequently, the aging of the rollers (i.e., the amount of wear on the rollers) is measured through all nonlinear trajectories, and the aging of the conveyor belts (i.e., the tension fluctuation index of the conveyor belts) is measured based on all linear trajectories. The probability of the conveyor belt running off the track is then evaluated through the aging of the rollers and the aging of the conveyor belt. In summary, the present application can provide early warning of abnormal conveyor belt deviation based on the aging of the conveying equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is an exemplary flow chart of a conveying monitoring method for a box folding machine according to some embodiments of the present application; Figure 2 It is a schematic diagram of the principle of dividing the displacement path according to some embodiments of the present application; Figure 3 is an exemplary flow chart for determining the amount of wear according to some embodiments of the present application; Figure 4 is a schematic structural diagram of a conveying monitoring system for a box folding machine according to some embodiments of the present application; Figure 5 It is a structural schematic diagram of a computer device for implementing a conveying monitoring method for a box folding machine according to some embodiments of the present application. DETAILED DESCRIPTION

[0018] In order to better understand the technical solution of the present application, the technical solution of the present application will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0019] refer to Figure 1 , which is an exemplary flow chart of a conveying monitoring method for a box folding machine according to some embodiments of the present application. The conveying monitoring method 100 for a box folding machine mainly includes the following steps: In step 101, two visual sensors are used to collect two videos from different perspectives during the paper box conveying process.

[0020] In specific implementation, collecting two videos of different perspectives during the carton conveying process by two visual sensors can be achieved in the following manner, namely: installing a visual sensor on the left and right sides of the same height above the conveyor belt, collecting videos of the conveyor belt during the carton conveying process by the two visual sensors, and using the videos collected by the two visual sensors as two videos of different perspectives during the carton conveying process, wherein the frame rate of the video collected by the visual sensor can be preset as required, for example, the frame rate is set to 30FPS in the present application.

[0021] In step 102, stereo matching is performed on the two videos to obtain a depth video stream of the conveyor belt.

[0022] In some embodiments, stereo matching of the two videos to obtain a depth video stream of the conveyor belt may be implemented by the following steps, namely: Performing frame synchronization on the two channels of video to obtain an image group for each timestamp; Extracting the region of interest from all images in each image group to obtain a conveyor belt image group of the conveyor belt area; Perform stereo matching on the conveyor belt image group at each timestamp to obtain a depth map of the conveyor belt at each timestamp; Reconstruct all depth maps into a conveyor belt depth video stream.

[0023] In a specific implementation, the two videos are frame synchronized to obtain an image group for each frame in the following manner: first, the timestamp of each frame image in the two videos is obtained; then, the images with the closest timestamps in the two videos are divided into the same set, and all the obtained sets are used as image groups for each timestamp.

[0024] It should be noted that the image group in this application is a collection of images collected by two visual sensors at the same time.

[0025] In specific implementation, the region of interest is extracted from all images in each image group, and the conveyor belt image group of the conveyor belt area can be obtained in the following manner, namely: select an image group as the selected image group, first, perform edge detection on two images in the selected image group respectively by using the Canny algorithm in the prior art to obtain edge images of the two images, then extract all straight lines in each edge image by using the Hough transform, then extract all straight lines in each edge image whose straight line length is greater than a preset length threshold, and select all two straight lines parallel to each other from the extracted straight lines, and calculate the span between each two straight lines parallel to each other, finally, the area between the two straight lines with the largest span in each image is respectively used as the conveyor belt image of each image, and the set composed of all conveyor belt images is used as the conveyor belt image group of the conveyor belt area, and continue to determine the conveyor belt image groups of the remaining selected image groups, wherein the length threshold can be preset according to the resolution of the visual sensor, for example, the resolution of the visual sensor in the present application is 1280x1024, and the length threshold is preset to half of the width in the resolution, that is, half of 1280, that is, 640 pixels.

[0026] It should be noted that the conveyor belt image group in the present application is a group of images with the background removed and only including the conveyor belt area.

[0027] In specific implementation, stereo matching is performed on the conveyor belt image group at each timestamp to obtain the depth map of the conveyor belt at each timestamp. This can be achieved in the following way, namely: for the conveyor belt image group at each timestamp, stereo matching can be performed on the two conveyor belt images in each conveyor belt image group using the AD-Census stereo matching algorithm in the prior art, and all the results of the stereo matching are used as the depth maps of each timestamp.

[0028] It should be noted that the depth map in this application is an image used to describe the concavity and convexity of the conveyor belt surface.

[0029] In a specific implementation, reconstructing all the depth maps into the depth video stream of the conveyor belt can be achieved in the following manner, namely: arranging all the depth maps in the order of the timestamp of each depth map, using the arranged images as each frame image in the video, and then using the obtained video as the depth video stream of the conveyor belt.

[0030] It should be noted that the depth video stream in the present application is a video composed of continuous depth maps, which is used to describe the concavity and convexity of the conveyor belt surface.

[0031] In step 103, multiple optical flow information in the depth video stream is extracted, and for each optical flow information, the displacement path of each location on the conveyor belt is fitted according to each optical flow information, and all displacement paths are divided into nonlinear trajectories in the roller contact area and linear trajectories in the non-roller contact area.

[0032] In some embodiments, extracting multiple optical flow information from the depth video stream refers to performing optical flow extraction on the depth video stream by an optical flow method to obtain multiple optical flow fields, and using all the extracted optical flow fields as optical flow information.

[0033] In a specific implementation, the optical flow method is used to extract the optical flow of the depth video stream, and the extracted results are used as the optical flow information, which can be implemented in the following way, that is, the optical flow can be extracted from the depth video stream through the sub-function CalcOpticalFlowFarneback of OpenCV.

[0034] It should be noted that the optical flow information in this application is a set of vectors that describe the motion trajectory of pixels in the depth video stream.

[0035] In some embodiments, fitting the displacement path of each location on the conveyor belt according to each optical flow information can be achieved by using the following steps, namely: Dividing a plurality of monitoring areas on the conveyor belt according to all depth images in the depth video stream; All optical flow information is filtered according to all monitoring areas to obtain the path optical flow set in each monitoring area; The optical flow set of each path is fitted separately to obtain the displacement path of each location on the conveyor belt.

[0036] In specific implementation, dividing the conveyor belt into multiple monitoring areas according to all depth maps in the depth video stream can be achieved in the following manner, namely: selecting a depth map in the depth video stream as the selected depth map, first, determining the difference in depth value between each pixel in the selected depth map and each pixel in the surrounding 3x3 range, then connecting all pixels whose difference is less than a preset depth threshold, and then using all areas formed by the connected pixels as texture areas, and continuing to determine multiple texture areas of the remaining depth maps in the depth video stream, and finally, superimposing the texture areas of all depth maps in the same image, and using each superimposed area as the monitoring area on the conveyor belt, wherein the depth threshold is a value preset according to the texture depth of the conveyor belt surface. For example, in the present application, the depth threshold is preset to 0.8 times the texture depth of the conveyor belt surface.

[0037] It should be noted that the monitoring area in the present application refers to the area on the surface of the conveyor belt selected for path detection, and the monitoring area is an area on the surface of the conveyor belt with obvious texture features.

[0038] In the specific implementation, all optical flow information is filtered according to all monitoring areas, and the path optical flow set in each monitoring area can be obtained by the following method, namely: all pixels in the monitoring area are filtered out, one monitoring area is selected as the selected monitoring area, and the optical flow vectors corresponding to the pixels in the selected monitoring area are extracted from all optical flow information, and then the set of all extracted optical flow vectors is used as the path optical flow set in the selected monitoring area, and the path optical flow sets in the remaining monitoring areas are continued to be determined.

[0039] It should be noted that the path optical flow set in this application refers to the set of all optical flow vectors in the monitoring area.

[0040] In the specific implementation, each path optical flow set is fitted separately, and then the displacement path of each point on the conveyor belt can be obtained in the following way, namely: first, the optical flow accumulation calculation is performed on all the optical flow vectors in the path optical flow set, that is, the optical flow vectors at each pixel point in the path optical flow set are accumulated frame by frame. For example, any pixel point a is selected, and the optical flow vector of the pixel point a in the first frame is obtained. The optical flow vector of the first frame is added to the coordinates of the pixel point a, and the obtained vector is used as the position of the pixel point b in the next frame. Subsequently, the optical flow vector of the pixel point b in the second frame is continued to be obtained, and the optical flow vector of the pixel point b in the second frame is added to the optical flow vector of the pixel point b in the second frame. The vector is added to the coordinates of the pixel point b, and the obtained vector is used as the position of the pixel point c in the next frame, and so on, until the boundary of the depth video stream is reached. Finally, for each monitoring area, the coordinates of all pixel points obtained by the path optical flow set of each monitoring area through the optical flow accumulation calculation are arranged from small to large according to the size of the ordinate, and the coordinates of all the arranged pixel points are fitted into curves respectively, and all the obtained curves are used as the displacement path of each monitoring area, wherein the ordinate of the pixel point is used as the independent variable and the abscissa as the dependent variable during fitting, and fitting is performed by the least squares algorithm in the prior art.

[0041] It should be noted that the displacement path in the present application is a curve that describes the displacement conditions at various locations on the conveyor belt during the paper box conveying process.

[0042] In some embodiments, the division of all displacement paths into nonlinear trajectories in the roller contact area and linear trajectories in the non-roller contact area can be achieved by the following steps, namely: determining the geometric characteristics of each displacement path; According to all geometric features, all displacement paths are divided into nonlinear trajectories in the roller contact area and linear trajectories in the non-roller contact area.

[0043] In specific implementation, the geometric features of each displacement path can be determined in the following manner, namely: first, select a displacement path as the selected displacement path, calculate the first-order derivative curve and the second-order derivative curve of the selected displacement path, then multiply the first-order derivative curve by the second-order derivative curve, take the absolute value of the obtained curve and divide it by the cube of the selected displacement path, and then perform a definite integral operation on the obtained curve, where the lower limit of the integral is 0 and the upper limit of the integral is the height of the depth map in this application, and finally, use the result of the definite integral operation as the geometric feature of the selected displacement path, and continue to determine the geometric features of the remaining displacement paths.

[0044] It should be noted that the geometric feature in this application is a parameter value that describes the geometric shape of the displacement path. The larger the geometric feature, the greater the difference between the geometric shape of the displacement path and the straight line. The smaller the geometric feature, the smaller the difference between the geometric shape of the displacement path and the straight line.

[0045] In specific implementation, all displacement paths are divided into nonlinear trajectories of the roller contact area and linear trajectories of the non-roller contact area according to all geometric features. This can be achieved in the following manner, namely: all geometric features are compared with a preset geometric threshold, and all displacement paths with geometric features greater than the geometric threshold are regarded as nonlinear trajectories of the roller contact area, and all displacement paths with geometric features less than or equal to the geometric threshold are regarded as linear trajectories of the non-roller contact area, wherein the geometric threshold is a value preset according to actual needs. For example, in the present application, the geometric threshold is preset to 0.2.

[0046] It should be noted that in the present application, the roller contact area refers to the area below the conveyor belt that is in direct contact with the roller, and the non-roller contact area refers to the area below the conveyor belt that is not in direct contact with the roller.

[0047] In addition, it should be noted that in the present application, a nonlinear trajectory refers to a displacement path whose geometric shape changes dynamically, and a linear trajectory refers to a displacement path whose geometric shape does not change.

[0048] In some embodiments, reference Figure 2 , this figure is a schematic diagram of the principle of dividing the displacement path shown in some embodiments of the present application, which is specifically explained as follows: at the contact point of the roller, due to the interaction force between the roller and the conveyor belt, the surface of the conveyor belt is deformed, which in turn causes the displacement path collected in the range around the contact point of the roller to be slightly bent, that is, the geometric shape of the nonlinear trajectory is constantly changing, while the surface of the conveyor belt in the range not in contact with the roller is flat, and the collected displacement path does not fluctuate, that is, the geometric shape of the linear trajectory does not change.

[0049] In step 104, the wear amount of the rollers in the conveyor belt is determined according to the curvature radius of each nonlinear trajectory and the forward inclination angle of the rollers.

[0050] In some embodiments, reference Figure 3 , which is an exemplary flow chart for determining the amount of wear according to some embodiments of the present application. In the present application, the amount of wear of the roller in the conveyor belt can be determined according to the radius of curvature of each nonlinear trajectory and the forward inclination angle of the roller by the following steps, namely: In step 1041, a plurality of curvature radii of each nonlinear trajectory are determined; In step 1042, multiple local deformation features of the conveyor belt surface are identified according to the variation features of all the curvature radii on the conveyor belt surface; In step 1043, the elasticity of the conveyor belt is fitted according to all local deformation characteristics to obtain the elastic coefficient of the conveyor belt; In step 1044, the wear amount of the roller in the conveyor belt is determined based on the elastic coefficient and the forward inclination angle of the roller.

[0051] In specific implementation, the determination of multiple curvature radii of each trajectory dynamic curve can be achieved in the following manner, namely: first, discrete sampling is performed on each trajectory dynamic curve to obtain multiple sampling points, and then the curvature radius of each trajectory dynamic curve at each sampling point is calculated, wherein the sampling interval of discrete sampling is one unit of the horizontal coordinate in the trajectory dynamic curve, that is, sampling is performed at a position where the horizontal coordinate is an integer.

[0052] In specific implementation, multiple local deformation features of the conveyor belt surface can be identified based on the change characteristics of all curvature radii on the conveyor belt surface. The following method is used, namely: first, all curvature radii are arranged according to the positions of corresponding sampling points on the nonlinear trajectory, and the arranged sequence is divided into multiple subsequences according to different scales, a scale is selected as the selected scale, and the ratio of the average value to the standard deviation of each subsequence obtained by dividing the selected scale is calculated, and then the average value of all ratios is calculated, and the average value is used as the local deformation feature of the selected scale, and the local deformation features of the remaining scales are further determined, wherein the different scales are multiple scales preset according to needs. For example, the present application presets 10 scales, namely 10, 15, 20, 25, 30, 35, 40, 45, 50, and 55. Dividing the arranged sequence into multiple subsequences according to different scales means dividing the arranged sequence into multiple subsequences of length n, wherein n is the scale, and the average value and standard deviation of the subsequences at different scales are the change characteristics of all curvature radii on the conveyor belt surface.

[0053] It should be noted that the local deformation feature in the present application is a parameter value used to measure the degree of local deformation of the conveyor belt. The larger the local deformation feature, the greater the degree of local deformation of the conveyor belt at a specific scale, and the smaller the local deformation feature, the smaller the degree of local deformation of the conveyor belt at a specific scale, wherein the scale refers to an area of ​​a specific length on the conveyor belt.

[0054] In specific implementation, the elasticity of the conveyor belt is fitted according to all local deformation features, and the elastic coefficient of the conveyor belt can be obtained in the following manner, namely: first, the scale corresponding to each local deformation feature is obtained, and all scales and division features are fitted into a linear function curve by the least squares method in the prior art, wherein the independent variable is the natural logarithm of the scale, and the ordinate is the natural logarithm of the division feature; finally, the slope of the linear function curve is used as the elastic coefficient of the conveyor belt.

[0055] It should be noted that the elastic coefficient in the present application is a parameter value used to measure the elasticity of the conveyor belt. The larger the elastic coefficient, the greater the elasticity of the conveyor belt, and the smaller the elastic coefficient, the smaller the elasticity of the conveyor belt.

[0056] In specific implementation, the wear amount of the roller in the conveyor belt can be determined based on the elastic coefficient and the inclination angle of the roller. That is, first, the inclination angle of the roller set during the carton conveying process is obtained, then the tangent value of the inclination angle is calculated, and the elastic coefficient is divided by the tangent value. Finally, the quotient is used as the wear amount of the roller in the conveyor belt.

[0057] It should be noted that the wear amount in this application is a parameter value used to measure the degree of wear of the rubber layer wrapped around the roller. The greater the wear amount, the more serious the wear of the rubber layer wrapped around the roller, and the smaller the wear amount, the lighter the wear of the rubber layer wrapped around the roller.

[0058] In step 105, the sag deviation of the midpoint of the span between every two linear tracks is determined, and the tension fluctuation index of the conveyor belt during the paper box conveying process is determined through all the sag deviations.

[0059] In some embodiments, determining the sag deviation of the midpoint of the span between each two linear tracks may be achieved by using the following steps, namely: Determine the midpoint of the span between every two linear tracks; Determine the sag from the midpoint of each span to each linear track on either side of the midpoint of each span; The sag deviation amount of the mid-span point between each two linear tracks is determined based on all sags at each mid-span point.

[0060] In specific implementation, the midpoint of the span between every two linear trajectories can be determined in the following manner, namely: first, two linear trajectories are selected as selected linear trajectories, the average values ​​of all dependent variables and the average values ​​of all independent variables in the curves of the two selected linear trajectories are calculated, the average values ​​of all dependent variables are used as the ordinate, and the average values ​​of all independent variables are used as the abscissa, and then the point determined by the ordinate and the abscissa is used as the midpoint of the span between the two selected linear trajectories, and the midpoint of the span between every remaining two linear trajectories is continued to be determined.

[0061] It should be noted that the midpoint of the span in the present application refers to the center point of the area surrounded by two linear tracks.

[0062] In specific implementation, the following method can be used to determine the verticality from each span midpoint to each linear track on both sides of each span midpoint, namely: first, in the coordinate axis of the linear track, a span midpoint is selected as the selected span midpoint, and a straight line perpendicular to the horizontal axis is made through the selected span midpoint. Then, the angle formed by the straight line and the two linear tracks of the selected span midpoint is used as the verticality from the selected span midpoint to the two linear tracks on both sides of the selected span midpoint, and the verticality from each other span midpoint to each other linear track on both sides of each span midpoint is continued to be determined.

[0063] It should be noted that the sag in the present application refers to the angle formed by the linear trajectory and the straight line of the conveyor belt inlet and outlet boundaries.

[0064] In specific implementation, the sag deviation amount of the midpoint of the span between every two linear tracks can be determined based on all the sags of the midpoint of each span, which can be achieved in the following manner: first, two linear tracks are selected as selected linear tracks, and the two sags of the midpoints of the spans of the two selected linear tracks are obtained. The two sags are subtracted and divided by 90°, and then the quotient is used as the sag deviation amount of the midpoint of the span between the two selected linear tracks, and the sag deviation amount of the midpoint of the span between the remaining two linear tracks is continued to be determined.

[0065] It should be noted that the sag deviation in the present application is a parameter for measuring the degree of verticality deviation between two linear tracks. The larger the sag deviation, the larger the verticality deviation between the two linear tracks, and the smaller the sag deviation, the smaller the verticality deviation between the two linear tracks.

[0066] In some embodiments, determining the tension fluctuation index of the conveyor belt during the carton conveying process by all sag deviations can be achieved by the following steps, namely: Determine a correlation sequence of sag deviation changes based on all sag deviations; Determining a cutoff point of the correlation sequence; The tension fluctuation index of the conveyor belt during the paper box conveying process is determined according to the cutoff point.

[0067] In specific implementation, the autocorrelation sequence of the sag deviation change determined based on all the sag deviations can be implemented in the following manner, namely: first, the ordinate of the midpoint of the span corresponding to each sag deviation is obtained, and then all the sag deviations are arranged into a sequence p in the order of the ordinate of the midpoint of the span from small to large, and then the ordinates of all the midpoints of the span are arranged into a sequence q from small to large, and then the mutual correlation coefficients of the sequence p and the sequence q at all lags between lag 1 and lag N are calculated, and all the mutual correlation coefficients are arranged from small to large according to the size of the lag, and finally, the sequence finally arranged is used as the correlation sequence of the sag deviation change.

[0068] It should be noted that the correlation sequence in the present application is a series that describes the degree of correlation between the change in the sag deviation and the change in the position on the conveyor belt.

[0069] In specific implementation, determining the truncation point of the correlation sequence may be achieved in the following manner, namely: comparing all the mutual correlation coefficients in the correlation sequence with a preset correlation threshold, and taking the position of the first mutual correlation coefficient smaller than the correlation threshold in the correlation sequence as the truncation point of the correlation sequence, wherein the correlation threshold may be preset to a smaller value greater than zero. For example, in the present application, the correlation threshold is preset to 0.1. In other embodiments, the correlation threshold may also be preset to other values, which are not limited here.

[0070] It should be noted that the truncation point in the present application refers to the point where the mutual correlation coefficient in the correlation sequence drops sharply.

[0071] In specific implementation, the tension fluctuation index of the conveyor belt during carton conveying can be determined according to the truncation point in the following manner, namely: first, the ordinate of the midpoint of the span corresponding to each sag deviation is obtained, and then all the sag deviations are arranged into a sequence in ascending order of the ordinate of the midpoint of the span, and then the arranged sequence is discretely sampled at an interval t to obtain a plurality of sag deviations, and all the sag deviations obtained by discrete sampling are fitted to a cubic function curve by the least squares method in the prior art, wherein the independent variable is the ordinate of the midpoint of the span corresponding to each sag deviation obtained by discrete sampling, and the dependent variable is all the sag deviations obtained by discrete sampling, and finally, the coefficient of the cubic term of the fitted curve is used as the tension fluctuation index of the conveyor belt during carton conveying, wherein t is the truncation point.

[0072] It should be noted that the tension fluctuation index in the present application is a parameter value that measures the uniformity of tension distribution at various locations on the conveyor belt during the carton conveying process. The larger the tension fluctuation index, the more uneven the tension distribution at various locations on the conveyor belt during the carton conveying process. The smaller the tension fluctuation index, the more uniform the tension distribution at various locations on the conveyor belt during the carton conveying process.

[0073] In step 106, the deviation probability of the conveyor belt during the paper box conveying process is determined in combination with the wear amount and the tension fluctuation index, and an alarm signal is sent to a monitoring center when the deviation probability is greater than a preset deviation threshold.

[0074] In some embodiments, the following steps may be used to determine the deviation probability of the conveyor belt during the paper box conveying process by combining the wear amount and the tension fluctuation index, namely: Determining a deviation factor of the conveyor belt during the paper box conveying process according to the wear amount and the tension fluctuation index; The probability of deviation of the conveyor belt during the paper box conveying process is determined according to the offset factor.

[0075] In specific implementation, determining the displacement factor of the conveyor belt during carton conveying according to the wear amount and the tension fluctuation index can be achieved in the following manner, namely: taking the sum of the wear amount and the tension fluctuation index as the displacement factor of the conveyor belt during carton conveying.

[0076] It should be noted that the offset factor in the present application is a parameter value that measures the degree of influence of the wear of the rollers and the tension distribution of the conveyor belt at the current moment on the conveying process. The larger the offset factor, the greater the influence of the wear of the rollers and the tension distribution of the conveyor belt at the current moment on the conveying process. The smaller the offset factor, the smaller the influence of the wear of the rollers and the tension distribution of the conveyor belt at the current moment on the conveying process.

[0077] In specific implementation, the probability of the conveyor belt running off course during the carton conveying process can be determined based on the offset factor in the following manner, namely: the opposite of the offset factor is used as the exponent of a natural constant, and the obtained value is negated and then added to one, and finally the added value is used as the probability of the conveyor belt running off course during the carton conveying process.

[0078] It should be noted that the deviation probability in the present application is a measure of the probability value of the conveyor belt running abnormally during the carton conveying process. The greater the deviation probability, the greater the probability of the conveyor belt running abnormally during the carton conveying process. The smaller the deviation probability, the smaller the probability of the conveyor belt running abnormally during the carton conveying process.

[0079] It should be noted that the deviation threshold in the present application is a value preset according to actual needs. For example, if the wear amount is greater than or equal to the tension fluctuation index, the uneven distribution of surface tension of the conveyor belt is partly caused by the imbalance of roller support. Therefore, the deviation threshold is preset to a larger value to reduce false alarms, that is, it is preset to 0.5. If the wear amount is less than the tension fluctuation index, the uneven distribution of surface tension of the conveyor belt is more caused by the conveyor belt itself (including deviation, aging, tearing, etc.), then the deviation threshold is preset to a smaller value to increase the sensitivity, that is, it is preset to 0.3. In other embodiments, the deviation threshold can also be preset by other methods, which are not limited here.

[0080] In addition, in another aspect of the present application, in some embodiments, the present application provides a conveying monitoring system for a folding box machine, referring to Figure 4 , which is a schematic diagram of the structure of a conveying monitoring system for a box folding machine according to some embodiments of the present application, the conveying monitoring system 400 for a box folding machine includes: a collection module 401, a processing module 402 and an execution module 403, which are respectively described as follows: Acquisition module 401, in this application, acquisition module 401 is mainly used to acquire two videos of different perspectives during the carton conveying process through two visual sensors; Processing module 402, in this application, processing module 402 is mainly used to perform stereo matching on the two videos to obtain a depth video stream of the conveyor belt; It should be noted that the processing module 402 in the present application is also used to extract multiple optical flow information in the depth video stream, and for each optical flow information, the displacement path of each location on the conveyor belt is fitted according to each optical flow information, and all displacement paths are divided into nonlinear tracks of the roller contact area and linear tracks of the non-roller contact area; It should be noted that the processing module 402 in the present application is also used to determine the wear amount of the rollers in the conveyor belt according to the curvature radius of each nonlinear trajectory and the forward inclination angle of the rollers; It should be noted that the processing module 402 in the present application is also used to determine the sag deviation of the midpoint of the span between every two linear tracks, and determine the tension fluctuation index of the conveyor belt during the paper box conveying process through all the sag deviations; Execution module 403, in the present application, execution module 403 is mainly used to determine the probability of deviation of the conveyor belt during the carton conveying process by combining the wear amount and the tension fluctuation index, and to send an alarm signal to the monitoring center when the deviation probability is greater than a preset deviation threshold.

[0081] In addition, the present application also provides a computer device, which includes a memory and a processor, wherein the memory stores codes, and the processor is configured to obtain the codes and execute the above-mentioned conveying monitoring method for a box folding machine.

[0082] In some embodiments, reference Figure 5 , which is a schematic diagram of the structure of a computer device for implementing a conveying monitoring method for a box folding machine according to some embodiments of the present application. The conveying monitoring method for a box folding machine in the above embodiment can be Figure 5 The computer device 500 shown in the figure is implemented, and the computer device 500 includes at least one processor 501, a communication bus 502, a memory 503 and at least one communication interface 504.

[0083] The processor 501 may be a general-purpose central processing unit (CPU) or an application specific integrated circuit (ASIC).

[0084] The communication bus 502 may be used to transmit information between the above-mentioned components.

[0085] The memory 503 may be a read only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read only memory (EEPROM), a compact disc read only memory (CD ROM) or other optical disc storage, an optical disc storage (including a compressed optical disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 503 may exist independently and be connected to the processor 501 via the communication bus 502. The memory 503 may also be integrated with the processor 501.

[0086] The memory 503 is used to store the program code for executing the solution of the present application, and the execution is controlled by the processor 501. The processor 501 is used to execute the program code stored in the memory 503. The program code may include one or more software modules. The conveying monitoring method for the folding machine in the above embodiment can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0087] The communication interface 504 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0088] In a specific implementation, as an embodiment, a computer device may include multiple processors, each of which may be a single-core (single CPU) processor or a multi-core (multi CPU) processor. The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0089] The above-mentioned computer device may be a general-purpose computer device or a special-purpose computer device. In a specific implementation, the computer device may be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device or an embedded device. The embodiment of the present application does not limit the type of computer device.

[0090] In addition, the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned conveying monitoring method for a box folding machine is implemented.

[0091] In summary, in the conveying monitoring method and system for a folding machine disclosed in the embodiment of the present application, first, two videos of different perspectives are collected by two visual sensors during the conveying process of the carton; the two videos are stereo matched to obtain a depth video stream of the conveyor belt; multiple optical flow information in the depth video stream are extracted, and for each optical flow information, the displacement paths of various locations on the conveyor belt are fitted according to each optical flow information, and all displacement paths are divided into non-linear trajectories in the roller contact area and linear trajectories in the non-roller contact area; the wear amount of the rollers in the conveyor belt is determined according to the curvature radius of each nonlinear trajectory and the inclination angle of the roller; the sag deviation amount of the midpoint of the span between each two linear trajectories is determined, and the tension fluctuation index of the conveyor belt during the conveying process of the carton is determined by all the sag deviation amounts; the deviation probability of the conveyor belt during the conveying process of the carton is determined in combination with the wear amount and the tension fluctuation index, and an alarm signal is sent to a monitoring center when the deviation probability is greater than a preset deviation threshold.

[0092] It can be seen that the present application targets the uneven surface of the conveyor belt and identifies the displacement paths at various locations on the conveyor belt through the depth map of the conveyor belt. It can more accurately extract the motion information at various locations on the conveyor belt, and divide the geometric shape of the displacement path into linear trajectories and nonlinear trajectories according to whether a large change occurs. Subsequently, the aging of the rollers (i.e., the amount of wear on the rollers) is measured through all nonlinear trajectories, and the aging of the conveyor belts (i.e., the tension fluctuation index of the conveyor belts) is measured based on all linear trajectories. The probability of the conveyor belt running off the track is then evaluated through the aging of the rollers and the aging of the conveyor belt. In summary, the present application can provide early warning of abnormal conveyor belt deviation based on the aging of the conveying equipment.

[0093] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0094] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A conveying monitoring method for a box folding machine, characterized in that: include: Two visual sensors are used to collect two videos from different perspectives during the carton conveying process; Performing stereo matching on the two videos to obtain a depth video stream of the conveyor belt; Extracting multiple optical flow information from the depth video stream, fitting the displacement paths of various locations on the conveyor belt according to each optical flow information, and dividing all displacement paths into nonlinear tracks in roller contact areas and linear tracks in non-roller contact areas; Determining the wear amount of the rollers in the conveyor belt according to the curvature radius of each nonlinear trajectory and the forward inclination angle of the rollers; Determine the sag deviation of the midpoint of the span between every two linear tracks, and determine the tension fluctuation index of the conveyor belt during the paper box conveying process through all the sag deviations; The deviation probability of the conveyor belt during the paper box conveying process is determined in combination with the wear amount and the tension fluctuation index, and an alarm signal is sent to a monitoring center when the deviation probability is greater than a preset deviation threshold.

2. The method according to claim 1, characterized in that Performing stereo matching on the two videos to obtain a depth video stream of the conveyor belt specifically includes: Performing frame synchronization on the two channels of video to obtain an image group for each timestamp; Extracting the region of interest from all images in each image group to obtain a conveyor belt image group of the conveyor belt area; Perform stereo matching on the conveyor belt image group at each timestamp to obtain a depth map of the conveyor belt at each timestamp; Reconstruct all depth maps into a conveyor belt depth video stream.

3. The method according to claim 1, characterized in that Fitting the displacement paths of various locations on the conveyor belt according to each optical flow information specifically includes: Dividing a plurality of monitoring areas on the conveyor belt according to all depth images in the depth video stream; All optical flow information is filtered according to all monitoring areas to obtain the path optical flow set in each monitoring area; The optical flow set of each path is fitted separately to obtain the displacement path of each location on the conveyor belt.

4. The method according to claim 1, characterized in that Determining the wear amount of the roller in the conveyor belt according to the curvature radius of each nonlinear trajectory and the forward inclination angle of the roller specifically includes: determining a plurality of radii of curvature for each nonlinear trajectory; According to the changing characteristics of all the curvature radii on the conveyor belt surface, multiple local deformation features of the conveyor belt surface are identified; Fit the elasticity of the conveyor belt according to all local deformation characteristics to obtain the elastic coefficient of the conveyor belt; The wear amount of the roller in the conveyor belt is determined according to the elastic coefficient and the forward inclination angle of the roller.

5. The method according to claim 1, characterized in that Determining the sag deviation of the mid-span between each two linear tracks specifically includes: Determine the midpoint of the span between every two linear tracks; Determine the sag from the midpoint of each span to each linear track on either side of the midpoint of each span; The sag deviation amount of the mid-span point between each two linear tracks is determined based on all sags at each mid-span point.

6. The method according to claim 1, characterized in that The tension fluctuation index of the conveyor belt during the carton conveying process is determined by all sag deviations, specifically including: Determine a correlation sequence of sag deviation changes based on all sag deviations; Determining a cutoff point of the correlation sequence; The tension fluctuation index of the conveyor belt during the paper box conveying process is determined according to the cutoff point.

7. The method according to claim 1, characterized in that Determining the deviation probability of the conveyor belt during the paper box conveying process in combination with the wear amount and the tension fluctuation index specifically includes: Determining a deviation factor of the conveyor belt during the paper box conveying process according to the wear amount and the tension fluctuation index; The probability of deviation of the conveyor belt during the paper box conveying process is determined according to the offset factor.

8. A conveying monitoring system for a box folding machine, characterized in that: include: The acquisition module is used to collect two videos from different perspectives during the carton conveying process through two visual sensors; A processing module, used for performing stereo matching on the two videos to obtain a depth video stream of the conveyor belt; The processing module is further used to extract multiple optical flow information from the depth video stream, and for each optical flow information, fit the displacement path of each location on the conveyor belt according to each optical flow information, and divide all displacement paths into nonlinear tracks in the roller contact area and linear tracks in the non-roller contact area; The processing module is also used to determine the wear amount of the rollers in the conveyor belt according to the curvature radius of each nonlinear track and the forward inclination angle of the rollers; The processing module is also used to determine the sag deviation of the midpoint of the span between every two linear tracks, and determine the tension fluctuation index of the conveyor belt during the paper box conveying process through all the sag deviations; The execution module is used to determine the deviation probability of the conveyor belt during the paper box conveying process by combining the wear amount and the tension fluctuation index, and send an alarm signal to a monitoring center when the deviation probability is greater than a preset deviation threshold.

9. A computer device, characterized in that: The computer device comprises a memory and a processor, wherein the memory stores codes, and the processor is configured to obtain the codes and execute the conveying monitoring method for a box folding machine according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the conveying monitoring method for a box folding machine according to any one of claims 1 to 7 is implemented.

Images