An online safety monitoring system for crane operations

Abstract

The present application relates to the field of engineering technology and proposes an online safety monitoring system for crane operations, including: obtaining point cloud image data; obtaining a ground roughness index based on the point cloud image data; obtaining a wheel support force uniformity index based on the ground roughness index; obtaining a crane boom extension suitability based on the wheel support force uniformity index; obtaining an optimal extension angle and an optimal extension boom length based on the crane boom extension suitability; obtaining a high-altitude operation hazard index based on the optimal extension angle and the optimal extension boom length; obtaining the stability of key-point operations based on the high-altitude operation hazard index; using an autoregressive moving average model to obtain a predicted value of the stability of key-point operations based on the stability of key-point operations, and alarm the high-altitude operation personnel online based on the predicted value of the stability of key-point operations. The present application alarms the high-altitude operation personnel online based on the prediction result of the stability of key-point operations, improving the accuracy and efficiency of online safety monitoring.

Description

Technical Field

[0001] This application relates to the field of engineering technology, and particularly to an on-line safety monitoring system for crane operations. Background Art

[0002] With the rapid development of infrastructure engineering construction, the on-site engineering construction operations tend to be large-scale, complex and modular. Cranes are the main key equipment for material lifting. During the crane engineering operation process, the lifting height and tonnage are continuously increasing, and the overall safety operation management difficulty is also continuously increasing, and major safety accidents are extremely likely to occur. Therefore, in order to ensure the safety and reliability of the crane during the hoisting operation, it is necessary to dynamically monitor the working state of the crane engineering operation.

[0003] In the prior art, various sensors such as position sensors and pressure sensors and mechanical models are mostly used to analyze the working range of the crane, but this method is slow and inefficient, and is more used for simulation analysis, and has little effect in practical applications. With the further development of optical remote sensing technology, active sensors represented by lidar and depth cameras can be very conveniently installed on the boom, so as to quickly obtain the working state of the crane. Summary of the Invention

[0004] This application provides an on-line safety monitoring system for crane operations to solve the problem that the monitoring accuracy rate and the recognition efficiency cannot be coordinated and unified. The specific technical solutions adopted are as follows:

[0005] An on-line safety monitoring system for crane operations according to an embodiment of this application, the system includes the following modules:

[0006] A working data acquisition module, which acquires denoised point cloud image data;

[0007] An optimal working state acquisition module, which obtains each crane working window according to the point cloud image data, and obtains the ground ruggedness index of the important point cloud data point set of each sampling of each crane working window according to each crane working window; obtains the wheel support force uniformity index of each crane working window according to the ground ruggedness index of all samplings of each crane working window; obtains the boom extension suitability of each crane working window according to the wheel support force uniformity index of each crane working window, and obtains the optimal working area of the crane according to the boom extension suitability of each crane working window; obtains the optimal extension angle and the optimal extension arm length of the crane manipulator according to the optimal working area of the crane;

[0008] The key-point operation analysis module obtains the actual extension angle and the actual extension arm length according to the actual working area of the crane; obtains the high-altitude operation danger index of the crane during actual operation according to the optimal extension angle, the optimal extension arm length, the actual extension angle and the actual extension arm length of the crane's robotic arm; obtains the key-point operation stability of each actual working moment of the crane according to the high-altitude operation danger index of the crane during actual operation.

[0009] The warning module uses the autoregressive moving average model to obtain the predicted value of the key-point operation stability based on the key-point operation stability, and alarms the high-altitude operation personnel online based on the predicted value of the key-point operation stability.

[0010] Preferably, the method for obtaining the ground ruggedness index of the important point cloud data point set for each sampling of each crane working window according to the point cloud image data is as follows:

[0011] Set a sliding window equal in size to the rectangular working area of the crane, and slide the sliding window in the point cloud image in the order from left to right and from top to bottom without overlap. Take the result of each slide of the sliding window as a crane working window.

[0012] For each crane working window, sample the first preset parameter times in the corresponding area of the crane working window according to the random sampling principle, and sample the second preset parameter of point cloud data points each time. The set composed of all the point cloud data points extracted each time is used as the important point cloud data point set extracted each time.

[0013] For any two point cloud data points in the important point cloud data point set extracted each time, take the reciprocal of the sum of 1 and the metric distance between the coordinates of the two point cloud data points projected onto the horizontal ground in the vertical direction as the wheel support force influence weight between the two point cloud data points.

[0014] Calculate the absolute value of the difference in height between the two point cloud data points from the horizontal ground, and take the mean value of the quadratic cumulative sum of the product of the absolute value and the wheel support force influence weight on the important point cloud data point set as the ground ruggedness index of the important point cloud data point set.

[0015] Preferably, the method for obtaining the wheel support force uniformity index of each crane working window according to the ground ruggedness index of the important point cloud data point sets for all samplings of each crane working window is as follows:

[0016] For each crane working window, take the sequence composed of the ground ruggedness indexes of the important point cloud data point sets for all samplings of the crane working window in the order of sampling as the ruggedness index sequence of the crane working window.

[0017] Taking the difference between the value of each data point in the ruggedness index sequence except the first data point and the value of the previous data point as the numerator, and taking the maximum value between the value of each data point in the ruggedness index sequence except the first data point and the value of the previous data point as the denominator, calculate the normalized result of the ratio of the numerator to the denominator;

[0018] Calculate the absolute value of the difference between the normalized result and 0.5, calculate the reciprocal of the sum of the absolute value and 1, and take the cumulative sum of the reciprocal on the ruggedness index sequence as the wheel support force uniformity index of the crane working window.

[0019] Preferably, the method for obtaining the boom extension suitability of each crane working window based on the wheel support force uniformity index of each crane working window and obtaining the optimal working area of the crane based on the boom extension suitability of each crane working window is as follows:

[0020] Taking the Euclidean distance between the projection points of the inflection point of the crane's robotic arm and the construction target point on the ground when the crane is in each crane working window as the working horizontal distance of the crane working window;

[0021] Taking the ratio between the angle between the robotic arm and the horizontal plane when the crane is working in each crane working window and the preset angle value as the working matching rate of the crane working window;

[0022] For each crane working window, calculate the negative mapping result with the natural constant as the base and the working horizontal distance as the exponent, and take the product of the negative mapping result, the working matching rate, and the wheel support force uniformity index as the boom extension suitability of the crane working window;

[0023] Taking the crane working window corresponding to the maximum value of the boom extension suitability of all crane working windows as the optimal working area of the crane.

[0024] Preferably, the method for obtaining the optimal extension angle and optimal extension length of the crane's robotic arm based on the optimal working area of the crane is as follows:

[0025] Based on the optimal working area of the crane, respectively collect the working horizontal distance of the crane in the optimal working area, the inflection point height of the crane's robotic arm inflection point from the horizontal ground, and the working height between the target construction point and the horizontal ground;

[0026] Taking the ratio of the difference between the working height and the inflection point height to the working horizontal distance as the mapping object, and taking the result of the inverse tangent mapping of the mapping object as the optimal extension angle of the crane's robotic arm;

[0027] Take the square root of the sum of the square of the mapped object and the square of the working horizontal distance as the optimal extended arm length of the crane's robotic arm.

[0028] Preferably, the method for obtaining the actual extended angle and the actual extended arm length according to the actual working area of the crane is as follows:

[0029] Obtain the preprocessed image of the crane's actual work, use the Otsu threshold segmentation technique to segment the crane from the preprocessed image of the crane's work, and use the canny edge detection algorithm to identify the edge image of the crane;

[0030] Count the number of pixel points in each direction in the edge image of the crane, take the direction with the largest number of pixel points as the actual robotic arm extension direction, take the included angle between the actual robotic arm extension direction and the horizontal direction as the actual extended angle, and take the length of the robotic arm in the actual robotic arm extension direction of the crane as the actual extended arm length.

[0031] Preferably, the method for obtaining the high-altitude operation risk index of the crane during actual work according to the optimal extended angle, the optimal extended arm length, the actual extended angle, and the actual extended arm length of the crane's robotic arm is as follows:

[0032] Calculate the absolute value of the difference between the actual extended angle and the optimal extended angle, and take the ratio of the absolute value to the preset angle value as the first ratio parameter; obtain the second ratio parameter according to the actual extended arm length and the optimal extended arm length;

[0033] Take the product of the first ratio parameter and the second ratio parameter as the high-altitude operation risk index of the crane during actual work.

[0034] Preferably, the method for obtaining the second ratio parameter according to the actual extended arm length and the optimal extended arm length is as follows:

[0035] Calculate the negative mapping result with the natural constant as the base and the difference between 1 and the ratio of the optimal extended arm length to the actual extended arm length as the exponent;

[0036] Calculate the sum of the wheel support force uniformity index and the crane arm span suitability when the crane is working in the actual working area; calculate the product of the negative mapping result and the sum;

[0037] Take the reciprocal of the sum of the product and 1 as the second ratio parameter.

[0038] Preferably, the method for obtaining the key point operation stability of each actual working moment of the crane according to the high-altitude operation risk index of the crane during actual work is as follows:

[0039] When the crane is actually working, sample the actual working moments of the crane at the third preset parameter, and calculate the high-altitude operation danger index at each actual working moment of the crane;

[0040] Take the fourth preset parameter of actual working moments with the closest time intervals to each actual working moment of the crane as each adjacent actual working moment of each actual working moment of the crane;

[0041] For each actual working moment of the crane, calculate the absolute value of the difference between the high-altitude operation danger index of each adjacent actual working moment except the first adjacent actual working moment and the previous adjacent actual working moment, and calculate the product of the absolute value and the high-altitude operation danger index of the actual working moment;

[0042] Take the cumulative sum of the reciprocals of the sum of the product and 1 over all the adjacent actual working moments as the key-point operation stability of the actual working moment.

[0043] Preferably, the method for obtaining the predicted value of the key-point operation stability based on the key-point operation stability using the autoregressive moving average model and performing online alarm for high-altitude operation personnel based on the predicted value of the key-point operation stability is as follows:

[0044] Take the key-point operation stability of all actual working moments of the crane as the input of the Otsu threshold segmentation algorithm, take the output of the Otsu threshold segmentation algorithm as the segmentation threshold of the key-point operation stability of all actual working moments of the crane, and take the segmentation threshold of the key-point operation stability of all actual working moments of the crane as the safety threshold for the crane to work;

[0045] Take the key-point operation stability of all actual working moments of the crane as the input of the autoregressive moving average model, and take the output of the autoregressive moving average model as the predicted value of the key-point operation stability of the current actual working moment of the crane;

[0046] When the predicted value of the key-point operation stability of the current actual working moment of the crane is greater than the safety threshold for the crane to work, the crane can continue to work at this time; when the predicted value of the key-point operation stability of the current actual working moment of the crane is less than or equal to the safety threshold for the crane to work, an alarm is issued through the warning module, and the crane is not suitable to continue working at this time.

[0047] The beneficial effects of this application are as follows: By analyzing the point cloud data within the workable area of the crane, this application constructs a ground ruggedness index to reflect the ruggedness of the ground within the working window of the crane. Based on this, a wheel support force uniformity index is constructed by combining the ground ruggedness indices corresponding to the point cloud data obtained through multiple samplings, which reflects the uniformity of the support force provided by the ground to the wheels within the working window. Based on this, a crane boom extension suitability is constructed by combining the height of the construction target point from the ground and the horizontal distance from the crane, thereby determining the optimal working area of the crane, and obtaining the optimal extension angle and optimal extension length of the crane's robotic arm. By identifying the actual extension angle and actual extension length of the crane's robotic arm in the image of the crane's working process, a high-altitude operation hazard index is constructed accordingly. Using the high-altitude operation hazard index, a key point working stability is constructed to reflect the difference between the actual working state and the optimal working state of the crane, and the obtained key point working stability is uploaded to the crane control system, thereby realizing online monitoring of the key point operations of the crane. By considering the changes in ground ruggedness and the uniformity of the support force provided by different wheels during the operation of the crane, this application constructs the stability of key point operations, and then uses the autoregressive moving average model to obtain the predicted value of the stability of key point operations based on the stability of key point operations, so as to achieve more accurate warning of high-altitude operators and improve the accuracy and efficiency of online safety detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0049] Figure 1 It is a schematic flow chart of a crane operation safety online monitoring system provided by an embodiment of this application;

[0050] Figure 2 It is an implementation flow chart of a crane operation safety online monitoring system provided by an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] The following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the drawings in the embodiments of this application. Obviously, the described embodiments are only some embodiments of this application, rather than all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application.

[0052] Please refer to Figure 1, which shows a flowchart of a crane operation safety online monitoring system provided by an embodiment of the present application. The system includes a working data acquisition module, an optimal working state acquisition module, a key point operation analysis module, and a warning module.

[0053] The working data acquisition module uses a camera to obtain image data to be processed, preprocesses the image data, and completes the acquisition of the image data.

[0054] The point cloud image of the working area of the crane at the construction site is collected by a depth camera, and the image of the crane during operation is collected by a high-definition camera. Since the construction site environment is complex, the collected images may have significant noise. To avoid the large impact of image noise on subsequent analysis, the present application performs denoising processing on the collected images. Traditional denoising methods include mean filtering, bilateral filtering, Gaussian filtering, etc. To retain the edge detail information in the image to a greater extent, the present application uses bilateral filtering denoising technology to denoise the collected images, and obtains the preprocessed images corresponding to each original image. Among them, the bilateral filtering denoising technology is a well-known technology, and the specific process is not described in detail in the present application.

[0055] So far, the point cloud image of the workable area of the crane at the construction site and the image of the crane during operation are obtained. It should be noted that the point cloud image of the workable area refers to the point cloud image of the workable area of the crane on the ground, and the image of the crane during operation is a color image in the RGB space.

[0056] The optimal working state acquisition module obtains the ground roughness index according to the point cloud image data, obtains the wheel support force uniformity index according to the ground roughness index, obtains the crane boom extension suitability according to the wheel support force uniformity index, and obtains the optimal extension angle and the optimal extension boom length according to the crane boom extension suitability.

[0057] When the crane is performing high-altitude operations, the ground in its working area needs to be relatively flat so that the four wheels of the crane can be evenly stressed and provide a stable support force for the crane to ensure the safety of high-altitude operations. Based on this, the present application constructs a wheel support force uniformity index to reflect the stability of the four wheels of the crane providing support force for the crane.

[0058] The preprocessed point cloud image of the workable area of the crane is denoted as the preprocessed point cloud image A of the workable area. Since the working area of the crane is rectangular, a sliding window of the same size as the rectangular working area of the crane is set. The sliding window slides in the point cloud image A in the order from left to right and from top to bottom to obtain each crane working window.

[0059] According to the point cloud data within each crane working window, since the point cloud data points are in space, when the ground flatness is poor, the distribution difference of the Euclidean space distances between the point cloud data points is relatively large. Here, a small amount of point cloud data points in the crane working window can reflect the overall characteristics. Based on the point cloud data within each crane working window area, the point cloud data in the crane working window is sampled K times according to the random sampling principle, and the number of point clouds for each sampling is N. The set of N point cloud data points for each sampling is used as the important point cloud data point set for each sampling. In this application, the empirical value of N is taken as 100, and the empirical value of K is taken as 50.

[0060] Based on the above analysis, calculate the ground ruggedness index of the important point cloud data point set during each sampling:

[0061]

[0062]

[0063] In the formula, represents the wheel support force influence weight between the a-th point cloud data point and the c-th point cloud data point in the important point cloud data point set during the p-th sampling, represents the Euclidean distance function, 、 respectively represent the coordinates of the a-th and c-th point cloud data points in the important point cloud data point set during the p-th sampling projected onto the horizontal ground in the vertical direction, represents the Euclidean distance between the coordinates of the a-th and c-th point cloud data points in the important point cloud data point set during the p-th sampling projected onto the horizontal ground in the vertical direction, is the error parameter to avoid the denominator from taking a value of 0, and the empirical value of the error parameter is 1;

[0064] represents the ground ruggedness index of the important point cloud data point set during the p-th sampling, N represents the number of point cloud data points in the important point cloud data point set during the p-th sampling, and the empirical value of N is taken as 100 in this application, 、 respectively represent the heights of the a-th and c-th point cloud data points in the important point cloud data point set during the p-th sampling from the horizontal ground.

[0065] In the important point cloud data point set during the p-th sampling, if the Euclidean distance between the projections of two point cloud data points onto the two-dimensional horizontal ground is smaller, that is The smaller it is, the closer the two point cloud data points are on the horizontal plane. Since the horizontal distance between the two point cloud data points is closer, the greater the influence of height on the wheel support force at this time, so the weight when calculating the ground roughness index is greater. Therefore, the calculated influence weight of the wheel support force is greater. At the same time, if the difference between the heights of the two point cloud data points from the horizontal ground is greater, that is The larger it is, the greater the degree of fluctuation of the two point cloud data points in height, that is, the greater the roughness of the ground. Therefore, the calculated ground roughness index is larger.

[0066] Based on the above analysis, the same processing is performed on the obtained important point cloud data point sets during each sampling to obtain the ground roughness index of the important point cloud data point sets for each sampling. If the ground within the crane working window is relatively flat, enabling the four wheels of the crane to provide stable and uniform support for the crane, then the values of the ground roughness index calculated for each sampling should be smaller.

[0067] Furthermore, in order to reflect the working conditions of the crane in each crane working window area, further analysis is carried out. Usually, a small number of point cloud data points in the crane working window can reflect the overall characteristics. However, when the overall complexity is high, the randomness of the results of a single sampling is large, and it cannot accurately reflect the working conditions of the crane in each crane working window area.

[0068] Specifically, in order to more accurately reflect the working conditions of the crane in each crane working window area, for each crane working window, the sequence formed by arranging the ground roughness indices of all sub-sampled important point cloud data point sets of the crane working window in the order of sampling is used as the roughness index sequence of the crane working window.

[0069] Based on the above analysis, calculate the wheel support force uniformity index for each crane working window to reflect the working conditions of the crane in each crane working window area:

[0070]

[0071] In the formula, represents the wheel support force uniformity index of the d-th crane working window in the preprocessed point cloud image A of the workable area of the crane. U represents the number of data points in the roughness index sequence of the d-th crane working window in the preprocessed point cloud image A of the workable area of the crane. represents the Sigmoid normalization function. 、 respectively represent the numerical values of the q-th and (q - 1)-th data points in the roughness index sequence of the d-th crane working window in the preprocessed point cloud image A of the workable area of the crane. represents the maximum value function. denotes the identification factor, which is used to centralize the Sigmoid function. In this application, the empirical value 0.5 is taken. denotes the error parameter, which is used to avoid the situation where the denominator is zero and makes the calculation impossible. In this application, the empirical value 1 is taken.

[0072] If the difference between the ground ruggedness indices of two important point cloud data point sets is large, that is the larger it is, it indicates that in the working window of this crane, the randomly selected point cloud data points are different, and the ground ruggedness is also different. That is, in the working window of the crane, the stability of the supporting force provided by the ground at different positions to the crane is different, indicating that the overall working window of this crane cannot provide a stable supporting force to the wheels of the crane. For example, if one side of the working window of the crane is high and the other side is low, when the crane works here, dangerous situations such as imbalance may occur. Therefore, the smaller the calculated wheel support force uniformity index is.

[0073] Furthermore, based on the wheel support force uniformity index, the crane boom extension suitability is constructed to determine the optimal working area of the crane.

[0074] Specifically, the Euclidean distance between the inflection point of the crane's robotic arm and the projection point of the construction target point on the ground is used as the working horizontal distance from the crane's working window to the construction target point, and the ratio between the angle between the robotic arm and the horizontal plane when the crane works in each crane working window and the preset angle value is used as the working matching rate. The preset angle value is 90 degrees.

[0075] When the crane operates in different working windows, the working states such as the extension arm length and angle of its robotic arm are different. In different working states, the reasonable degree of the force on the robotic arm is different, and the corresponding danger levels are also different. Based on this, the crane boom extension suitability can be constructed to reflect the degree to which each crane working window is suitable as the working area of the crane:

[0076]

[0077] In the formula, denotes the crane boom extension suitability of the i-th crane working window in the workable area, denotes the working matching rate of the i-th crane working window in the workable area, denotes the working horizontal distance between the i-th crane working window and the construction target point, denotes the wheel support force uniformity index of the i-th crane working window.

[0078] The working horizontal distance between the crane and the construction target point when the crane works in the i-th working window The smaller it is, it indicates that when the aerial working personnel stand on the crane platform structure for aerial work, the angle between the extension and inclination direction of the robotic arm and the ground is closer to 90 degrees. Then, the component of the robotic arm in the direction perpendicular to the ground is larger, the corresponding stability is stronger, and the suitability of the crane arm span is greater. At the same time, the working matching rate of the i-th crane working window in the workable area of the crane The larger it is, it indicates that the length of the robotic arm that the crane needs to extend during construction is shorter, the support force provided by the crane to the robotic arm is larger, the stability is stronger, and the suitability of the crane arm span is greater. The wheel support force uniformity index of the crane in the i-th working window The larger it is, it indicates that when the crane is working in this window, the chassis can provide a uniform and stable support force to the crane, that is, the stability of the crane is stronger, and this crane working window is more suitable as the working area of the crane, and the suitability of the crane arm span is greater.

[0079] Furthermore, based on the optimal working area of the crane and combined with the location of the target construction site, the optimal working state of the crane robotic arm is predicted. Based on the above analysis, the suitability of the crane arm span for each working window in the workable area of the obtained crane is used, and the working window with the highest suitability of the crane arm span is used as the optimal working area of the crane.

[0080] Specifically, according to the working horizontal distance and working height between the target construction site and the optimal working area of the crane, the optimal working state of the crane robotic arm is analyzed, that is, the optimal extension angle and optimal extension arm length of the crane robotic arm. The analysis process is as follows:

[0081] When the crane is working on the optimal working window, the height between the construction target point and the ground is used as the working height H, the height between the inflection point of the robotic arm and the ground is used as the inflection point height L, and the horizontal distance between the inflection point of the robotic arm and the construction target point is used as the working horizontal distance .

[0082] Calculate the optimal extension angle of the robotic arm according to the Pythagorean theorem and the optimal extension arm length , and its calculation formula is as follows:

[0083]

[0084]

[0085] In the formula, represents the optimal extension angle of the robotic arm when the crane is working on the optimal working window, represents the working height between the construction target point and the ground, represents the working horizontal distance between the optimal working window and the construction target point, Denotes the inflection point height between the inflection point of the robotic arm and the ground. Denotes the optimal extended arm length of the robotic arm when the crane is operating within the optimal working window.

[0086] Thus, obtaining the optimal extended angle and the optimal extended arm length of the crane's robotic arm in the optimal working area characterizes the optimal working state of the crane.

[0087] The key point operation analysis module obtains the high-altitude operation danger index based on the optimal extended angle and the optimal extended arm length, and obtains the stability of the key point operation based on the high-altitude operation danger index.

[0088] Furthermore, based on the obtained optimal working state of the crane's robotic arm, by comparing it with the actual working state of the crane, a high-altitude operation danger index is constructed.

[0089] Since the color of the crane is usually mainly engineering yellow, its color is relatively conspicuous both during the day and at night. Therefore, it is relatively easy to identify the crane in the image through the machine vision algorithm. Accordingly, in this application, an image of the crane during operation is collected by a high-definition camera, that is, the obtained preprocessed image of the crane's actual operation. The Otsu threshold segmentation technique is used to segment the crane from the preprocessed image of the crane's actual operation, and the canny edge detection algorithm is used to identify the edge image of the crane. According to the edge image of the crane, the sum of the number of pixel points in each direction is statistically calculated. The direction with the largest sum of the number of pixel points is used as the actual extended direction of the robotic arm. The angle between the actual extended direction of the crane's robotic arm and the horizontal direction is used as the actual extended angle, and the length of the robotic arm in the actual extended direction of the crane's robotic arm is used as the actual extended arm length. Among them, the Otsu threshold segmentation technique and the canny edge detection algorithm are both well-known techniques and will not be elaborated further. It should be noted that the sum of the number of pixel points in each direction is statistically calculated, where each direction is within the range of 0 degrees to 180 degrees, with a step of 10 degrees, for a total of 19 directions.

[0090] Based on the actual and theoretical optimal extended angles and the optimal extended arm lengths of the robotic arm obtained in the above steps, combined with the wheel support force uniformity index and the crane arm extension suitability, the high-altitude operation danger index of the crane during actual operation can be constructed. The construction process is as follows:

[0091]

[0092] In the formula, Denotes the high-altitude operation danger index of the crane during actual operation. Denotes the optimal extended angle of the crane's robotic arm. Denotes the optimal extended arm length of the crane's robotic arm. Denotes the actual extended angle of the crane's robotic arm. represents the actual extended arm length of the crane's robotic arm, represents a preset angle value, represents an error parameter, 、 respectively represent the wheel support force uniformity index and the crane arm extension suitability when the crane is working in the optimal working area. It should be noted that the preset angle value is 90 degrees and the error parameter is 1.

[0093] During the actual operation of the crane, the greater the difference between the actual extended angle of the robotic arm and the optimal extended angle, and the greater the difference between the actual extended arm length of the robotic arm and the optimal extended arm length, that is, the greater, the smaller, it indicates that the working stability of the robotic arm is worse, the corresponding working state is more dangerous, and the calculated high-altitude operation danger index is greater; the sum of the wheel support force uniformity index and the crane arm extension suitability the smaller, to a certain extent, it shows that the uniformity of the wheel support force and the suitability of the crane arm extension are smaller, that is, the higher the danger level of the high-altitude operation at this time, and the greater the high-altitude operation danger index

[0094] By collecting the working images of the crane at X actual working moments in the above manner, X takes the empirical value of 100 in this application, and the collection time interval is denoted as Y, and Y takes the empirical value of 10s in this application. Through the above calculation method, the high-altitude operation danger index of each actual working moment of the crane can be obtained. In addition, the T actual working moments with the closest time interval to each actual working moment are used as the adjacent working moments of each actual working moment respectively, where the empirical value of T is 10.

[0095] Furthermore, calculate the key point operation stability of each actual working moment of the crane:

[0096]

[0097] In the formula, represents the key point operation stability of the crane at the t-th actual working moment, T represents the number of adjacent actual working moments of the crane at the t-th actual working moment, and the empirical value of T is 10, represents the high-altitude operation danger index of the crane at the t-th actual working moment, 、 respectively represent the high-altitude operation danger indices of the j-th and (j - 1)-th adjacent actual working moments of the crane at the t-th actual working moment, represents the error parameter.

[0098] High-altitude operation danger index the smaller, and at the same time the difference between the high-altitude operation danger indices of adjacent actual working moments is smaller, that is, The smaller it is, the higher the stability degree of the crane during operation at this time, the higher the safety of the key point selection, that is, the higher the safety of the crane during operation at this time. Therefore, the calculated working stability degree of the key point is greater.

[0099] Thus, the working stability degree of the key point at each actual working moment of the crane is obtained.

[0100] The warning module uses the autoregressive moving average model to obtain the predicted value of the working stability degree of the key point based on the working stability degree of the key point, and alarms the high-altitude operators online based on the predicted value of the working stability degree of the key point.

[0101] Furthermore, the working stability degrees of the key points at all actual working moments of the crane are used as the input of the Otsu threshold segmentation algorithm, the output of the Otsu threshold segmentation algorithm is used as the segmentation threshold of the working stability degrees of the key points at all actual working moments of the crane, and the segmentation threshold of the working stability degrees of the key points at all actual working moments of the crane is used as the safety threshold for the crane to work. The Otsu threshold segmentation algorithm is a well-known technology, and the specific process will not be elaborated here. The implementation flowchart of this application is as Figure 2 shown.

[0102] The working stability degrees of the key points at all actual working moments of the crane are used as the input of the autoregressive moving average model, that is, the working stability degrees of the key points at the previous X actual working moments of the crane are used as the input, and the output of the autoregressive moving average model is used as the predicted value of the working stability degree of the key point at the (X + 1)-th actual working moment of the crane, that is, the predicted value of the working stability degree of the key point at the current actual working moment. The autoregressive moving average model is a well-known technology, and the specific process will not be elaborated here.

[0103] The predicted value of the working stability degree of the key point at the current actual working moment of the crane is transmitted to the warning module. When the predicted value of the working stability degree of the key point at the current actual working moment of the crane is greater than the safety threshold for the crane to work, it indicates that the stability degree of the crane during operation at this time is relatively high, and the crane can continue to work; when the predicted value of the working stability degree of the key point at the current actual working moment of the crane is less than or equal to the safety threshold for the crane to work, the warning module alarms, indicating that the stability degree of the crane during operation at this time is relatively low, and the crane is not suitable to continue working. At this time, the working position of the crane should be adjusted to a safe working position and then continue to work.

[0104] Thus, a safety online monitoring system for crane operation is completed.

[0105] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. The above description is only the preferred embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the principle of the present application shall be included in the protection scope of the present application.

Claims

1. An on-line safety monitoring system for crane operation, characterized in that, The system includes the following modules: A working data acquisition module, which acquires the denoised point cloud image data; An optimal working state acquisition module, which acquires each crane working window according to the point cloud image data, and acquires the ground roughness index of the important point cloud data point set of each sampling in each crane working window according to each crane working window; acquires the wheel support force uniformity index of each crane working window according to the ground roughness indices of all samplings of the important point cloud data point set of each crane working window; acquires the crane boom extension suitability of each crane working window according to the wheel support force uniformity index of each crane working window, and acquires the optimal working area of the crane according to the crane boom extension suitability of each crane working window; acquires the optimal extension angle and optimal extension arm length of the crane arm according to the optimal working area of the crane; A key point operation analysis module, which acquires the actual extension angle and actual extension arm length according to the actual working area of the crane; acquires the high-altitude operation danger index of the crane during actual operation according to the optimal extension angle, optimal extension arm length, actual extension angle and actual extension arm length of the crane arm; acquires the key point operation stability of each actual working moment of the crane according to the high-altitude operation danger index of the crane during actual operation; A warning module, which uses an autoregressive moving average model to obtain a predicted value of the key point operation stability based on the key point operation stability, and alarms the high-altitude operation personnel online based on the predicted value of the key point operation stability.

2. The online safety monitoring system for crane operation according to claim 1, wherein The method for acquiring each crane working window according to the point cloud image data and acquiring the ground roughness index of the important point cloud data point set of each sampling in each crane working window is as follows: Set a sliding window with the same size as the rectangular working area of the crane, and slide the sliding window in the point cloud image in the order from left to right and from top to bottom without overlap, and take the result of each sliding of the sliding window as a crane working window; For each crane working window, sample the first preset parameter times in the corresponding area of the crane working window according to the random sampling principle, and sample the second preset parameter of point cloud data points each time, and take the set composed of all the point cloud data points extracted each time as the important point cloud data point set of each extraction; For any two point cloud data points in the important point cloud data point set extracted each time, take the reciprocal of the sum of 1 and the metric distance between the coordinates of the two points projected onto the horizontal ground in the vertical direction as the wheel support force influence weight between the two point cloud data points; Calculate the absolute value of the height difference between the two point cloud data points from the horizontal ground, and take the mean value of the quadratic cumulative sum of the product of the absolute value and the wheel support force influence weight on the important point cloud data point set as the ground roughness index of the important point cloud data point set.

3. The on-line safety monitoring system for crane operation according to claim 1, characterized in that The method for acquiring the wheel support force uniformity index of each crane working window according to the ground roughness indices of all samplings of the important point cloud data point set of each crane working window is as follows: For each crane working window, the sequence formed by arranging the ground roughness indices of all subsampled important point cloud data points of the crane working window in the order of sampling is used as the roughness index sequence of the crane working window; Taking the difference between the value of each data point in the roughness index sequence except the first data point and the value of the previous data point as the numerator, and taking the maximum value between the value of each data point in the roughness index sequence except the first data point and the value of the previous data point as the denominator, calculate the normalized result of the ratio of the numerator to the denominator; Calculate the absolute value of the difference between the normalized result and 0.5, calculate the reciprocal of the sum of the absolute value and 1, and take the cumulative sum of the reciprocal on the roughness index sequence as the wheel support force uniformity index of the crane working window.

4. The on-line safety monitoring system for crane operation according to claim 1, characterized in that, The method for obtaining the crane boom extension suitability of each crane working window based on the wheel support force uniformity index of each crane working window and obtaining the optimal working area of the crane according to the crane boom extension suitability of each crane working window is as follows: Taking the Euclidean distance between the projection points on the ground of the inflection point of the crane robotic arm and the construction target point when the crane is in each crane working window as the working horizontal distance of the crane working window; Taking the ratio between the angle between the robotic arm and the horizontal plane when the crane is working in each crane working window and the preset angle value as the working matching rate of the crane working window; For each crane working window, calculate the negative mapping result with the natural constant as the base and the working horizontal distance as the exponent, and take the product of the negative mapping result, the working matching rate, and the wheel support force uniformity index as the crane boom extension suitability of the crane working window; Taking the crane working window corresponding to the maximum value of the crane boom extension suitability of all crane working windows as the optimal working area of the crane.

5. The on-line safety monitoring system for crane operation according to claim 1, characterized in that The method for obtaining the optimal extension angle and optimal extension arm length of the crane robotic arm according to the optimal working area of the crane is as follows: Based on the optimal working area of the crane, respectively collect the working horizontal distance of the crane in the optimal working area, the inflection point height of the crane robotic arm inflection point from the horizontal ground, and the working height between the target construction point and the horizontal ground; Taking the ratio of the difference between the working height and the inflection point height to the working horizontal distance as the mapping object, and taking the result of the arctangent mapping of the mapping object as the optimal extension angle of the crane robotic arm; Taking the square root of the sum of the square of the mapping object and the square of the working horizontal distance as the optimal extension arm length of the crane robotic arm.

6. The online safety monitoring system for crane operation according to claim 1, characterized in that, The method for obtaining the actual extension angle and actual extension arm length according to the actual working area of the crane is as follows: Obtain the preprocessed image of the crane's actual work, use the Otsu threshold segmentation technique to segment the crane from the preprocessed image of the crane's work, and use the canny edge detection algorithm to identify the edge image of the crane; Count the number of pixel points in each direction of the edge image of the crane, take the direction with the largest number of pixel points as the actual mechanical arm extension direction, take the angle between the actual mechanical arm extension direction and the horizontal direction as the actual extension angle, and take the length of the mechanical arm of the crane in the actual mechanical arm extension direction as the actual extension arm length.

7. The on-line safety monitoring system for crane operation according to claim 1, characterized in that The method for obtaining the high-altitude operation danger index of the crane during actual operation according to the optimal extension angle, optimal extension arm length, actual extension angle and actual extension arm length of the crane mechanical arm is as follows: Calculate the absolute value of the difference between the actual extension angle and the optimal extension angle, and take the ratio of the absolute value to the preset angle value as the first ratio parameter; obtain the second ratio parameter according to the actual extended arm length and the optimal extended arm length; Take the product of the first ratio parameter and the second ratio parameter as the high-altitude operation danger index of the crane during actual operation.

8. An on-line safety monitoring system for crane operation according to claim 7, characterized in that, The method for obtaining the second ratio parameter according to the actual extended arm length and the optimal extended arm length is as follows: Calculate the negative mapping result with the natural constant as the base and the difference between 1 and the ratio of the optimal extended arm length to the actual extended arm length as the exponent; Calculate the sum of the wheel support force uniformity index and the crane arm extension suitability when the crane is working in the actual working area; calculate the product of the negative mapping result and the sum; Take the reciprocal of the sum of the product and 1 as the second ratio parameter.

9. The online safety monitoring system for crane operation according to claim 1, wherein The method for obtaining the key point operation stability at each actual working moment of the crane according to the high-altitude operation danger index of the crane during actual operation is as follows: Sample the actual working moments of the crane at the third preset parameter during the actual operation of the crane, and calculate the high-altitude operation danger index at each actual working moment of the crane; Take the fourth preset parameter of actual working moments with the closest time interval to each actual working moment of the crane as each adjacent actual working moment of each actual working moment of the crane; For each actual working moment of the crane, calculate the absolute value of the difference between the high-altitude operation danger index of each adjacent actual working moment except the first adjacent actual working moment and the previous adjacent actual working moment at the actual working moment, and calculate the product of the absolute value and the high-altitude operation danger index at the actual working moment; Take the cumulative sum of the reciprocal of the sum of the product and 1 over all the adjacent actual working moments as the key point operation stability at the actual working moment.

10. The on-line safety monitoring system for crane operation according to claim 1, characterized in that, The method for obtaining the predicted value of the key point operation stability based on the autoregressive moving average model and alarming the high-altitude operators online based on the predicted value of the key point operation stability is as follows: Take the key point operation stability at all the actual working moments of the crane as the input of the Otsu threshold segmentation algorithm, take the output of the Otsu threshold segmentation algorithm as the segmentation threshold of the key point operation stability at all the actual working moments of the crane, and take the segmentation threshold of the key point operation stability at all the actual working moments of the crane as the safety threshold for the crane operation; Take the key-point operation stability at all actual working moments of the crane as the input of the autoregressive moving average model, and take the output of the autoregressive moving average model as the predicted value of the key-point operation stability at the current actual working moment of the crane; When the predicted value of the key-point operation stability at the current actual working moment of the crane is greater than the safety threshold for the crane to work, the crane can continue to work at this time; when the predicted value of the key-point operation stability at the current actual working moment of the crane is less than or equal to the safety threshold for the crane to work, an alarm is given through the warning module, and the crane is not suitable to continue working at this time.

Images