Winch overshoot monitoring method, processor, engineering equipment and storage medium

Abstract

The invention discloses a winch overshoot monitoring method, a processor, engineering equipment and a storage medium, the winch overshoot monitoring method is applied to engineering equipment of a fuel architecture and a new energy architecture, the engineering equipment comprises a winch mechanism, the winch mechanism comprises a suspension arm and a pulley, the head of the suspension arm is provided with an image acquisition device, and the pulley is provided with an identification pattern. The identification pattern is located in the shooting range of the image acquisition device, and the method comprises the steps that an image acquired by the image acquisition device is acquired, and the image comprises the identification pattern; determining contour information of the identification pattern in the image according to the image based on a pre-constructed neural network model; determining an actual distance between the pulley and the image acquisition device according to the contour information and a predetermined calibration parameter of the image acquisition device; and under the condition that the actual distance is smaller than the preset safety distance, the overshoot phenomenon of the hoisting mechanism is determined. According to the invention, the judgment error is reduced, and the accuracy of winch overshoot monitoring is improved.

Description

Technical Field

[0001] The present application relates to the technical field of engineering machinery, and in particular to a method, a processor, an engineering device and a storage medium for monitoring winch overshoot. Background Art

[0002] The winch monitoring method plays a vital role in the fields of industrial automation and lifting operations. In the field of automation, especially in lifting operations, the performance monitoring of the winch is crucial. Traditional winch monitoring methods mainly rely on manual observation and simple mechanical measurements, such as using a ruler or pulley block to measure the elongation of the wire rope or the rotation angle of the winch to determine whether there is an overshoot risk in the winch process. However, the manual monitoring method is affected by experience and environment and cannot guarantee accuracy. Therefore, the winch overshoot monitoring method in the prior art has the problem of large errors. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a method, processor, engineering equipment and machine-readable storage medium for monitoring winch overshoot, so as to solve the problem of large errors in the winch overshoot monitoring method in the prior art.

[0004] In order to achieve the above-mentioned purpose, a first aspect of an embodiment of the present application provides a method for monitoring winch overshoot, which is applied to engineering equipment, wherein the engineering equipment includes a winch mechanism, the winch mechanism includes a boom and a pulley, an image acquisition device is installed on the head of the boom, and an identification pattern is provided on the pulley, and the identification pattern is within the shooting range of the image acquisition device, and the method includes: Acquire an image captured by an image acquisition device, wherein the image includes a logo pattern; Based on the pre-built neural network model, the contour information of the logo pattern in the image is determined according to the image; determining an actual distance between the pulley and the image acquisition device based on the profile information and predetermined calibration parameters of the image acquisition device; When the actual distance is less than the preset safety distance, it is determined that the hoisting mechanism has overshoot.

[0005] In an embodiment of the present application, the pre-constructed neural network model includes a target detection branch, a marker segmentation branch and a precise segmentation branch. Based on the pre-constructed neural network model, the position information of the marker pattern in the image is determined according to the image, including: based on the target detection branch, identifying the area where the marker pattern is located in the image to obtain a local image area containing the marker pattern; based on the marker segmentation branch, segmenting the local image area to obtain a marker area image; based on the precise segmentation branch, determining the contour information of the marker pattern according to the marker area image.

[0006] In an embodiment of the present application, based on the precise segmentation branch, the contour information of the identification pattern is determined according to the identification area image, including: according to the precise segmentation branch, the first contour, the second contour and the third contour of the identification area image are segmented, the first contour is the contour formed by the starting point of the change of the identification edge value in the identification area image, the second contour is the contour formed by the end point of the gradient change in the identification area image, and the third contour is the contour formed by the midpoint of the first contour and the second contour; the first contour, the second contour and the third contour are merged to obtain the contour information of the identification pattern.

[0007] In the embodiment of the present application, based on the precise segmentation branch, before determining the contour information of the identification pattern according to the identification area image, it also includes: completing the identification pattern in the identification area image according to the geometric feature information of the identification pattern.

[0008] In an embodiment of the present application, before the local image area is segmented based on the identification segmentation branch to obtain the identification area image, it also includes: determining the maximum grayscale value and the minimum grayscale value in the local image area; determining the adjustment coefficient according to the maximum grayscale value and the minimum grayscale value; and performing pixel enhancement processing on the local image area according to the adjustment coefficient and the minimum grayscale value.

[0009] In an embodiment of the present application, the identification pattern includes a first figure and a second figure, the first figure surrounds the second figure, and the contour information includes an outer contour corresponding to the first figure and an inner contour corresponding to the second figure; according to the contour information and a predetermined calibration parameter of the image acquisition device, the actual distance between the pulley and the image acquisition device is determined, including: correcting the outer contour size of the outer contour according to the geometric feature information of the first figure, and correcting the inner contour size of the inner contour according to the geometric feature information of the second figure to obtain the corrected contour information; according to the corrected contour information and the calibration parameters, the actual distance between the pulley and the image acquisition device is determined.

[0010] In an embodiment of the present application, the actual distance between the pulley and the image acquisition device is determined according to the corrected contour information and the calibration parameters, including: determining the identification image area of ​​the identification pattern in the image according to the corrected contour information; and determining the actual distance between the pulley and the image acquisition device according to the actual area of ​​the identification pattern, the identification image area and the calibration parameters.

[0011] A second aspect of an embodiment of the present application provides a processor configured to execute the above-mentioned method for winch overshoot monitoring.

[0012] A second aspect of an embodiment of the present application provides an engineering equipment, including: a hoisting mechanism, the hoisting mechanism includes a boom and a pulley, an image acquisition device is installed on the head of the boom, an identification pattern is provided on the pulley, and the identification pattern is within the shooting range of the image acquisition device; the above-mentioned processor.

[0013] A fourth aspect of an embodiment of the present application provides a machine-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the above-mentioned method for winch overshoot monitoring is implemented.

[0014] The above technical solution is applied to engineering equipment, the engineering equipment includes a hoisting mechanism, the hoisting mechanism includes a boom and a pulley, an image acquisition device is installed on the head of the boom, an identification pattern is set on the pulley, the identification pattern is in the shooting range of the image acquisition device, by acquiring an image captured by the image acquisition device, the image includes the identification pattern, and then based on the pre-constructed neural network model, the contour information of the identification pattern in the image is determined according to the image, and then the actual distance between the pulley and the image acquisition device is determined according to the contour information and the calibration parameters of the predetermined image acquisition device, and when the actual distance is less than the preset safety distance, it is determined that the hoisting mechanism has an overshoot phenomenon. The present application can process the image through a pre-constructed neural network to determine the contour information of the identification pattern in the image, and then determine the actual distance between the pulley and the image acquisition device according to the contour information, and use it as a basis for judging the overshoot of the hoisting mechanism, which is conducive to reducing the judgment error and improving the accuracy of the overshoot monitoring of the hoisting mechanism.

[0015] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings: Figure 1 A schematic flow chart of a method for monitoring winch overshoot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0018] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), such directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0019] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0020] The engineering equipment of this embodiment can be a fuel architecture and a new energy architecture. Taking the engineering equipment as an engineering machinery vehicle as an example, it can include traditional engineering machinery vehicles and new energy vehicles used in the field of engineering machinery, such as new energy mixer trucks, new energy pump trucks, new energy excavators, etc.; in addition, the engineering machinery vehicles of this embodiment are also intelligent networked vehicles. The engineering machinery vehicles include a sensor / perception system, a communication system, etc., and the vehicle operation data and vehicle surrounding environment information are collected through the sensor / perception system in the vehicle, and the network connection with other vehicles and the cloud is realized through the communication system, and the collected vehicle operation data, vehicle surrounding environment information, etc. are shared to the cloud and other authorized vehicles to realize data sharing, remote analysis, intelligent driving and other operations. Figure 1 A flow chart of a winch overshoot monitoring method provided in an embodiment of the present application. Figure 1 As shown, an embodiment of the present application provides a method for monitoring winch overshoot, which is applied to engineering equipment. The engineering equipment includes a winch mechanism, and the winch mechanism includes a boom and a pulley. An image acquisition device is installed on the head of the boom, and an identification pattern is provided on the pulley. The identification pattern is within the shooting range of the image acquisition device. The method is applied to a processor of engineering equipment as an example for explanation. The method may include the following steps.

[0021] Step S101, acquiring an image captured by an image acquisition device, wherein the image includes a logo pattern.

[0022] Step S102: determining contour information of the identification pattern in the image based on the image based on the pre-built neural network model.

[0023] Step S103, determining the actual distance between the pulley and the image acquisition device according to the contour information and the predetermined calibration parameters of the image acquisition device.

[0024] Step S104: when the actual distance is less than the preset safety distance, it is determined that the hoisting mechanism has an overshoot phenomenon.

[0025] In the embodiment of the present application, the image acquisition device includes a shooting unit, a positioning base and a counterweight, the shooting unit is connected to the counterweight, the counterweight is movably connected to the positioning base, the positioning base is arranged below the head of the boom, and the counterweight is used to drive the shooting angle of the shooting unit to be vertically downward. In this way, the shooting unit can always shoot vertically downward, and the identification pattern on the pulley is always displaced within the shooting range of the shooting unit.

[0026] Specifically, during the hoisting operation of the engineering equipment, in order to monitor the overshoot of the hoisting in real time, the processor can obtain the image captured by the image acquisition device in real time, and the image contains the identification pattern on the pulley. Further, the processor can use the pre-built neural network model to analyze the acquired image, so as to determine the contour information of the identification pattern in the image. Among them, the pre-built neural network model can be used to accurately identify and extract the contour information of a specific identification pattern in the image.

[0027] Next, the processor can determine the actual distance between the pulley and the image acquisition device through geometric calculation or triangulation principle according to the contour information of the identification pattern and predetermined calibration parameters of the image acquisition device, such as focal length and sensor size.

[0028] Finally, the processor can compare the calculated actual distance with a preset safety distance. If the actual distance is less than the preset safety distance, the processor determines that the hoisting mechanism has overshot and triggers the corresponding alarm or safety measures. Among them, the preset safety distance is a safety threshold set according to the actual structure and application scenario of the engineering equipment. When the actual distance is less than the preset safety distance, it means that the pulley of the hoisting mechanism may hit the boom and there is a safety risk. At this time, an alarm message can be output so that the operator can take targeted risk avoidance measures. The alarm information can be output in the form of sound, light, and text. In this way, the operator can monitor the operating status of the hoisting mechanism in real time, especially when there are potential dangerous situations such as overshoot, and can quickly take countermeasures to ensure the safety and efficiency of engineering operations.

[0029] The above technical solution is applied to engineering equipment, the engineering equipment includes a hoisting mechanism, the hoisting mechanism includes a boom and a pulley, an image acquisition device is installed on the head of the boom, an identification pattern is set on the pulley, the identification pattern is in the shooting range of the image acquisition device, by acquiring an image captured by the image acquisition device, the image includes the identification pattern, and then based on the pre-constructed neural network model, the contour information of the identification pattern in the image is determined according to the image, and then the actual distance between the pulley and the image acquisition device is determined according to the contour information and the calibration parameters of the predetermined image acquisition device, and when the actual distance is less than the preset safety distance, it is determined that the hoisting mechanism has an overshoot phenomenon. The present application can process the image through a pre-constructed neural network to determine the contour information of the identification pattern in the image, and then determine the actual distance between the pulley and the image acquisition device according to the contour information, and use it as a basis for judging the overshoot of the hoisting mechanism, which is conducive to reducing the judgment error and improving the accuracy of the overshoot monitoring of the hoisting mechanism.

[0030] In an embodiment of the present application, the pre-constructed neural network model includes a target detection branch, a marker segmentation branch and a precise segmentation branch. Based on the pre-constructed neural network model, determining the position information of the marker pattern in the image according to the image may include: based on the target detection branch, identifying the area where the marker pattern is located in the image to obtain a local image area containing the marker pattern; based on the marker segmentation branch, segmenting the local image area to obtain a marker area image; based on the precise segmentation branch, determining the contour information of the marker pattern according to the marker area image.

[0031] It can be understood that in order to accurately monitor the operating status of the hoisting mechanism, especially to prevent the occurrence of overshoot, the embodiment of the present application adopts a pre-built neural network model to determine the contour information of the identification pattern in the image. The neural network model includes a target detection branch, a identification segmentation branch and a precise segmentation branch. Specifically, the target detection branch in the neural network model is used to perform a preliminary analysis of the acquired image. The target detection branch can identify the area where the identification pattern is located in the image, and extract the local image area containing the identification pattern, which is conducive to reducing the amount of calculation for subsequent processing and improving the accuracy of recognition.

[0032] After obtaining the local image area, the identification segmentation branch in the neural network model is used to further segment the area. The identification segmentation branch can accurately identify the boundary between the identification pattern and the background, thereby obtaining a clear identification area image. Then, the identification area image is refined using the precise segmentation branch in the neural network model. The precise segmentation branch can further refine the contour information of the identification pattern to ensure the accuracy and integrity of the contour. This step is crucial for the subsequent calculation of the actual distance between the pulley and the image acquisition device.

[0033] In this way, the pre-built neural network model is used to accurately identify and segment the pulley identification pattern, providing an accurate data basis for the subsequent calculation of the actual distance between the pulley and the image acquisition device, improving the accuracy and efficiency of monitoring, and providing a strong guarantee for the safe operation of engineering equipment.

[0034] In an embodiment of the present application, based on the precise segmentation branch, determining the contour information of the identification pattern according to the identification area image can include: segmenting the identification area image into a first contour, a second contour and a third contour according to the precise segmentation branch, the first contour being a contour formed by a point where the identification edge value in the identification area image starts to change, the second contour being a contour formed by an end point of the gradient change in the identification area image, and the third contour being a contour formed by a midpoint between the first contour and the second contour; and merging the first contour, the second contour and the third contour to obtain the contour information of the identification pattern.

[0035] It can be understood that in order to more accurately determine the contour information of the identification pattern, the embodiment of the present application adopts the precise segmentation branch in the pre-built neural network model, which can accurately segment multiple contours of the identification pattern based on the identification area image, and obtain the final contour information by fusing these contours. Specifically, first, the local image area containing the identification pattern, that is, the identification area image, is extracted from the original image through the target detection branch and the identification segmentation branch in the neural network model.

[0036] Next, the precise segmentation branch is used to further process the identification area image. This branch can identify and segment the first contour, the second contour and the third contour, where the first contour is the contour formed by the pixel points where the identification edge value begins to change in the identification area image, corresponding to the preliminary boundary between the identification pattern and the background; the second contour is the contour formed by the pixel points where the gradient change ends in the identification area image, because the end point of the gradient change often corresponds to the sharp turning of the edge, so the second contour can more accurately reflect the boundary of the identification pattern; the third contour is the contour formed by the midpoint of the first contour and the second contour. By calculating the midpoint of the two contours, we can get a smoother and more stable third contour, which helps to reduce the influence of noise and abnormal points.

[0037] After obtaining the above three contours, the final identification pattern contour information can be obtained through fusion processing. In one example, the fusion processing includes aligning the contours to ensure that the three contours are aligned in space, that is, they correspond to different representations of the same identification pattern; then weighted averaging the points on each contour to obtain a comprehensive contour, and the weight of the weighted average can be determined according to the reliability or importance of the contour, and finally smoothing the fused contour to eliminate possible noise or jagged edges to obtain the final contour information.

[0038] In this way, by utilizing the multiple contour information provided by the precise segmentation branch, the contour of the identification pattern can be determined more accurately, which is beneficial to improving the accuracy of subsequent winch overshoot monitoring and providing reliable guarantee for the safe operation of engineering equipment.

[0039] In the embodiment of the present application, based on the precise segmentation branch, before determining the contour information of the identification pattern according to the identification area image, the method may also include: completing the identification pattern in the identification area image according to the geometric feature information of the identification pattern.

[0040] It can be understood that since the operation of engineering equipment is subject to changes in weather, daytime, nighttime, on-site lighting, etc., the operating environment is extremely unstable. In the process of image acquisition, due to various reasons, such as occlusion and uneven lighting, the identification pattern in the image may be partially missing or unclear. In order to more accurately determine the contour information of the identification pattern, the identification pattern in the identification area image can be completed before using the precise segmentation branch. Specifically, since the identification pattern has a specific graphic composition, the geometric feature information of the identification pattern is known, and the geometric feature information includes information such as the shape, size, direction, and possible symmetry of the identification pattern.

[0041] Furthermore, the identification pattern in the identification area image can be completed based on the geometric feature information of the identification pattern. In one example, a complete identification pattern matching the identification pattern can be found in a pre-built image library or template library based on the geometric features of the identification pattern. In another example, an image restoration algorithm (such as sample-based restoration, structure-based restoration, etc.) can be used to fill and repair the missing or unclear parts.

[0042] By using this method, we can complete the logo pattern before determining its contour information, thereby improving the accuracy and reliability of contour determination. This not only helps to improve the accuracy of monitoring, but also provides a more solid guarantee for the safe operation of engineering equipment.

[0043] In an embodiment of the present application, before segmenting the local image area based on the identification segmentation branch to obtain the identification area image, it can also include: determining the maximum grayscale value and the minimum grayscale value in the local image area; determining an adjustment coefficient based on the maximum grayscale value and the minimum grayscale value; and performing pixel enhancement processing on the local image area based on the adjustment coefficient and the minimum grayscale value.

[0044] It can be understood that in order to improve the quality of the image and make the subsequent segmentation and contour determination process more accurate and reliable, the local image area can also be subjected to pixel enhancement processing before the local image area is segmented using the identification segmentation branch. Specifically, the maximum grayscale value and the minimum grayscale value in the local image area are calculated, and these two values ​​define the range of brightness and darkness in the image.

[0045] According to the maximum grayscale value and the minimum grayscale value, an adjustment coefficient is determined, and the adjustment coefficient is used to control the intensity of pixel enhancement. The adjustment coefficient is proportional to the difference between the maximum grayscale value and the minimum grayscale value, that is, the greater the difference, the greater the adjustment coefficient, so as to enhance the contrast of the image. Further, according to the adjustment coefficient and the minimum grayscale value, the pixel enhancement processing is performed on the local image area. In one example, the grayscale value of the local image area can be linearly stretched according to the adjustment coefficient and the minimum grayscale value to expand the contrast range of the local image area, and the grayscale histogram of the local image area can be adjusted so that the grayscale value distribution of the local image area is more uniform, thereby improving the contrast of the image.

[0046] Thus, before segmenting the local image area, pixel enhancement is performed on it, thereby improving the image quality and contrast, which helps to improve the accuracy of segmentation and provide more reliable image data for subsequent contour determination and actual distance calculation.

[0047] In an embodiment of the present application, the identification pattern includes a first figure and a second figure, the first figure surrounds the second figure, and the contour information includes an outer contour corresponding to the first figure and an inner contour corresponding to the second figure; determining the actual distance between the pulley and the image acquisition device according to the contour information and a predetermined calibration parameter of the image acquisition device may include: correcting the outer contour size of the outer contour according to the geometric feature information of the first figure, and correcting the inner contour size of the inner contour according to the geometric feature information of the second figure to obtain corrected contour information; determining the actual distance between the pulley and the image acquisition device according to the corrected contour information and the calibration parameters.

[0048] It can be understood that the identification pattern can be composed of a first figure and a second figure, wherein the first figure surrounds the second figure, and the contour information includes an outer contour corresponding to the first figure and an inner contour corresponding to the second figure. In order to accurately determine the actual distance between the pulley and the image acquisition device, the outer contour corresponding to the first figure and the inner contour corresponding to the second figure can be dimensionally corrected respectively. Specifically, the outer contour is dimensionally corrected according to the geometric feature information of the first figure, including the shape, size, and proportion of each edge. The inner contour is dimensionally corrected according to the geometric feature information of the second figure. Then, after obtaining the corrected contour information, the processor can determine the actual distance between the pulley and the image acquisition device according to the corrected contour information and the calibration parameters of the image acquisition device.

[0049] In one example, one or more measurement reference points of the identification pattern can be determined based on the corrected contour information. These reference points can be specific points on the contour, such as vertices and center points, or certain features of the contour, such as intersections of straight line segments and circular arcs. Then, combined with the calibration parameters of the image acquisition device, including focal length, sensor size, and pixel size, the measurement reference points are converted from the image coordinate system to the actual coordinate system, and the actual distance between the pulley and the image acquisition device is determined by geometric calculation or triangulation principle using the converted measurement reference point coordinates.

[0050] In another example, the logo pattern is composed of a rectangular outer contour and a circular inner contour, and the circle is located inside the rectangle. For the outer contour correction, since the side length ratio of the rectangular outer contour is fixed, the length difference between adjacent sides can be calculated and the corresponding expansion and contraction adjustment can be made to correct the expansion and contraction deformation of the outer contour, and the angle between the long and short sides can be calculated and compared with the angle of the standard rectangle to make the corresponding rotation adjustment to correct the distortion. For the inner contour correction, since the distance between the center of the circular inner contour and the vertex of the right-angle side of the rectangle on the outer contour is fixed, the deformation inside the logo can be corrected by using this fixed distance relationship, and the ratio of the circle area to the area of ​​the entire logo pattern (i.e., the area of ​​the rectangle) is fixed, so the internal deformation can be further corrected by calculating the ratio of the area of ​​the inner contour to the area of ​​the entire logo and comparing it with the preset fixed ratio. In this way, the logo pattern in the logo area image is finally corrected by combining the correction results of the outer contour and the inner contour to ensure that its shape and size meet the preset standards.

[0051] In this way, by correcting the dimensions of the outer contour and the inner contour of the identification pattern, the actual distance between the pulley and the image acquisition device can be more accurately determined, which is beneficial to improving the accuracy of winch overshoot monitoring.

[0052] In an embodiment of the present application, determining the actual distance between the pulley and the image acquisition device according to the corrected contour information and the calibration parameters may include: determining the identification image area of ​​the identification pattern in the image according to the corrected contour information; and determining the actual distance between the pulley and the image acquisition device according to the actual area of ​​the identification pattern, the identification image area and the calibration parameters.

[0053] Specifically, the processor can also calculate the area of ​​the identification pattern in the image, i.e., the identification image area, based on the corrected contour information using image processing techniques such as pixel counting and shape analysis. The actual area of ​​the identification pattern and the calibration parameters of the image acquisition device are obtained. The actual area of ​​the identification pattern is known during production or design, and it represents the real size of the identification pattern in the physical world. The calibration parameters of the image acquisition device include focal length, sensor size, pixel size, etc. These parameters are inherent in the image acquisition system and are used to convert the size in the image into the actual size. Further, by comparing the actual area of ​​the identification pattern and the identification image area, a proportional factor can be calculated, which reflects the proportional relationship between the size in the image and the actual size. Then, using the proportional factor and the calibration parameters of the image acquisition device, especially the focal length and sensor size, the actual distance between the pulley and the image acquisition device can be determined by geometric calculation or triangulation principle. This process needs to consider factors such as the geometric layout of the image acquisition system, the viewing angle of the camera, and the position of the pulley in the image.

[0054] In this way, by combining the corrected contour information, the actual area of ​​the identification pattern and the calibration parameters of the image acquisition device, the actual distance between the pulley and the image acquisition device can be accurately determined.

[0055] In a specific embodiment of the present application, the method for monitoring winch overshoot in the above embodiment is applied to crane operation as an example for description. The winch mechanism of the crane includes a boom, an image acquisition device, a movable pulley, a steel wire and a hook. Among them, the hook is connected to the movable pulley, and the hook and the movable pulley are driven to rise and fall at the same time by the steel wire wound on the movable pulley. The movable pulley is provided with an identification pattern on the other side opposite to the side where the hook is located. The identification pattern is used for distance conversion. The identification pattern includes a rectangular pattern and a circular pattern. The number of circular patterns is multiple and located in the rectangular pattern. The image acquisition device is installed at the head of the boom. The image acquisition device includes a camera, a fixed seat and a counterweight. Under the action of the counterweight, the camera is driven to always face vertically downward. When the camera is turned on, since the movable pulley connected to the hook is also always located vertically below the head of the boom, the image acquisition device always automatically tracks and shoots the identification pattern on the movable pulley. During the crane operation process, the monitoring of the crane winch overshoot can be realized based on the method for monitoring winch overshoot in the above embodiment.

[0056] Specifically, during the operation of the crane, an image of a space hook with an identification pattern is collected in real time by an image acquisition device, and then the image is input into a pre-built neural network model, and the position of the identification pattern in the image is detected by the neural network model. Furthermore, based on a geometric transformation model from image coordinates to world coordinates established in advance according to the calibration parameters of the camera and the known size of the identification pattern, the position of the identification pattern in the image is converted to the position in the world coordinate system using the pre-built geometric transformation model, and the distance between the identification pattern and the image reference device is determined according to the position of the identification in the world coordinate system and the position of the image acquisition device. This process involves distance calculation in three-dimensional space, which can be implemented using vector operations or geometric formulas, and will not be described in detail here.

[0057] Specifically, in order to improve data processing efficiency, the image can be input into a neural network for target detection to detect the area where the hook and the logo pattern are located in the image, and obtain a local area image, which can reduce the amount of data processing and improve data processing efficiency. Then, considering the problem of image distortion, the regional image can be preliminarily corrected according to the geometric features of the rectangular pattern in the logo pattern to obtain a preliminarily corrected regional image.

[0058] Furthermore, considering that the operation of engineering equipment is affected by factors such as weather, daytime, nighttime, and on-site lighting, the working environment is extremely unstable. In order to reduce the impact of environmental factors, the regional image after preliminary correction can be enhanced to enhance the contour edge information of the identification pattern.

[0059] In one example, the contour edge information can be enhanced by grayscale scaling image features to make the black parts of the image darker and the bright parts brighter. First, the adjustment coefficient is determined according to the maximum grayscale value and the minimum grayscale value in the current regional image. Then, the grayscale value of each pixel in the regional image is scaled according to the adjustment coefficient and the minimum grayscale value to meet the following formula: Gt=Gs×(255 / (Gmax- Gmin))- Gmin×(255 / (Gmax- Gmin)); Among them, Gt is the gray value after transformation, Gs is the gray value of the pixel in the original area image, Gmax is the maximum gray value in the original area image, and Gmin is the minimum gray value in the original image.

[0060] In another example, specific edges can be enhanced according to the shape of the identification pattern. Taking the case in which the identification pattern in this specific embodiment includes a rectangular pattern, the right-angled edges and straight line gradients that can be detected in the area image are enhanced, and the pixel points on the right-angled edges are determined by dividing the square of the grayscale value of the pixel point itself by the grayscale value of the adjacent pixel point to determine the adjustment value of the pixel point, thereby enhancing the right-angled edges.

[0061] Further, the trained neural network is used to perform identification segmentation on the enhanced regional image to segment the background of the identification pattern and obtain an identification regional image containing only the identification pattern. Then, considering that the camera is installed on the head of the boom and the hook is connected by a wire rope, the identification pattern may be blocked when the image is collected, resulting in excessive errors in the subsequent distance conversion. Therefore, the identification pattern of the blocked part can be completed first. Since the geometric shape of the outer contour of the identification pattern is a regular and fixed rectangle, the identification regional image can be completed by geometric relationship. Specifically, the straight lines in the identification regional image are first detected, and then the angles between the straight lines are calculated among the detected straight lines and it is determined whether they are close to 90 degrees to screen out pairs of straight lines that can form right angles. According to the geometric features of the rectangle, when three right angles are detected in the identification regional image, the fourth right angle can be completed according to the geometric relationship. After the four right angles of the rectangle are determined, the blocked identification edge is completed according to the position and direction of these right angles. In this way, the initial completed identification regional image is obtained.

[0062] Furthermore, in order to accurately determine the area of ​​the identification pattern in the image, a trained neural network can be used to accurately segment the identification contour of the above-mentioned completed identification area image to obtain the contour information of the identification pattern, including the outer contour of the rectangle and the inner contour of the circle. Among them, the input data of the neural network model is the identification area image to be segmented. During training, the first contour formed by the starting point of the identification edge value change, the second contour formed by the end point of the gradient change, and the third contour formed by the midpoint of the first contour and the second contour are used as three different labels to train the neural network model, and then the trained neural network model is used to predict the above three different contours. The three contour labels play a strong supervisory role in the model expectation, and the contour segmentation accuracy is 30% higher than the single MASK or single contour output accuracy. Therefore, according to the neural network model, the first contour, the second contour and the third contour of the identification area image can be obtained, and then the three contours are fused to obtain the contour information of the identification pattern.

[0063] Furthermore, in order to ensure the accuracy of the detection results, the logo pattern in the image can be corrected twice based on the contour information, including the correction of the logo outer contour size and the logo inner contour size. Since the outer contour is a rectangle and the side length ratio of the rectangle is fixed, the distortion of the outer contour can be corrected by the expansion and contraction deformation of the outer contour on adjacent sides and the change of the angle between the long and short sides. Since the inner contour is a circle, the distance between the point on the inner contour circle and the corresponding point on the outer contour is fixed, and the ratio of the circle area to the area of ​​the entire logo pattern is fixed. For the outer contour of the circle, the internal deformation can be corrected accordingly, so as to obtain the corrected contour information of the logo pattern.

[0064] Finally, the area of ​​the identification pattern in the image is determined based on the contour information, and then the actual distance between the pulley and the image acquisition device is determined in combination with the calibration parameters of the shooting unit including the object distance and the focal length.

[0065] In this way, by adopting the precise segmentation method based on neural network and the correction method based on geometric relationship, the marked area is accurately obtained, thereby improving the accuracy of measurement, providing a reliable basis for distance conversion and overshoot monitoring, and improving the accuracy of overshoot monitoring.

[0066] The embodiment of the present application also provides a processor configured to execute the method for monitoring winch overshoot in the implementation manner.

[0067] An embodiment of the present application also provides an engineering equipment, including: a hoisting mechanism, the hoisting mechanism includes a boom and a pulley, an image acquisition device is installed on the head of the boom, an identification pattern is provided on the pulley, and the identification pattern is within the shooting range of the image acquisition device; the processor in the above embodiment.

[0068] An embodiment of the present application also provides a machine-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the method for monitoring winch overshoot in the above-mentioned embodiment is implemented.

[0069] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0070] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0071] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0073] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0074] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0075] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0076] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0077] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for monitoring winch overshoot, characterized in that: Applied to engineering equipment, the engineering equipment includes a hoisting mechanism, the hoisting mechanism includes a boom and a pulley, an image acquisition device is installed on the head of the boom, an identification pattern is provided on the pulley, and the identification pattern is within the shooting range of the image acquisition device, the method includes: Acquire an image captured by the image acquisition device, wherein the image includes the identification pattern; Determine, based on the pre-built neural network model, contour information of the identification pattern in the image according to the image; Determining an actual distance between the pulley and the image acquisition device according to the profile information and predetermined calibration parameters of the image acquisition device; When the actual distance is less than the preset safety distance, it is determined that an overshoot phenomenon occurs in the hoisting mechanism.

2. The method according to claim 1, characterized in that The pre-built neural network model includes a target detection branch, a marker segmentation branch, and a precise segmentation branch. The pre-built neural network model is used to determine the position information of the marker pattern in the image according to the image, including: Based on the target detection branch, identifying the area where the identification pattern is located in the image to obtain a local image area containing the identification pattern; Based on the identification segmentation branch, segmenting the local image area to obtain an identification area image; Based on the precise segmentation branch, contour information of the identification pattern is determined according to the identification area image.

3. The method according to claim 2, characterized in that The step of determining the contour information of the identification pattern according to the identification area image based on the precise segmentation branch includes: According to the precise segmentation branch, a first contour, a second contour and a third contour of the marked area image are segmented, wherein the first contour is a contour formed by a point where the marked edge value in the marked area image starts to change, the second contour is a contour formed by a point where the gradient changes in the marked area image ends, and the third contour is a contour formed by a midpoint between the first contour and the second contour; The first contour, the second contour and the third contour are merged to obtain contour information of the identification pattern.

4. The method according to claim 2, characterized in that: Before determining the contour information of the identification pattern according to the identification area image based on the precise segmentation branch, the method further includes: The identification pattern in the identification area image is completed according to the geometric feature information of the identification pattern.

5. The method according to claim 2, characterized in that: Before segmenting the local image region based on the identification segmentation branch to obtain the identification region image, the method further includes: Determine the maximum grayscale value and the minimum grayscale value in the local image area; Determine an adjustment coefficient according to the maximum grayscale value and the minimum grayscale value; Perform pixel enhancement processing on the local image area according to the adjustment coefficient and the minimum grayscale value.

6. The method according to claim 1, characterized in that The identification pattern includes a first graphic and a second graphic, the first graphic surrounds the second graphic, and the contour information includes an outer contour corresponding to the first graphic and an inner contour corresponding to the second graphic; Determining the actual distance between the pulley and the image acquisition device according to the profile information and a predetermined calibration parameter of the image acquisition device includes: Correcting the outer contour size of the outer contour according to the geometric feature information of the first figure, and correcting the inner contour size of the inner contour according to the geometric feature information of the second figure, so as to obtain corrected contour information; The actual distance between the pulley and the image acquisition device is determined according to the corrected contour information and the calibration parameters.

7. The method according to claim 6, characterized in that The determining the actual distance between the pulley and the image acquisition device according to the corrected profile information and the calibration parameters includes: Determine the identification image area of ​​the identification pattern in the image according to the corrected contour information; The actual distance between the pulley and the image acquisition device is determined according to the actual area of ​​the identification pattern, the area of ​​the identification image and the calibration parameters.

8. A processor, characterized in that: The device is configured to perform the method for monitoring winch overshoot according to any one of claims 1 to 7.

9. An engineering equipment, characterized in that: include: A hoisting mechanism, the hoisting mechanism comprising a boom and a pulley, an image acquisition device is installed at the head of the boom, an identification pattern is provided on the pulley, and the identification pattern is within the shooting range of the image acquisition device; a processor according to claim 8.

10. A machine-readable storage medium storing a program or an instruction, characterized in that: When the program or the instruction is executed by a processor, the method for monitoring winch overshoot according to any one of claims 1 to 7 is implemented.

Images