A monorail crane locomotive anti-collision intelligent early warning method and system

Abstract

The present invention relates to the field of anti-collision warning, and in particular to an intelligent anti-collision warning method and system for a monorail crane locomotive. The method comprises: obtaining multiple images of the monorail crane locomotive's travel path, constructing multiple segmentation lines for each image, and taking the pixel points corresponding to the segmentation lines as feature pixel points; calculating the optical flow field of two adjacent images, extracting the optical flow of each feature pixel point and calculating the motion vector; taking the feature pixel points with equal length from the center point of the image as the same category; calculating the possibility of a collision of the monorail crane locomotive at each moment, and taking the difference in the possibility of a collision of the monorail crane locomotive at two adjacent moments as the possibility change; issuing a collision warning in response to the possibility change being greater than 0. The present invention predicts the probability of a collision between a monorail crane locomotive and an obstacle by calculating the probability of adjacent moments, and further adjusts the travel speed of the monorail crane to avoid a collision between the monorail crane and the obstacle.

Description

Technical Field

[0001] The present invention relates to the field of anti-collision warning, and in particular to an anti-collision intelligent warning method and system for a monorail crane locomotive. Background Art

[0002] In the process of industrial transportation, monorail cranes are used in warehouses and factories as an efficient material handling tool. The cranes can operate flexibly in narrow spaces, greatly improving the efficiency of material handling. With the complexity of the industrial environment, cranes face more and more safety hazards during operation, especially the risk of collision with other mobile equipment (such as forklifts) and pedestrians.

[0003] A Chinese patent document with announcement number CN204117307U discloses a pump truck boom anti-collision warning device, wherein each pump truck includes a pump truck body and a pump truck boom, and the boom anti-collision warning device includes a first component, a second component, a third component and an alarm; there are multiple first components, which are respectively installed at each joint point of the pump truck boom and are used to send wireless ranging signals; the second component, the third component and the alarm are all installed on the pump truck body, and the second component is used to receive and forward the wireless ranging signal; the third component is used to receive the distance information sent by the second component and process the received information; when the distance between the two first components at the ends of two adjacent pump truck booms exceeds the set alarm distance, the alarm is activated and an alarm is sounded to stop the two adjacent pump truck booms from moving outside a safe distance.

[0004] The safety protection measures in the existing technology mainly rely on the monitoring of the crane operator and sensors. During use, the sensors have strict distance requirements and are more sensitive within a certain distance range. Beyond a certain distance range, the sensitivity is poor, which leads to poor accuracy of the monitored data. During operation, the monorail crane needs to predict obstacles in the driving path in advance so that the monorail crane has sufficient reaction time to adjust the speed and avoid accidents. The existing safety protection measures cannot meet actual needs. Summary of the invention

[0005] In order to predict obstacles in a driving path in advance to avoid collision between a monorail crane and the obstacles, the present invention provides a monorail crane anti-collision intelligent early warning method and system.

[0006] In a first aspect, the present invention provides a monorail crane locomotive anti-collision intelligent early warning method, which adopts the following technical solution:

[0007] Acquire multiple images of the monorail locomotive travel path, construct multiple segmentation lines for each image, and use the pixel points corresponding to the segmentation lines as feature pixel points;

[0008] The Lucas-Kanade algorithm is used to calculate the optical flow field of two adjacent images, extract the optical flow of each feature pixel and calculate the motion vector; the feature pixels with the same distance from the center of the image are regarded as the same category to classify the feature pixels;

[0009] Calculate the probability of a collision between the monorail crane and the locomotive at each moment, and use the difference between the probability of a collision between the monorail crane and the locomotive at two adjacent moments as the probability variation; in response to the probability variation being greater than 0, issue a collision warning;

[0010] Among them, the expression of the possibility of collision between monorail crane and locomotive is:

[0011]

[0012] In the formula, represents the possibility of a crane-car collision at the j+1th moment, represents the preset correlation coefficient, Indicates the preset collision index; represents the cosine value of the angle between the motion vector of the nth feature pixel on the mth segmentation line in the image at the jth moment and the horizontal axis, N represents the number of feature pixels corresponding to the segmentation line, It represents the absolute value of the cosine of the angle between the mth segmentation line and the horizontal axis, and M represents the total number of segmentation lines in the image at the jth moment.

[0013] The possibility of a monorail crane collision is calculated by the inclination angle of the motion vector of the feature pixel point. The influencing factors of multiple feature pixels are referred to to improve the accuracy of the calculation results, which makes it easier to judge the probability of a monorail crane collision with an obstacle. Therefore, the monorail crane can be adjusted in speed to avoid a collision with an obstacle.

[0014] Preferably, the method further comprises:

[0015] The motion speed of the feature pixel is calculated based on the motion vector. The expression of the motion speed is:

[0016]

[0017]

[0018] In the formula, Represents the motion vector of the feature pixel, Indicates the horizontal moving speed of the i-th feature pixel in the image. Indicates the vertical moving speed of the i-th feature pixel in the image. represents the moving speed of the i-th feature pixel in the image at the j-th moment, It represents the horizontal moving speed of the i-th feature pixel in the image at the j-th moment. Indicates the vertical moving speed of the i-th feature pixel in the image at the j-th moment.

[0019] By calculating the movement speed of the feature pixel points, we can intuitively understand the moving speed and moving direction of the feature pixel points.

[0020] Preferably, the expression of the collision index is:

[0021]

[0022] In the formula, Indicates the preset collision index, Indicates the kth feature pixel in the image at the jth moment The speed corresponding to the feature pixel point, L represents the total number of the k-th type of feature pixel points in the image at the j-th moment, represents the average motion speed of the k-th feature pixel in the image at the j-th moment, K represents the number of feature pixel categories in the image at the j-th moment, Represents the distance between the k-th feature pixel in the image at the j-th moment and the center of the circle.

[0023] The collision index can reflect the probability of collision between the monorail crane and obstacles.

[0024] Preferably, the expression of the correlation coefficient is: ;

[0025] In the formula, α represents the correlation coefficient, It represents the number of feature pixels whose motion vectors of feature pixels in the image at the jth moment have an angle with the segmentation line. Represents the total number of feature pixels in the image at the jth moment.

[0026] The correlation coefficient can reflect the chaotic state of the characteristic pixels in the image, and the chaotic state can be used to further understand whether there are obstacles in the driving path.

[0027] Preferably, the method for constructing the dividing line is:

[0028] A coordinate system is established with the center of the image as the origin, the horizontal direction as the X-axis, and the vertical direction as the Y-axis to obtain the position coordinates of each pixel point in the coordinate system; segmentation lines are evenly drawn with the origin as the center, and the angles between adjacent segmentation lines are the same.

[0029] By constructing segmentation lines to obtain feature pixels, the feature pixels in the image are analyzed, reducing the amount of calculation.

[0030] Preferably, the angle between the motion vector and the horizontal direction is calculated using the atan2 function.

[0031] Preferably, after acquiring the image of the monorail crane locomotive travel path, the method also includes the steps of graying and denoising the image.

[0032] In a second aspect, the present invention provides a monorail crane locomotive anti-collision intelligent warning system, which adopts the following technical solutions:

[0033] A monorail crane locomotive anti-collision intelligent warning system, a processor and a memory, the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the monorail crane locomotive anti-collision intelligent warning method described above is implemented.

[0034] The above-mentioned monorail crane locomotive anti-collision intelligent warning method is generated into a computer program and stored in a memory so as to be loaded and executed by a processor. Thus, a system is made based on the memory and the processor for easy use.

[0035] The present invention has the following technical effects:

[0036] The possibility of a monorail crane collision is calculated by using multiple dimensions such as the inclination angle, moving speed and correlation coefficient of the motion vector of the feature pixel point, which improves the accuracy of the calculation results. The probability of a monorail crane collision with an obstacle is predicted by the possibility of adjacent moments, so that the traveling speed of the monorail crane can be adjusted to avoid a collision between the monorail crane and the obstacle, which can meet the needs of safety protection in actual operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-restrictive manner, and the same or corresponding reference numerals represent the same or corresponding parts.

[0038] Figure 1 The present invention is a method flow chart of a monorail crane locomotive anti-collision intelligent early warning method. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0040] It should be understood that when the terms "first", "second", etc. are used in the claims, descriptions, and drawings of the present invention, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the description and claims of the present invention indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections.

[0041] The embodiment of the present invention discloses a monorail crane locomotive anti-collision intelligent early warning method, referring to Figure 1 , including the following steps, as follows:

[0042] S1: Acquire multiple images of the monorail locomotive's travel path.

[0043] An image acquisition device is installed on the monorail crane to ensure that its viewing angle can clearly capture the image of the path ahead of the monorail crane during its travel. After the crane starts to travel, the image of the path ahead of the crane during its travel is collected in real time at set time intervals. When the monorail crane is actually traveling, the area it passes through is determined according to the track, and the position of the track is fixed. Therefore, according to the area that the crane will actually pass through during its travel, a rectangular area is marked in the acquired image, so that the image in the area is the image of the area that the crane is about to pass through during its travel. By marking the rectangular area, other areas in the image that are not related to the travel path of the crane are excluded from the interference of image analysis, and the rectangular area of ​​each image is cropped, and the cropped image is used as the image of the monorail crane's travel path. The image of the monorail crane's travel path is grayed and denoised.

[0044] S2: For each image, multiple segmentation lines are constructed, and the pixels corresponding to the segmentation lines are taken as feature pixels. The feature pixels with the same length from the center point of the image are taken as the same category to classify the feature pixels.

[0045] A coordinate system is established with the center of the image as the origin, the horizontal direction as the X-axis, the horizontal right direction as the positive direction, the vertical direction as the Y-axis, and the vertical upward direction as the positive direction to obtain the position coordinates of each pixel in the coordinate system; segmentation lines are evenly drawn with the origin as the center, and the angles between adjacent segmentation lines are the same, and the pixels corresponding to the segmentation lines are feature pixels. For the segmentation lines, feature pixels with the same distance from the origin are regarded as feature pixels of the same category.

[0046] S3: Use the Lucas-Kanade algorithm to calculate the optical flow field of two adjacent images, extract the optical flow of each feature pixel and calculate the motion vector.

[0047] The expression of the motion vector of the feature pixel is: ; In the formula, Represents the motion vector of the feature pixel, Indicates the horizontal moving speed of the i-th feature pixel in the image. Indicates the vertical speed of the i-th feature pixel in the image. Use the atan2 function to calculate the angle between the motion vector and the horizontal direction.

[0048] The moving speed of the feature pixel is calculated based on the motion vector of the feature pixel. The expression of the moving speed of the feature pixel is:

[0049]

[0050] In the formula, represents the moving speed of the i-th feature pixel in the image at the j-th moment, It represents the horizontal moving speed of the i-th feature pixel in the image at the j-th moment. Indicates the vertical moving speed of the i-th feature pixel in the image at the j-th moment.

[0051] Exemplarily, for the images at the nth moment and the n+1th moment, the Lucas-Kanade algorithm is used to calculate the optical flow fields of the two images. The optical flow field is composed of multiple optical flows, and each pixel corresponds to an optical flow. The optical flow represents the displacement of the same pixel from the image at the nth moment to the image at the n+1th moment. The motion vector of the pixel can be obtained based on the displacement of the same pixel and the time interval between the two images. The motion vector represents the horizontal movement speed and movement direction of the pixel, as well as the vertical movement speed and movement direction. The movement speed represents the total movement speed of the pixel.

[0052] When a monorail crane is traveling, there are normally no obstacles on its travel path, and the ground and background of the path are relatively still as a whole. The absolute value of the motion vector of the feature pixel point on the same dividing line between two images at adjacent moments is approximately equal to the cosine value of the horizontal axis angle, and the speed of the feature pixel points of the same category is also approximately equal. The closer the feature pixel point is to the edge of the image, the faster the speed. In the optical flow field, the optical flow of all feature pixel points tends to spread outward with the center of the image as the center.

[0053] During actual operation, there may be pedestrians or other vehicles carrying goods passing by on the path of the monorail crane. If the pedestrians or vehicles do not leave in time before the monorail crane approaches, they may collide with the monorail crane.

[0054] When pedestrians and vehicles appear on the path of the monorail crane, they move relative to the background. Therefore, the optical flow speed and direction of each feature pixel between images at adjacent moments will be quite different, and the optical flow distribution in the optical flow field will be more chaotic. If pedestrians and vehicles continue to stay on the path, as the crane moves forward, the proportion of pedestrians and vehicles in the image will increase, and the proportion in the feature pixels will also increase, resulting in a more chaotic state. Based on this phenomenon, the probability of collision between the monorail crane and the obstacle is calculated.

[0055] S4: Calculate the probability of a collision between the monorail locomotives at each moment.

[0056] The expression of the correlation coefficient is: ; In the formula, α represents the correlation coefficient, It represents the number of feature pixels whose motion vectors of feature pixels in the image at the jth moment have an angle with the segmentation line. Represents the total number of feature pixels in the image at the jth moment.

[0057] The expression of the collision index is: ; In the formula, Indicates the preset collision index, Indicates the kth feature pixel in the image at the jth moment The speed corresponding to the feature pixel point, L represents the total number of the k-th type of feature pixel points in the image at the j-th moment, represents the average motion speed of the k-th feature pixel in the image at the j-th moment, K represents the number of feature pixel categories in the image at the j-th moment, Represents the distance between the k-th feature pixel in the image at the j-th moment and the center of the circle.

[0058] The probability of a locomotive collision is expressed as:

[0059]

[0060] In the formula, represents the possibility of a crane-car collision at the j+1th moment, represents the preset correlation coefficient, Indicates the preset collision index; represents the cosine value of the angle between the motion vector of the nth feature pixel on the mth segmentation line in the image at the jth moment and the horizontal axis, N represents the number of feature pixels corresponding to the segmentation line, It represents the absolute value of the cosine of the angle between the mth segmentation line and the horizontal axis, and M represents the total number of segmentation lines in the image at the jth moment.

[0061] The motion vector of the feature pixel point has an angle with the segmentation line, indicating that the moving direction of the feature pixel point is different from the direction of the segmentation line, indicating that the feature pixel point is likely to be an obstacle. The larger the value is, the more chaotically distributed characteristic pixels are in the image, indicating that the possibility of obstacles in the path is greater, and the closer the monorail crane is to the obstacles.

[0062] The larger the value is, the greater the difference in the direction of movement of the characteristic pixel points on each tangent line is, indicating that the possibility of an obstacle in front of the crane is greater; The larger the value is, the greater the difference in the movement speeds of the feature pixels of the same category is, the more chaotic the distribution of the feature vectors in the image is, and the greater the possibility that there is an obstacle in front of the crane. The larger the value is, the closer the obstacle in front of the crane is to the crane. The larger the value of , the greater the probability that the monorail locomotive will collide with an obstacle in the future.

[0063] S5: taking the difference between the probability of a collision between the monorail crane vehicles at two adjacent moments as a probability variation, and issuing a collision warning in response to the probability variation being greater than 0.

[0064] The expression of the probability change is: , where F represents the change in possibility, represents the possibility of a crane-car collision at time j+2, It represents the possibility of crane vehicle collision at the j+1th moment.

[0065] When the possibility change is less than or equal to 0, it means that the obstacle is moving away from the monorail locomotive's travel path, and the probability of the obstacle colliding with the monorail locomotive is gradually decreasing. At this time, no warning is required. When the possibility change is greater than 0, it means that the obstacle is still trapped in the monorail locomotive's travel path, and the probability of the obstacle colliding with the monorail locomotive is gradually increasing. It is necessary to issue a warning to prompt the operator to reduce the speed of the monorail locomotive or stop driving.

[0066] An embodiment of the present invention further discloses a monorail crane locomotive anti-collision intelligent warning system, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a monorail crane locomotive anti-collision intelligent warning method according to the present invention is implemented.

[0067] The above system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface, and their configuration and functions are known in the art, so they will not be described in detail here.

[0068] In the present invention, the aforementioned memory may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus or device. For example, a computer-readable storage medium may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high bandwidth memory HBM (High Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc., or any other medium that can be used to store the required information and can be accessed by an application, a module or both. Any such computer storage medium may be part of a device or accessible or connectable to a device.

[0069] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

[0070] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A monorail crane locomotive anti-collision intelligent early warning method, characterized in that: Includes steps: Acquire multiple images of the monorail locomotive travel path, construct multiple segmentation lines for each image, and use the pixel points corresponding to the segmentation lines as feature pixel points; The Lucas-Kanade algorithm is used to calculate the optical flow field of two adjacent images, extract the optical flow of each feature pixel and calculate the motion vector; the feature pixels with the same distance from the center of the image are regarded as the same category to classify the feature pixels; Calculate the probability of a collision between the monorail crane and the locomotive at each moment, and take the difference between the probability of a collision between the monorail crane and the locomotive at two adjacent moments as the probability variation; In response to the possibility change being greater than 0, issuing a collision warning; Among them, the expression of the possibility of locomotive collision is: In the formula, represents the possibility of a crane-car collision at the j+1th moment, represents the preset correlation coefficient, Indicates the preset collision index; represents the cosine value of the angle between the motion vector of the nth feature pixel on the mth segmentation line in the image at the jth moment and the horizontal axis, N represents the number of feature pixels corresponding to the segmentation line, represents the absolute value of the cosine of the angle between the mth segmentation line and the horizontal axis, and M represents the total number of segmentation lines in the image at the jth moment; The method further includes: calculating the motion speed of the feature pixel point according to the motion vector, and the expression of the motion speed is: In the formula, Represents the motion vector of the feature pixel, Indicates the horizontal moving speed of the i-th feature pixel in the image. Indicates the vertical moving speed of the i-th feature pixel in the image. represents the moving speed of the i-th feature pixel in the image at the j-th moment, It represents the horizontal moving speed of the i-th feature pixel in the image at the j-th moment. Indicates the moving speed of the i-th feature pixel in the image at the j-th moment in the vertical direction; The expression of the collision index is: In the formula, Indicates the kth feature pixel in the image at the jth moment The speed corresponding to the feature pixel point, L represents the total number of the k-th type of feature pixel points in the image at the j-th moment, represents the average motion speed of the k-th feature pixel in the image at the j-th moment, K represents the number of feature pixel categories in the image at the j-th moment, Indicates the distance between the k-th feature pixel in the image at the j-th moment and the center of the circle; The expression of the correlation coefficient is: In the formula, It represents the number of feature pixels whose motion vectors of feature pixels in the image at the jth moment have an angle with the segmentation line. Represents the total number of feature pixels in the image at the jth moment.

2. The method for intelligent anti-collision warning of a monorail crane locomotive according to claim 1 is characterized in that: The construction method of the dividing line is: A coordinate system is established with the center of the image as the origin, the horizontal direction as the X-axis, and the vertical direction as the Y-axis to obtain the position coordinates of each pixel point in the coordinate system; segmentation lines are evenly drawn with the origin as the center, and the angles between adjacent segmentation lines are the same.

3. The monorail crane locomotive anti-collision intelligent early warning method according to claim 1 is characterized in that: Use the atan2 function to calculate the angle between the motion vector and the horizontal direction.

4. The monorail crane locomotive anti-collision intelligent early warning method according to claim 1 is characterized in that: After acquiring the image of the monorail locomotive travel path, the method also includes the steps of graying and denoising the image.

5. A monorail crane locomotive anti-collision intelligent warning system, characterized in that: include: A processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a monorail crane locomotive anti-collision intelligent warning method according to any one of claims 1-4 is implemented.

Images