Crown block operation control method and crown block equipment

Abstract

The invention discloses a crown block operation control method and crown block equipment, and the crown block operation control method comprises the steps: obtaining an image collected by visual detection equipment on a crown block in real time in the moving process of the crown block; determining a traffic environment in a certain range in front of the crown block based on an image acquired by visual inspection equipment on the crown block and acquired in real time and information acquired from a scheduling system; whether the crown block where the visual detection equipment is located needs to enter a protection operation mode or not is determined at least according to the passing environment within a certain range in front of the crown block; and controlling the moving speed of the visual inspection equipment based on the real-time distance between the crown block on which the visual inspection equipment is located and another crown block moving in the same direction in front of the crown block determined by the real-time acquired image acquired by the visual inspection equipment when entering a protection operation mode. Compared with traditional radar or laser positioning, the method has higher precision and reliability through a visual scheme.

Description

Technical Field

[0001] The invention relates to the field of automatic material handling systems, in particular to an overhead crane operation control method and an overhead crane device. Background Art

[0002] The OHT (Overhead Hoist Transfer) system, also known as the overhead crane handling system, is widely used in automated factories such as semiconductor manufacturing.

[0003] In an OHT system, there will be multiple overhead cranes running at the same time. Therefore, anti-collision between overhead cranes in the OHT system is very important.

[0004] The commonly used anti-collision method is to perform anti-collision control through radar or laser positioning, but in the OHT system, the radar or laser will be subject to more interference, there will be certain misjudgments, affecting the operating efficiency of the overhead crane handling system. Summary of the invention

[0005] The purpose of the present invention is to solve the above problems existing in the prior art and to provide an overhead crane operation control method and an overhead crane device.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] An overhead crane operation control method, during the movement of an overhead crane, acquires images captured by a visual detection device on the overhead crane in real time, and determines a traffic environment within a certain range in front of the overhead crane based on the images captured by the visual detection device on the overhead crane acquired in real time and information acquired from a dispatching system;

[0008] Determining whether the overhead crane where the visual inspection device is located needs to enter a protective operation mode at least according to the traffic environment within a certain range in front of the overhead crane;

[0009] When entering the protection operation mode, the moving speed of the visual inspection device is controlled based on the real-time distance between the overhead crane where the visual inspection device is located and another overhead crane in front of it moving in the same direction, which is determined based on the image captured by the visual inspection device acquired in real time.

[0010] Preferably, the traffic environment within a certain range in front of the overhead crane is one of the following situations, including no overhead crane, another overhead crane traveling in the same direction as the overhead crane and maintaining a normal moving speed, another overhead crane traveling in the same direction as the overhead crane and decelerating, another overhead crane traveling straight in the same direction as the overhead crane and accelerating, another overhead crane stopped on its moving route, and another overhead crane turning on its moving route.

[0011] Preferably, determining the traffic environment within a certain range in front of the overhead crane based on the images captured by the visual detection equipment on the overhead crane acquired in real time and the information acquired from the dispatching system includes the following process:

[0012] The image acquired by the visual detection device in real time is input into the trained target detection model after being processed in a predetermined manner to determine the situation of the overhead crane within a certain range in front of the overhead crane, wherein the situation of the overhead crane is one of the following: there is no overhead crane, there is another overhead crane traveling in the same direction as the overhead crane, and there is another overhead crane turning on its moving route;

[0013] When it is determined that there is no overhead crane in a certain range in front of the overhead crane or another overhead crane is turning on its moving route, it is determined that the traffic environment in a certain range in front of the overhead crane is that there is no overhead crane or another overhead crane is turning on its moving route;

[0014] When it is determined that the situation of the overhead crane within a certain range in front of the overhead crane is that there is another overhead crane traveling in the same direction as the overhead crane, it is determined based on the speed situation of the other overhead crane obtained from the dispatching system that the traffic environment within the certain range in front of the overhead crane is that there is another overhead crane traveling in the same direction as the overhead crane and is decelerating, or there is another overhead crane traveling straight in the same direction as the overhead crane and is accelerating, or there is another overhead crane stopped on its moving route, or there is another overhead crane traveling straight in the same direction as the overhead crane and maintaining a normal moving speed.

[0015] Preferably, the image collected by the visual inspection device and acquired in real time is subjected to predetermined processing, including:

[0016] Performing image enhancement processing on the images collected by the two cameras respectively to obtain an enhanced right view image and an enhanced left view image;

[0017] Constructing image pyramids corresponding to the enhanced right view image and the enhanced left view image respectively and obtaining a left image feature map and a right image feature map based on the image pyramids;

[0018] Based on the left image feature map and the right image feature map, a preliminary disparity estimation is performed at the lowest resolution layer to obtain a disparity map;

[0019] After compensating the disparity map, a compensated disparity map is obtained.

[0020] Preferably, the image enhancement processing includes the following process: firstly, performing histogram equalization on the image, then performing Laplace operator enhancement, and finally performing Camma correction.

[0021] Preferably, the sequentially compensating the disparity map includes structured light compensation, temperature compensation and vibration compensation.

[0022] Preferably, if it is determined that the traffic environment within a certain range in front of the overhead crane is that there is another overhead crane traveling in the same direction as it and slowing down, another overhead crane stops on its moving route, or another overhead crane is turning on its moving route, it is determined that the overhead crane where the visual inspection device is located needs to enter the protective operation mode.

[0023] Preferably, the overhead crane where the visual inspection device is located receives in real time the running trajectory of the other overhead crane fed back by the dispatching system. If it is determined that the traffic environment within a certain range in front of the overhead crane is that there is another overhead crane traveling in the same direction as the overhead crane and maintaining a normal moving speed, and the running trajectory of the other overhead crane received from the dispatching system is about to turn, it is determined that the overhead crane where the visual inspection device is located needs to enter the protective operation mode.

[0024] Preferably, the overhead crane where the visual inspection device is located receives the running tracks of other overhead cranes fed back by the dispatching system in real time. For example, when it is determined that another overhead crane and the overhead crane where the visual inspection device is located are on two intersecting lines, and the other overhead crane needs to turn at the intersection of the two lines, if it is determined that the overhead crane where the visual inspection device is located is going to go straight through the intersection, then when the visual inspection device on the overhead crane detects the other overhead crane that is going to turn at the intersection, the other overhead crane is stopped, and it is determined that the overhead crane where the visual inspection device is located needs to enter the protection operation mode;

[0025] If it is determined that the overhead crane where the visual inspection device is located is going to turn at the intersection, when the visual inspection device on the overhead crane detects the other overhead cranes that are going to turn at the intersection, the overhead crane where the visual inspection device is located is put into a protective operation mode, and the other overhead cranes are turned and passed through the intersection first.

[0026] Preferably, the real-time distance is determined according to the following formula:

[0027] Z = (b*f) / d;

[0028] Among them, Z is the real-time distance, b is the baseline length, f is the camera focal length, and d is the parallax.

[0029] Preferably, in the protection operation mode, the moving speed of the overhead travelling vehicle where the visual inspection device is located is controlled according to the following process;

[0030] Determining whether the real-time distance is less than the speed reduction distance;

[0031] If not, the overhead travelling crane where the visual inspection device is located is driven at a normal moving speed and repeatedly determines whether the real-time distance reaches the speed reduction distance;

[0032] If yes, the overhead travelling crane where the visual inspection device is located is slowed down and the change of the real-time distance is determined;

[0033] If it is determined that the real-time distance is greater than the speed reduction distance, the overhead travelling crane where the visual inspection device is located is restored to a normal moving speed;

[0034] If it is determined that the real-time distance is between the deceleration distance and the parking distance, the overhead travelling vehicle where the visual inspection device is located continues to decelerate or maintains the current moving speed;

[0035] If it is determined that the real-time distance is less than the parking distance, the moving speed of the overhead crane where the visual inspection device is located is controlled to 0, and it is continued to be determined whether the real-time distance exceeds the parking distance. If so, the overhead crane where the visual inspection device is located is accelerated and the change of the real-time speed is determined.

[0036] An overhead crane device comprises a memory and a processor, wherein the memory stores a program executable by the processor, and when the program is executed, any of the above-mentioned overhead crane operation control methods is implemented.

[0037] The advantages of the technical solution of the present invention are mainly reflected in:

[0038] The present invention uses a visual solution, and the overhead crane can use the image information captured by the camera, combined with image processing technology and algorithms, to achieve accurate recognition and positioning of an overhead crane in front of it. This positioning method has higher accuracy and reliability than traditional radar or laser positioning. The use of a visual solution can also achieve dynamic tracking of moving targets, ensuring that the overhead crane can accurately follow the target path during transportation, reducing offsets and errors. At the same time, corresponding obstacle avoidance processing can be performed based on higher recognition accuracy, thereby ensuring the safety of the transportation process and adopting a step-by-step speed regulation method to be conducive to balancing operating efficiency.

[0039] The visual solution of the present invention can realize multi-dimensional information acquisition, including color, shape, texture, etc. This information helps the overhead crane to perceive the surrounding environment more comprehensively and thus identify obstacles more accurately.

[0040] The present invention combines artificial intelligence technologies such as machine learning and deep learning, and the visual solution can also achieve autonomous learning and evolution, continuously improving its own perception, recognition and decision-making capabilities, making the overhead crane more intelligent.

[0041] The image processing method of the present invention can effectively improve the image quality, thereby creating favorable conditions for model training and subsequent image recognition, and is beneficial to improving the accuracy of image recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a schematic diagram of two image acquisition devices arranged on the overhead travelling crane of the present invention;

[0043] Figure 2It is a process schematic diagram of the method of the present invention;

[0044] Figure 3 is a calculation principle diagram for determining the real-time distance between another overhead crane and the overhead crane where the visual inspection device is located in the present invention;

[0045] Figure 4 It is a schematic diagram of the control process of the protection operation mode in the present invention. DETAILED DESCRIPTION

[0046] The purpose, advantages and features of the present invention will be illustrated and explained by the non-limiting description of the following preferred embodiments. These embodiments are only typical examples of the application of the technical solution of the present invention, and any technical solution formed by equivalent replacement or equivalent transformation falls within the scope of protection claimed by the present invention.

[0047] In the description of the scheme, it should be noted that the terms "center", "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience and simplification of description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0048] Example 1

[0049] The overhead crane operation control method disclosed by the present invention is described below in conjunction with the accompanying drawings. Figure 1 As shown, the method of the present invention is based on setting a visual detection device on the forward side of each overhead crane 100 when it moves. The visual detection device preferably includes two image acquisition devices 200 arranged at the same height. The image acquisition device 200 can be a known camera. The two image acquisition devices 200 simulate the visual effect of human eyes by taking left and right images of the same target, thereby obtaining three-dimensional information of the target. At the same time, a specific filter is installed in front of the lens of the camera to filter out unnecessary light, such as ambient light and infrared light, which is conducive to subsequent image recognition.

[0050] When each overhead crane is performing a handling task, the traffic environment in a certain range in front of each overhead crane is analyzed in real time based on the images collected by the two cameras, so as to accurately control the operation of each overhead crane.

[0051] Specifically, as attached Figure 2As shown, during the movement of an overhead crane, images captured by a visual detection device on the overhead crane are acquired in real time, and the traffic environment within a certain range in front of the overhead crane is determined based on the images captured by the visual detection device on the overhead crane acquired in real time and the information obtained from the dispatching system;

[0052] Determining whether the overhead crane where the visual inspection device is located needs to enter a protective operation mode at least according to the traffic environment within a certain range in front of the overhead crane;

[0053] When entering the protection operation mode, the moving speed of the visual inspection device is controlled based on the real-time distance between the overhead crane where the visual inspection device is located and another overhead crane in front of it moving in the same direction, which is determined based on the image captured by the visual inspection device acquired in real time.

[0054] The traffic environment within a certain range in front of the overhead crane includes no overhead crane, another overhead crane traveling in the same direction as the overhead crane and maintaining a normal moving speed, another overhead crane traveling in the same direction as the overhead crane and slowing down, another overhead crane traveling straight in the same direction as the overhead crane and accelerating, another overhead crane stopping on its moving route, and another overhead crane turning on its moving route. Here, another overhead crane turning on its moving route includes that the other overhead crane has turned from the same moving direction to another moving direction and that the other overhead crane has turned from another moving direction to the same moving direction.

[0055] Determining the traffic environment within a certain range in front of the overhead crane based on the images acquired in real time by the visual detection equipment on the overhead crane and the information acquired from the dispatching system includes the following process:

[0056] The image captured by the visual inspection device acquired in real time is input into the trained target detection model after predetermined processing to determine the situation of the overhead crane within a certain range in front of the overhead crane, the overhead crane situation is one of no overhead crane, another overhead crane traveling in the same direction as the overhead crane, and another overhead crane turning on its moving route, where traveling in the same direction as the overhead crane means that the moving direction of the other overhead crane is the same as the moving direction of the overhead crane where the visual inspection device is located, and its moving route refers to the moving route of the overhead crane where the visual inspection device is located; further, the overhead crane situation may also include whether there is another overhead crane within a certain range in front of the overhead crane, and the track where the other overhead crane is currently located intersects with the track where the overhead crane where the visual inspection device is located is currently located.

[0057] When it is determined that there is no overhead crane in a certain range in front of the overhead crane or another overhead crane is turning on its moving route, it is determined that the traffic environment in a certain range in front of the overhead crane is that there is no overhead crane or another overhead crane is turning on its moving route;

[0058] When it is determined that there is another overhead crane traveling in the same direction as the overhead crane within a certain range in front of the overhead crane, the traffic environment within a certain range in front of the overhead crane is determined based on the speed of the other overhead crane obtained from the dispatching system, whether there is another overhead crane traveling in the same direction as the overhead crane and decelerating, or another overhead crane traveling straight in the same direction as the overhead crane and accelerating, or another overhead crane stopped on its moving route, or another overhead crane traveling straight in the same direction as the overhead crane and maintaining a normal moving speed. The running speed of each overhead crane is fed back to the dispatching system in real time, so that the dispatching system knows the speed of each overhead crane, that is, whether each overhead crane is decelerating, accelerating, stopping, or traveling at a constant speed. The dispatching system also knows the position of each overhead crane and its moving trajectory, so that the dispatching system will feed back the speed of the other overhead crane, the moving trajectory, etc. to the overhead crane where the visual inspection device is located.

[0059] The target detection model can adopt a known SSD model, preferably a VGG16 model, and its specific construction method and training method are known technologies and will not be described in detail here. Regarding the image data required for model training, the present invention first collects images of one overhead crane running a circle in a real scene, and then collects images of two overhead cranes running a circle one after the other, and then performs predetermined processing on the collected images before model training.

[0060] The predetermined processing includes performing image enhancement processing on the images collected by the two cameras to obtain an enhanced right view image and an enhanced left view image; the image enhancement processing includes the following process: first, the image is histogram equalized, then Laplace operator enhancement is performed, and finally Camma correction is performed. The histogram equalization is used to adjust the grayscale distribution of the image, the Laplace operator enhancement strengthens the edge part of the image through the edge detection operator to make the structural information in the image more obvious; the Camma correction adjusts the brightness of the image to make the dark details clearer and the bright part not overexposed.

[0061] Then, the image pyramids corresponding to the enhanced right view image and the enhanced left view image are constructed respectively, and the left image feature map and the right image feature map are obtained based on the image pyramids; specifically, the enhanced right view image and the enhanced left view image are downsampled layer by layer to generate a series of image layers with different resolutions to obtain the image pyramids corresponding to the enhanced right view image and the enhanced left view image. Each image layer is a certain ratio reduction of the previous image layer, so that the image can be analyzed and processed at multiple scales.

[0062] The acquired pyramid images are input into feasible convolutional neural networks (such as SSD models) respectively, and the multi-layer convolutional neural network feature extraction method is used. Through a series of convolutional layers, pooling layers and activation functions, the complex combinations in the image can be automatically learned and extracted, thereby extracting the left image feature map and the right image feature map at different scales. Among them, the convolution layer slides on the image through the convolution kernel (filter) to extract the features of the local area; the pooling layer reduces the size of the feature map through downsampling operations while retaining important feature information; the activation function (such as ReLU) enables the network to learn more complex patterns through nonlinear transformation.

[0063] Based on the left image feature map and the right image feature map, preliminary disparity estimation is performed at the lowest resolution layer to obtain the left image disparity map and the right image disparity map; the disparity estimation here mainly adopts the block matching method, by searching for fixed-size windows in the left image feature map and the right image feature map, finding the window pair with the highest similarity, and calculating the disparity between them.

[0064] The specific calculation is based on the following formula:

[0065]

[0066] Among them, d init (x, y) is the disparity map, d is the disparity value, which represents the displacement difference of the pixel position of the same target on the left image feature map and the right image feature map; w is the window size, which represents the local neighborhood used to calculate the difference in the disparity estimation process, and defines the pixel range considered around each pixel; E is the lowest resolution layer, which represents the lowest resolution layer of the multi-scale pyramid; r is the preset weight coefficient, which ensures that the difference between the feature maps is reasonably processed in the calculation, making the matching process more accurate; F L (E) (x, y) is the image feature at the position with coordinates (x, y) in the left image feature map of the Eth layer; F R (E) (x, y) is the image feature at the position with coordinates (x, y) in the right image feature map of the Eth layer, F R (E) (x, y+d) is the image feature at the position with coordinates (x, y+d) in the right image feature map of the Eth layer.

[0067] The advantage of performing preliminary disparity estimation at the lowest resolution layer is not only its computational efficiency, but also its ability to provide a stable and robust initial disparity map.

[0068] After compensating the left image disparity map and the right image disparity map, the compensated left image disparity map and the right image disparity map are obtained and used as the corrected left and right image pairs; the compensation of the left image disparity map and the right image disparity map includes structured light compensation, temperature compensation and vibration compensation. The structured light compensation is to introduce a structured light pattern of a known pattern into the disparity map, such as a laser grid, stripes or dot matrix. These structured light patterns can form clear marks in the disparity map, and these marks will appear in the left and right image pairs. The temperature compensation is to monitor the ambient temperature around the visual detection equipment in real time through a temperature sensor, combined with the temperature response curve of the camera, and use a digital signal processing method to adjust the grayscale value and color balance of the disparity map. Vibration compensation is to cope with the tiny vibrations and displacements that the camera may suffer during the measurement process. The compensation can monitor the motion state of the camera in real time through a gyroscope and an acceleration sensor, and dynamically correct the image in combination with a known image stabilization algorithm.

[0069] The corrected left and right image pairs are input into the built target detection model for training until the model converges. After the target detection model is trained, the real-time captured images are processed according to the above predetermined processing to obtain corrected left and right images, which are input into the trained target detection model. The target detection model processes the left and right image pairs into left and right regions of interest, adds classification labels to each frame of the image, performs stereo matching on the left and right image regions, and obtains a classification detection result, which is one of the conditions of the overhead crane within a certain range in front of the overhead crane.

[0070] After determining the traffic environment within a certain range in front of the overhead crane, it can be determined whether the overhead crane where the visual inspection device is located should enter the protection operation mode according to different traffic environments. Specifically:

[0071] If it is determined that the traffic environment within a certain range in front of the overhead crane is that there is another overhead crane traveling in the same direction as the overhead crane and is decelerating, another overhead crane is stopped on its moving route, or another overhead crane is turning on its moving route, it is determined that the overhead crane where the visual inspection device is located needs to enter the protection operation mode.

[0072] Furthermore, while determining the traffic environment within a certain range in front of the overhead crane in real time, the overhead crane where the visual inspection device is located also receives in real time the running trajectory of the other overhead crane fed back by the dispatching system. For example, when it is determined that the other overhead crane and the overhead crane where the visual inspection device is located are currently traveling in the same direction and the running trajectory of the other overhead crane is received as about to turn, it is determined that the overhead crane where the visual inspection device is located needs to enter the protective operation mode.

[0073] At the same time, the overhead crane where the visual inspection device is located also receives in real time the running trajectories of other overhead cranes fed back by the dispatching system. The overhead crane where the visual inspection device is located receives in real time the running trajectories of other overhead cranes fed back by the dispatching system. If it is determined that another overhead crane and the overhead crane where the visual inspection device is located are on two intersecting lines, and the other overhead crane needs to turn at the intersection of the two lines, if it is determined that the overhead crane where the visual inspection device is located is going to go straight through the intersection, then when the visual inspection device on the overhead crane detects the other overhead crane that is going to turn at the intersection, the other overhead crane is stopped, and it is determined that the overhead crane where the visual inspection device is located needs to enter the protective operation mode.

[0074] If it is determined that the overhead crane where the visual inspection device is located is going to turn at the intersection, when the visual inspection device on the overhead crane detects the other overhead cranes that are going to turn at the intersection, the overhead crane where the visual inspection device is located is put into a protective operation mode, and the other overhead cranes are turned and passed through the intersection first.

[0075] After determining the protection operation mode, it is necessary to control the operation speed of the overhead crane where the visual inspection device is located according to the real-time distance between the other overhead crane and the overhead crane where the visual inspection device is located. Therefore, it is necessary to determine the real-time distance between the other overhead crane and the overhead crane where the visual inspection device is located. Specifically, based on the compensated disparity map, the binocular visual image is calculated to obtain the real-time distance between the overhead crane where the visual inspection device is located and the other overhead crane in front of it.

[0076] As attached Figure 3 As shown, the real-time distance is determined according to the following formula:

[0077] Z = (b*f) / d;

[0078] Among them, Z is the real-time distance, b is the baseline length, f is the focal length of the camera, d is the parallax, d = x L -x R , assuming that the optical axes of the two cameras are parallel and located on the same horizontal line, the horizontal coordinates of the projection points of the same target point in the left and right images are x L 、x R , the x L 、x R The specific method for determining is a known technology and will not be described in detail here.

[0079] In the protection operation mode, the moving speed of the overhead travelling crane where the visual inspection device is located is controlled according to the following process;

[0080] Determining whether the real-time distance is less than the speed reduction distance;

[0081] If not, the overhead crane where the visual inspection device is located is made to travel at a normal moving speed and repeatedly determine whether the real-time distance reaches the deceleration distance. The deceleration distance can be set as needed. When the real-time distance is lower than the deceleration distance, it indicates that the distance between the overhead crane where the visual inspection device is located and another overhead crane in front of it is lower than the safe distance, and it is necessary to slow down to keep the distance between the overhead crane where the visual inspection device is located and another overhead crane in front of it at a safe distance.

[0082] If yes, the overhead travelling crane where the visual inspection device is located is slowed down and the change of the real-time distance is determined;

[0083] If it is determined that the real-time distance is greater than the speed reduction distance, the overhead travelling crane where the visual inspection device is located is restored to a normal moving speed;

[0084] If it is determined that the real-time distance is between the deceleration distance and the stopping distance, the overhead crane where the visual inspection device is located continues to decelerate or maintain the current moving speed; the stopping distance can be designed as needed and is smaller than the deceleration distance.

[0085] If it is determined that the real-time distance is less than the parking distance, the moving speed of the overhead crane where the visual inspection device is located is controlled to 0, and it is continued to be determined whether the real-time distance exceeds the parking distance. If so, the overhead crane where the visual inspection device is located is accelerated and the change of the real-time speed is determined.

[0086] Example 2

[0087] This embodiment discloses an overhead crane device, including a memory and a processor, wherein the memory stores a program executable by the processor, and when the program is executed, any of the above-mentioned overhead crane operation control methods is implemented.

[0088] There are many implementation methods of the present invention, and all technical solutions formed by equivalent transformation or equivalent transformation fall within the protection scope of the present invention.

Claims

1. A method for controlling the operation of an overhead crane, characterized in that: During the movement of the overhead crane, images captured by the visual detection device on the overhead crane are acquired in real time, and the traffic environment within a certain range in front of the overhead crane is determined based on the images captured by the visual detection device on the overhead crane acquired in real time and the information obtained from the dispatching system; Determining whether the overhead crane where the visual inspection device is located needs to enter a protective operation mode at least according to the traffic environment within a certain range in front of the overhead crane; When entering the protection operation mode, the moving speed of the visual inspection device is controlled based on the real-time distance between the overhead crane where the visual inspection device is located and another overhead crane in front of it moving in the same direction, which is determined based on the image captured by the visual inspection device acquired in real time.

2. The method for controlling the operation of an overhead crane according to claim 1, characterized in that: The traffic environment within a certain range in front of the overhead crane is one of the following situations, including no overhead crane, another overhead crane traveling in the same direction as the overhead crane and maintaining a normal moving speed, another overhead crane traveling in the same direction as the overhead crane and decelerating, another overhead crane traveling straight in the same direction as the overhead crane and accelerating, another overhead crane stopping on its moving route, and another overhead crane turning on its moving route.

3. The method for controlling the operation of an overhead crane according to claim 1, characterized in that: Determining the traffic environment within a certain range in front of the overhead crane based on the images acquired in real time by the visual detection equipment on the overhead crane and the information acquired from the dispatching system includes the following process: The image acquired by the visual detection device in real time is input into the trained target detection model after being processed in a predetermined manner to determine the situation of the overhead crane within a certain range in front of the overhead crane, wherein the situation of the overhead crane is one of the following: there is no overhead crane, there is another overhead crane traveling in the same direction as the overhead crane, and there is another overhead crane turning on its moving route; When it is determined that there is no overhead crane in a certain range in front of the overhead crane or another overhead crane is turning on its moving route, it is determined that the traffic environment in a certain range in front of the overhead crane is that there is no overhead crane or another overhead crane is turning on its moving route; When it is determined that the situation of the overhead crane within a certain range in front of the overhead crane is that there is another overhead crane traveling in the same direction as the overhead crane, it is determined based on the speed situation of the other overhead crane obtained from the dispatching system that the traffic environment within the certain range in front of the overhead crane is that there is another overhead crane traveling in the same direction as the overhead crane and is decelerating, or there is another overhead crane traveling straight in the same direction as the overhead crane and is accelerating, or there is another overhead crane stopped on its moving route, or there is another overhead crane traveling straight in the same direction as the overhead crane and maintaining a normal moving speed.

4. The method for controlling the operation of an overhead crane according to claim 3, characterized in that: The image collected by the visual inspection device and acquired in real time is subjected to predetermined processing, including: Performing image enhancement processing on the images collected by the two cameras respectively to obtain an enhanced right view image and an enhanced left view image; Constructing image pyramids corresponding to the enhanced right view image and the enhanced left view image respectively and obtaining a left image feature map and a right image feature map based on the image pyramids; Based on the left image feature map and the right image feature map, a preliminary disparity estimation is performed at the lowest resolution layer to obtain a disparity map; After compensating the disparity map, a compensated disparity map is obtained.

5. The method for controlling the operation of an overhead crane according to claim 4, characterized in that: The image enhancement process includes the following steps: firstly, performing histogram equalization on the image, then performing Laplace operator enhancement, and finally performing Camma correction.

6. The method for controlling the operation of an overhead crane according to claim 4, characterized in that: The sequentially compensating the disparity map includes structured light compensation, temperature compensation and vibration compensation.

7. The method for controlling the operation of an overhead crane according to claim 2, characterized in that: If it is determined that the traffic environment within a certain range in front of the overhead crane is that there is another overhead crane traveling in the same direction as the overhead crane and is decelerating, another overhead crane is stopped on its moving route, or another overhead crane is turning on its moving route, it is determined that the overhead crane where the visual inspection device is located needs to enter the protection operation mode.

8. The method for controlling the operation of an overhead crane according to claim 1, characterized in that: The overhead crane where the visual inspection device is located receives the running trajectory of the other overhead crane fed back by the dispatching system in real time. If it is determined that the traffic environment within a certain range in front of the overhead crane is that there is another overhead crane traveling in the same direction as the overhead crane and maintaining a normal moving speed, and the running trajectory of the other overhead crane received from the dispatching system is about to turn, it is determined that the overhead crane where the visual inspection device is located needs to enter the protective operation mode.

9. The method for controlling the operation of an overhead travelling crane according to claim 1, characterized in that: The overhead crane where the visual inspection device is located receives the running tracks of other overhead cranes fed back by the dispatching system in real time. For example, when it is determined that another overhead crane and the overhead crane where the visual inspection device is located are on two intersecting lines, and the other overhead crane needs to turn at the intersection of the two lines, if it is determined that the overhead crane where the visual inspection device is located will go straight through the intersection, then when the visual inspection device on the overhead crane detects the other overhead crane that is about to turn at the intersection, the other overhead crane is stopped, and it is determined that the overhead crane where the visual inspection device is located needs to enter the protection operation mode; If it is determined that the overhead crane where the visual inspection device is located is going to turn at the intersection, when the visual inspection device on the overhead crane detects the other overhead cranes that are going to turn at the intersection, the overhead crane where the visual inspection device is located is put into a protective operation mode, and the other overhead cranes are turned and passed through the intersection first.

10. The method for controlling the operation of an overhead crane according to claim 1, characterized in that: The real-time distance is determined according to the following formula: Z = (b*f) / d; Among them, Z is the real-time distance, b is the baseline length, f is the camera focal length, and d is the parallax.

11. The method for controlling the operation of an overhead crane according to any one of claims 1 to 10, characterized in that: In the protection operation mode, the moving speed of the overhead travelling crane where the visual inspection device is located is controlled according to the following process; Determining whether the real-time distance is less than the speed reduction distance; If not, the overhead travelling crane where the visual inspection device is located is driven at a normal moving speed and repeatedly determines whether the real-time distance reaches the speed reduction distance; If yes, the overhead travelling crane where the visual inspection device is located is slowed down and the change of the real-time distance is determined; If it is determined that the real-time distance is greater than the speed reduction distance, the overhead travelling crane where the visual inspection device is located is restored to a normal moving speed; If it is determined that the real-time distance is between the deceleration distance and the parking distance, the overhead travelling vehicle where the visual inspection device is located continues to decelerate or maintains the current moving speed; If it is determined that the real-time distance is less than the parking distance, the moving speed of the overhead crane where the visual inspection device is located is controlled to 0, and it is continued to be determined whether the real-time distance exceeds the parking distance. If so, the overhead crane where the visual inspection device is located is accelerated and the change of the real-time speed is determined.

12. An overhead crane device, comprising a memory and a processor, wherein the memory stores a program executable by the processor, characterized in that: When the program is executed, the overhead crane operation control method as described in any one of claims 1-11 is implemented.

Images