Judgment Method, Device, Equipment and Container Crane for Lifting Cable Out of Frame

Abstract

The present application provides a method, device, equipment and container crane for judging whether a spreader cable exits a frame. By means of a camera disposed on a trolley for controlling the movement of the spreader, a to-be-processed image including the spreader cable and a cable slot is acquired. The contour of the spreader cable and the contour of the cable slot in the to-be-processed image are identified to obtain the position of the contour of the spreader cable and the position of the contour of the cable slot. And based on the position of the contour of the spreader cable and the position of the contour of the cable slot, it is judged whether the spreader cable exits the frame, so as to realize the judgment of whether the spreader cable exits the frame without manual participation and solve the problem that it is labor-consuming to manually confirm whether the spreader cable exits the frame.

Description

Technical Field

[0001] This application relates to the technical field of spreaders, and particularly relates to a method, device, equipment and container crane for judging whether the spreader cable is out of the frame. Background Art

[0002] When the spreader cable is out of the frame, that is, it is out of the cable groove, it may affect the normal operation of the spreader and even pose a safety problem. In the prior art, whether the spreader cable is out of the frame is usually confirmed manually, which is labor-consuming. How to automatically judge whether the spreader cable is out of the frame has become an urgent problem to be solved. Summary of the Invention

[0003] Based on the above defects and deficiencies of the prior art, this application proposes a method, device, equipment and container crane for judging whether the spreader cable is out of the frame, which can judge whether the spreader cable is out of the frame based on the positions of the contour of the spreader cable and the contour of the cable groove recognized from the image, without manual participation, and solves the problem of labor consumption for manually confirming whether the spreader cable is out of the frame.

[0004] According to the first aspect of the embodiments of this application, a method for judging whether the spreader cable is out of the frame is provided, including:

[0005] Obtain a to-be-processed image through a camera arranged on the trolley, where the to-be-processed image includes a spreader cable and a cable groove;

[0006] Identify the contour of the spreader cable and the contour of the cable groove in the to-be-processed image to obtain the position of the contour of the spreader cable and the position of the contour of the cable groove;

[0007] Based on the position of the contour of the spreader cable and the position of the contour of the cable groove, judge whether the spreader cable is out of the frame.

[0008] According to the second aspect of the embodiments of this application, a device for judging whether the spreader cable is out of the frame is provided, including:

[0009] An acquisition module, configured to obtain a to-be-processed image through a camera arranged on the trolley, where the to-be-processed image includes a spreader cable and a cable groove;

[0010] An identification module, configured to identify the contour of the spreader cable and the contour of the cable groove in the to-be-processed image to obtain the position of the contour of the spreader cable and the position of the contour of the cable groove;

[0011] A judgment module, configured to judge whether the spreader cable is out of the frame based on the position of the contour of the spreader cable and the position of the contour of the cable groove.

[0012] According to the third aspect of the embodiments of this application, an electronic device is provided, including a memory and a processor;

[0013] The memory is connected to the processor and is used to store programs;

[0014] The processor is configured to implement the method for judging whether the spreader cable is out of the frame as described in the first aspect by running the program in the memory.

[0015] According to a fourth aspect of the embodiments of the present application, a storage medium is provided. A computer program is stored on the storage medium. When the computer program is run by a processor, the method for judging whether the spreader cable is out of the frame as described in the first aspect is implemented.

[0016] According to a fifth aspect of the embodiments of the present application, a quay crane is provided. The quay crane is provided with the judging device for whether the spreader cable is out of the frame as described in the second aspect or the electronic device as described in the third aspect.

[0017] In the above method, device, equipment for judging whether the spreader cable is out of the frame and the container crane, by using a camera disposed on a trolley for controlling the movement of the spreader, a to-be-processed image including the spreader cable and the cable groove is acquired, the contour of the spreader cable and the contour of the cable groove in the to-be-processed image are identified, the position of the contour of the spreader cable and the position of the contour of the cable groove are obtained, and based on the position of the contour of the spreader cable and the position of the contour of the cable groove, it is judged whether the spreader cable is out of the frame, so as to realize the judgment of whether the spreader cable is out of the frame without manual participation, and solve the problem that it is labor-consuming to manually confirm whether the spreader cable is out of the frame. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without creative efforts.

[0019] Figure 1 It is a schematic flow chart of a method for judging whether a spreader cable is out of the frame according to an embodiment of the present application;

[0020] Figure 2 It is a schematic diagram of the installation position of a camera according to an embodiment of the present application;

[0021] Figure 3 It is a schematic diagram of an image coordinate system according to an embodiment of the present application;

[0022] Figure 4 It is a schematic diagram of the contours of a spreader cable and a cable groove according to an embodiment of the present application;

[0023] Figure 5Schematic diagram of a judgment process for a sling cable out of the frame proposed in an embodiment of the present application;

[0024] Figure 6 Schematic diagram of the structure of a judgment device for a sling cable out of the frame proposed in an embodiment of the present application;

[0025] Figure 7 Schematic diagram of the structure of an electronic device proposed in an embodiment of the present application. Detailed implementation manners

[0026] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0027] Overview

[0028] As described in the background art, when the sling cable comes out of the frame, that is, it detaches from the wire groove, it may affect the normal operation of the sling and even pose a safety problem. In the prior art, whether the sling cable is out of the frame is usually confirmed manually, which is labor-consuming. How to automatically judge whether the sling cable is out of the frame has become an urgent problem to be solved.

[0029] On this basis, the inventor further found through research that by using a camera installed on a trolley for controlling the movement of the sling, an image to be processed including the sling cable and the wire groove is obtained, the contours of the sling cable and the wire groove in the image to be processed are identified, the positions of the contour of the sling cable and the position of the contour of the wire groove are obtained, and based on the positions of the contour of the sling cable and the position of the contour of the wire groove, it is judged whether the sling cable is out of the frame, so as to realize the judgment of whether the sling cable is out of the frame without manual participation and solve the problem of labor consumption in manually confirming whether the sling cable is out of the frame.

[0030] Based on the above concept, the embodiments of this specification provide a method for judging whether a sling cable is out of the frame. Next, the method for judging whether a sling cable is out of the frame will be described exemplarily in conjunction with the accompanying drawings.

[0031] Exemplary Method

[0032] Please refer to Figure 1 , in an exemplary embodiment, a method for judging whether a sling cable is out of the frame is provided, which is applied to any electronic device that can obtain an image to be processed. As Figure 1 shown, the method includes steps S101 - S103:

[0033] S101: Obtain the image to be processed through a camera installed on the trolley.

[0034] Among them, the trolley is used to control the movement of the spreader.

[0035] In addition, the image to be processed includes spreader cables and cable troughs.

[0036] Optionally, the camera can be installed on the side of the trolley.

[0037] Specifically, the angle of the camera can be rotated.

[0038] Exemplarily, the camera installed on the trolley can be a dome camera. Since the field of view angle of the dome camera is relatively large, generally 360°, it can well ensure that the states of the spreader cables and cable troughs can be observed through this camera.

[0039] Specifically, after the operation signal is sent, the spreader starts to work. At this time, the image to be processed is obtained through the camera installed on the trolley. Among them, the operation signal is used to control the spreader to start working, such as starting to rise or starting to fall.

[0040] Among them, the operation signal can be sent by the control unit in the electronic device.

[0041] S102: Identify the contours of the spreader cables and the cable troughs in the image to be processed, and obtain the positions of the contours of the spreader cables and the positions of the contours of the cable troughs.

[0042] Specifically, using the segmentation algorithm of deep learning, extract the edge area of the spreader cables in the image to be processed, that is, the contour of the spreader cables, and obtain the position of the contour of the spreader cables.

[0043] Similarly, using the segmentation algorithm of deep learning, extract the edge contour of the cable trough in the image to be processed, that is, the contour of the spreader cables, and obtain the position of the contour of the cable trough.

[0044] S103: Based on the positions of the contours of the spreader cables and the positions of the contours of the cable troughs, determine whether the spreader cables are out of the frame.

[0045] Optionally, based on the positions of the contours of the spreader cables and the positions of the contours of the cable troughs, determine the positional relationship between the contour of the spreader cables and the contour of the cable troughs, and based on this positional relationship, determine whether the spreader cables are out of the frame.

[0046] Among them, the positional relationship can be whether the contour of the spreader cables is located within the contour of the cable trough.

[0047] Specifically, when determining whether the spreader cables are out of the frame based on the positional relationship, if the contour of the spreader cables is located within the contour of the cable trough, it is determined that the spreader cables are out of the frame; if the contour of the spreader cables is located within the contour of the cable trough, it is determined that the spreader cables are not out of the frame.

[0048] In this embodiment, a camera is provided on a trolley for controlling the movement of a spreader to obtain a to-be-processed image including a spreader cable and a cable duct, identify the contours of the spreader cable and the cable duct in the to-be-processed image, obtain the positions of the contour of the spreader cable and the position of the contour of the cable duct, and determine whether the spreader cable is out of the frame based on the positions of the contour of the spreader cable and the position of the contour of the cable duct, so as to realize the judgment of whether the spreader cable is out of the frame without manual participation, and solve the problem of consuming manpower for manually confirming whether the spreader cable is out of the frame.

[0049] Since the position of an object can be represented by coordinates in a coordinate system, the positions of the contour of the spreader cable and the position of the contour of the cable duct can also be represented by coordinates.

[0050] In some embodiments, when identifying the contours of the spreader cable and the cable duct in the to-be-processed image to obtain the positions of the contour of the spreader cable and the position of the contour of the cable duct, an image coordinate system is established for the to-be-processed image, the contour of the spreader cable is extracted from the to-be-processed image, and the coordinates of the contour of the spreader cable in the image coordinate system are determined. The contour of the cable duct is extracted from the to-be-processed image, and the coordinates of the contour of the cable duct in the image coordinate system are determined.

[0051] Among them, the image coordinate system is generally established with the lower left corner of the image as the origin. Based on different actual requirements, other positions can also be selected as the origin.

[0052] Generally, when establishing the image coordinate system, the contour of the spreader cable and the contour of the cable duct are placed in the first quadrant of the coordinate system, that is, the values of the coordinates of the contour of the cable duct in the image coordinate system and the coordinates of the contour of the spreader cable in the image coordinate system are both positive.

[0053] Specifically, the diameter direction of the cable duct contour is used as the horizontal axis, and the direction perpendicular to this diameter direction is used as the vertical axis.

[0054] Specifically, a deep learning algorithm is used to identify the cable duct and the spreader cable in the to-be-processed image, a segmentation algorithm is used to extract the contour of the cable duct, and at the same time, the contour of the spreader cable is extracted.

[0055] In this embodiment, an image coordinate system is established in the to-be-processed image, and the positioning of the contour of the spreader cable and the positioning of the contour of the cable duct are realized based on the coordinates of the image coordinate system, and the coordinates of the contour of the spreader cable and the coordinates of the contour of the cable duct are obtained. In this way, the positions of the spreader cable and the cable duct can be observed more intuitively.

[0056] Since there may be many lines in the wire duct, which are rather messy and difficult to identify and extract, when extracting the contour of the wire duct, only part of the contour in the wire duct can be extracted to reduce the difficulty of image recognition and extraction and improve the efficiency and accuracy of recognition and extraction. In some embodiments, when extracting the contour of the wire duct from the image to be processed and determining the coordinates of the contour of the wire duct in the image coordinate system, the maximum contour and the minimum contour of the wire duct are extracted and located.

[0057] Optionally, the maximum contour of the wire duct is extracted from the image to be processed, and the coordinates of the maximum contour of the wire duct in the image coordinate system are determined. The minimum contour of the wire duct is extracted from the image to be processed, and the coordinates of the minimum contour of the wire duct in the image coordinate system are determined.

[0058] Specifically, a segmentation algorithm is used to extract the maximum and minimum contours of the wire duct from the image to be processed.

[0059] Specifically, the range of the maximum diameter (i.e., the diameter of the top circle of the wire duct) and the minimum diameter (i.e., the diameter of the bottom circle of the wire duct) in the contour of the wire duct is extracted to obtain the maximum contour and the minimum contour of the wire duct.

[0060] Exemplarily, the positional relationship between the contour of the sling cable, the contour of the wire duct, and the image coordinate system can be as Figure 3 shown. Among them, the solid circle represents the maximum contour of the wire duct, the dashed circle represents the minimum contour of the wire duct, and the thicker solid curve represents the sling cable. The contours of the wire duct and the sling cable are both located in the first quadrant of the image coordinate system.

[0061] In this embodiment, only the maximum contour and the minimum contour in the contour of the wire duct are extracted, which can reduce the workload of contour extraction, improve the contour extraction efficiency, and further improve the judgment efficiency.

[0062] In some embodiments, based on the position of the contour of the sling cable and the position of the contour of the wire duct, it is determined whether the sling cable is out of the frame, that is, based on the coordinates of the contour of the target cable and the coordinates of the contour of the wire duct, it is determined whether the sling cable is out of the frame.

[0063] At this time, first, based on the coordinates of the maximum contour of the wire duct in the image coordinate system and the coordinates of the minimum contour of the wire duct in the image coordinate system, the target abscissa interval corresponding to each ordinate in the target ordinate interval is determined. Then, based on the coordinates of the contour of the sling cable and the target abscissa interval, it is determined whether the sling cable is out of the frame.

[0064] Among them, the upper limit of the target ordinate interval is the ordinate of the maximum contour of the wire duct, and the lower limit of the target ordinate interval is the ordinate of the minimum contour of the wire duct.

[0065] Exemplarily, the ordinate of the maximum contour of the wire groove is y1, and the ordinate of the minimum contour of the wire groove is y2. Then, the upper limit of the target ordinate interval is determined as y1, and the lower limit of the target ordinate interval is determined as y2. At this time, the target ordinate interval can be expressed as [y2, y1].

[0066] Among them, the ordinate of the maximum contour of the wire groove, that is, the ordinate when the abscissa of the maximum contour of the wire groove is the smallest / largest in the image coordinate system.

[0067] Similarly, the ordinate of the minimum contour of the wire groove, that is, the ordinate when the abscissa of the minimum contour of the wire groove is the smallest / largest in the image coordinate system.

[0068] It can be understood that the target abscissa interval corresponding to each ordinate within the above target ordinate interval, that is, the maximum and minimum values of the abscissa of the contour of the wire groove at this ordinate in the image coordinate system. Only when the abscissa of the contour of the spreader cable at this ordinate does not exceed the maximum value and the minimum value, can it be ensured that the spreader cable has not separated from the wire groove, that is, the spreader cable has not gone out of the frame.

[0069] In this embodiment, based on the coordinates of the maximum and minimum contours of the wire groove in the image coordinate system, the target abscissa interval corresponding to each ordinate within the ordinate (or height) interval corresponding to the wire groove, that is, the target ordinate interval, is determined. Since the target abscissa interval represents the maximum and minimum abscissas of the contour of the wire groove at each ordinate, based on the coordinates of the contour of the spreader cable and the target abscissa interval, it can be accurately judged how the spreader cable goes out of the frame.

[0070] In some embodiments, based on the coordinates of the maximum contour of the wire groove in the image coordinate system and the coordinates of the minimum contour of the wire groove in the image coordinate system, determining the target abscissa interval corresponding to each ordinate within the target ordinate interval includes:

[0071] Connect the first coordinate and the second coordinate to obtain the first connection line, and connect the third coordinate and the fourth coordinate to obtain the second connection line. Then, the abscissas of the first connection line and the second connection line at the target ordinate are respectively determined as the upper and lower limits of the target abscissa interval corresponding to the target ordinate, so as to obtain the target abscissa interval corresponding to the target ordinate.

[0072] Among them, the first coordinate is the coordinate when the abscissa of the maximum contour of the wire groove is the largest in the image coordinate system, the second coordinate is the coordinate when the abscissa of the minimum contour of the wire groove is the largest in the image coordinate system, the third coordinate is the coordinate when the abscissa of the maximum contour of the wire groove is the smallest in the image coordinate system, and the fourth coordinate is the coordinate when the abscissa of the minimum contour of the wire groove is the smallest in the image coordinate system.

[0073] In addition, the target ordinate is any ordinate within the target ordinate interval.

[0074] That is, for each height of the wire groove, determine the maximum abscissa and the minimum abscissa of the contour of the wire groove in the image coordinate system. And determine the maximum and minimum abscissas as the target abscissa interval at this height.

[0075] Among them, the target abscissa interval is the value range of the abscissa when the sling cable is located in the wire groove.

[0076] Specifically, the coordinates of each point on the first connecting line and the second connecting line can be determined by calculation. For example, based on the first and second coordinates, determine the slope of the first connecting line, and then determine the linear equation of the first connecting line, so as to determine the coordinates of each point on the first connecting line, determine the abscissas of the points corresponding to different ordinates on the first connecting line, and obtain the lower limit values of the target abscissa intervals corresponding to different ordinates within the target ordinate interval. Similarly, in the same way, the upper limit values of the target abscissa intervals corresponding to different ordinates within the target ordinate interval can be determined.

[0077] Through this embodiment, the target abscissa interval corresponding to each ordinate within the target ordinate interval can be determined, that is, the maximum value and the minimum value of the abscissa of the contour of the wire groove in the image coordinate system at this ordinate. In this way, based on the target abscissa interval and the coordinates of the sling cable, it is possible to more accurately determine whether the sling cable is out of the frame, achieving the effect of automatically judging whether the sling cable is out of the frame and reducing labor consumption.

[0078] In some embodiments, when judging whether the sling cable is out of the frame based on the coordinates of the contour of the sling cable and the target abscissa interval, for each ordinate within the target ordinate interval, judge whether the abscissa of the sling cable is within the target abscissa interval corresponding to this ordinate, and then determine whether the sling cable is out of the frame.

[0079] Among them, if for each ordinate within the target ordinate interval, the abscissa of the contour of the sling cable is within the corresponding target abscissa interval, it is determined that the sling cable is not out of the frame. If for any ordinate within the target ordinate interval, the abscissa of the contour of the sling cable is outside the corresponding target abscissa interval, it is determined that the sling cable is out of the frame.

[0080] Exemplarily, taking the target ordinate interval including y1 and y2 as an example, the target abscissa interval corresponding to y1 is [x11, x12], the target abscissa interval corresponding to y2 is [x21, x22], the abscissa of the sling cable corresponding to y1 is x1, and the abscissa of the sling cable corresponding to y2 is x2. At this time, if x11 < x1 < x12 and x21 < x2 < x22, it is determined that the sling cable is not out of the frame. If x11 > x1 or x1 > x12 or x21 > x2 or x2 > x22, it is determined that the sling cable is out of the frame.

[0081] In this embodiment, based on whether the abscissa of the contour of the sling cable at each ordinate within the target ordinate interval is located within the corresponding target abscissa interval, it is determined whether the sling cable is out of the frame. Since whether the abscissa of the contour of the sling cable is located within the corresponding target abscissa interval can represent whether the contour of the sling cable is within the contour of the wire groove, therefore, based on whether the abscissa of the contour of the sling cable is located within the corresponding target abscissa interval, it can be more accurately determined whether the sling cable is out of the frame.

[0082] In some cases, due to the limited viewing angle of a single camera, there will be a blind spot in the field of view when observing the sling cable and the wire groove. When determining the position of the contour of the sling cable and the position of the contour of the wire groove based on the image collected by the single camera, and then judging whether the sling is out of the frame, there may be errors, which will affect the accuracy of the judgment.

[0083] Exemplarily, when a single camera observes the sling cable and the wire groove and there is no blind spot in the field of view, the contour of the sling cable and the contour of the wire groove in the to-be-processed image collected from the single camera can be as Figure 4 shown in a or b of Figure 4 where the sling cable is outside the wire groove. At this time, based on

[0084] the contour of the sling cable and the contour of the wire groove in a or b of Figure 4 to make an out-of-frame judgment, it can be determined that the sling cable is in the out-of-frame state. Figure 4 Exemplarily, when a single camera observes the sling cable and the wire groove and there is a blind spot in the field of view, that is, for this single camera, the sling cable is behind the wire groove. At this time, the contour of the sling cable and the contour of the wire groove in the to-be-processed image collected from the single camera can be as

[0085] shown in c or d of Figure 4 In this way, when making an out-of-frame judgment based on the contour of the sling cable and the contour of the wire groove in c or d of

[0086] Figure 4 there may be errors and it cannot be accurately determined whether the cable is out of the frame. Among them, the arrows in c and d are used to represent the viewing direction of this single camera. In c, this single camera is located to the right of the contour of the sling cable and the wire groove, and in d, this single camera is located below the contour of the sling cable and the wire groove.

[0085] At this time, in order to accurately judge whether the sling cable is out of the frame, image recognition can be performed by combining the to-be-processed images collected by cameras with different viewing angles, and it can be determined whether the sling cable is out of the frame. Among them, the contours of the sling cable and the wire groove in the images collected by the cameras with different viewing angles can be as Figure 4 shown in e and f of

[0086] Figure 4 The arrows in e and f represent the viewing direction of the camera. In e, the camera that collects the image is located to the left of the sling cable and the wire groove in e, and in f, the camera that collects the image is located above the sling cable and the wire groove in e.

[0086] Therefore, in some embodiments, the number of cameras disposed on the trolley is multiple.

[0087] These multiple cameras cooperate with each other to achieve 360° observation of the spreader cable and the cable duct, so as to ensure that the images of each perspective collected based on these multiple cameras can be used to accurately judge whether the spreader cable is out of the frame.

[0088] Specifically, the projections of these multiple cameras on the target plane surround the cable duct, where the target plane is the horizontal plane where the cable duct is located.

[0089] Specifically, the number of cameras is 3 or more.

[0090] Generally, when the number of cameras is 3, these 3 cameras are respectively disposed on any three sides of the trolley. When the number of cameras is 4, these 4 cameras are respectively disposed on the 4 sides of the trolley. At this time, 1 camera is disposed on each side of the trolley.

[0091] Specifically, the cameras can be disposed at a position near the lower part of the side surface of the trolley, or on the edge where the side surface and the lower surface of the trolley meet, or at a position on the lower surface of the trolley close to the side surface.

[0092] Exemplarily, taking the number of cameras being 4 as an example, the installation positions of the cameras can be as Figure 2 shown. There is a trolley for controlling the movement of the spreader installed on the yard crane. One camera is respectively disposed on three sides of the trolley, that is, 4 cameras are disposed on the trolley. These 4 cameras are respectively camera 1, camera 2, camera 3, and camera 4.

[0093] Specifically, the number of cameras can be determined based on the viewing angle parameters of the cameras, as long as it can ensure 360° observation of the spreader cable and the cable duct.

[0094] Specifically, one of these multiple cameras is the main observation camera. Exemplarily, taking Figure 2 as an example, camera 1 is the main observation camera.

[0095] Exemplarily, the above cameras can be dome cameras.

[0096] It can be understood that disposing cameras on any three side edges of the trolley can effectively ensure the total viewing field range of the cameras and can obtain images of each surface of the cable duct and the spreader cable in real time.

[0097] Optionally, when obtaining the to-be-processed images through the cameras disposed on the trolley, for each camera, image acquisition is performed on the spreader cable and the cable duct to obtain multiple to-be-processed images.

[0098] Specifically, the number of times of image acquisition can be at least once, and at least one image to be processed can be acquired through each camera.

[0099] Specifically, through each camera, real-time video acquisition is performed on the spreader cable and the cable trough to obtain a video to be processed, and each image frame in the video to be processed is extracted as an image to be processed.

[0100] In this embodiment, through multiple cameras arranged on each side of the trolley, images to be processed in all directions of the spreader cable and the cable trough can be acquired. Based on the positions of the contours of the spreader cable and the cable trough recognized from the images to be processed obtained by each camera, a comprehensive judgment on whether the spreader cable is out of the frame is realized, thereby effectively improving the accuracy of the judgment.

[0101] It should be understood that after the images to be processed are obtained through multiple cameras, for each image to be processed, it is judged whether the contour of the spreader cable is located within the contour of the cable trough according to the above embodiment, and then it is determined whether the spreader cable is out of the frame. If for each image to be processed, it is determined that the contour of the spreader cable is located within the contour of the cable trough, it is determined that the spreader cable is not out of the frame; if for any image to be processed, it is determined that the contour of the spreader cable is not located within the contour of the cable trough, it is determined that the spreader cable is out of the frame.

[0102] Exemplarily, the judgment process for the spreader cable to be out of the frame can be as Figure 5 Taking it as an example, after the spreader starts to work, through deep learning, for Camera 1, Camera 2, and Camera 3, the cable trough and the spreader cable in the acquired images to be processed are recognized, the contour of the cable trough and the contour of the spreader cable are extracted, and it is judged whether the contour of the spreader cable is located within the contour of the cable trough. If not, it is determined that the spreader cable is out of the frame. If for Camera 1, Camera 2, and Camera 3, it is judged that the contour of the spreader cable is located within the contour of the cable trough, it is determined that the spreader cable is not out of the frame.

[0103] Exemplarily, the control unit of the electronic device issues a job signal to control the spreader to start working, and sends the real-time video frames obtained by the acquisition unit in the electronic device through the camera installed on the trolley to the detection unit in the electronic device. Then, the detection unit uses a deep learning algorithm to identify the wire grooves and cables in each frame of the picture, and uses a segmentation algorithm to extract the range of the maximum diameter (top circle) and the minimum diameter (bottom circle) of the contour of the wire groove, obtaining the maximum contour and the minimum contour of the wire groove. At the same time, the contour of the spreader cable is extracted. Calculate whether the abscissa of the contour of the spreader cable in the picture frame exceeds the range between the maximum and minimum abscissas of the contour of the wire groove at the same height, that is, the same ordinate (i.e., whether it is within this range). If it exceeds this range, the spreader cable is out of the frame. At the same time, for the video frames collected by the three cameras, it is judged whether the abscissa of the contour of the spreader cable in the picture frame exceeds the range between the maximum and minimum abscissas of the contour of the wire groove at the same height, that is, the same ordinate. If it is within the range, it is determined that the spreader cable is within the frame, that is, not out of the frame. If it is judged that it exceeds the range for the video frames collected by any one of the cameras, it is judged that the spreader cable is out of the frame. Finally, the detection unit sends the judgment result, that is, whether the spreader cable is out of the frame, to the control unit so that the control unit can control the spreader to stop or continue working, etc.

[0104] Exemplary Apparatus

[0105] As Figure 6 shown, the embodiment of the present application further provides a judging device for the spreader cable being out of the frame, including an obtaining module 601, an identifying module 602 and a judging module 603.

[0106] Among them,

[0107] The obtaining module 601 is configured to obtain a to-be-processed image through a camera installed on the trolley, and the to-be-processed image includes a spreader cable and a wire groove;

[0108] The identifying module 602 is configured to identify the contour of the spreader cable and the contour of the wire groove in the to-be-processed image, and obtain the position of the contour of the spreader cable and the position of the contour of the wire groove;

[0109] The judging module 603 is configured to judge whether the spreader cable is out of the frame based on the position of the contour of the spreader cable and the position of the contour of the wire groove.

[0110] The determination device for the hoist cable out of the frame provided in this embodiment belongs to the same inventive concept as the determination method for the hoist cable out of the frame provided in the above embodiment of the present application. It can execute the method provided in any of the above embodiments of the present application and has the corresponding functional modules and beneficial effects of the execution method. For the technical details not described in detail in this embodiment, reference can be made to the specific processing content of the determination method for the hoist cable out of the frame provided in the above embodiment of the present application, and no further elaboration will be provided here.

[0111] The functions implemented by the above acquisition module 601, recognition module 602, and judgment module 603 can be respectively implemented in the form of software called by the same or different processors, and the embodiments of the present application do not make limitations.

[0112] Exemplary Electronic Device

[0113] Another embodiment of the present application also proposes an electronic device. Refer to Figure 7 As shown, the electronic device includes: a memory 700 and a processor 710.

[0114] Among them, the memory 700 is connected to the processor 710 and is used to store programs.

[0115] The processor 710 is used to implement the determination method for the hoist cable out of the frame disclosed in any of the above embodiments by running the program stored in the memory 700.

[0116] Specifically, the electronic device may further include: a bus, a communication interface 720, an input device 730, and an output device 740.

[0117] The processor 710, the memory 700, the communication interface 720, the input device 730, and the output device 740 are interconnected through the bus. Among them:

[0118] The bus may include a path for transmitting information between various components of the computer system.

[0119] The processor 710 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the solution of the present application. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0120] The processor 710 may include a main processor and may also include a baseband chip, a modem, etc.

[0121] The program for implementing the technical solution of this application is stored in the memory 700. The operating system and other key services may also be stored. Specifically, the program may include program code, and the program code includes computer operation instructions. More specifically, the memory 700 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk memory, a flash memory, etc.

[0122] The input device 730 may include devices for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor, etc.

[0123] The output device 740 may include devices for allowing information to be output to a user, such as a display screen, a printer, a speaker, etc.

[0124] The communication interface 720 may include devices of any transceiver type for communicating with other devices or communication networks, such as Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0125] The processor 710 executes the program stored in the memory 700 and invokes other devices, and can be used to implement each step of any method for determining the out-of-frame of the spreader cable provided in the above embodiments of this application.

[0126] Those skilled in the art can understand that Figure 7 the structure shown in

[0127] merely represents a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the electronic device to which the solution of this application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0128] This application embodiment also proposes a chip, which includes a processor and a data interface. The processor reads and runs the program stored on the memory through the data interface to execute the method for determining the out-of-frame of the spreader cable introduced in any of the above embodiments. The specific processing process and its beneficial effects can be referred to the embodiment introduction of the method for determining the out-of-frame of the spreader cable above.

[0129] This container crane may be, for example, a yard crane, etc.

[0130] In addition to the above methods and devices, embodiments of the present application provide a computer program product, which includes computer program instructions that, when run by a processor, cause the processor to execute the steps in the method for judging the out-of-frame of the sling cable according to various embodiments of the present application described in the "Exemplary Method" section above of this specification.

[0131] The computer program product can be written in any combination of one or more programming languages to execute the program code for the operations of the embodiments of the present application. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0132] In addition, embodiments of the present application also provide a storage medium, on which a computer program is stored, and the computer program is executed by a processor to execute the steps in the method for judging the out-of-frame of the sling cable according to various embodiments of the present application described in the "Exemplary Method" section above of this specification.

[0133] It can be understood that the specific examples herein are only for helping those skilled in the art to better understand the embodiments of this specification, rather than limiting the scope of this specification.

[0134] It can be understood that in various embodiments of this specification, the magnitudes of the sequence numbers of the various processes do not mean the order of execution is prior or subsequent. The order of execution of the various processes should be determined by their functions and internal logics, and should not constitute any limitation to the implementation process of the embodiments of this specification.

[0135] It can be understood that the various embodiments described in this specification can be implemented alone or in combination, and this specification does not limit this.

[0136] Unless otherwise specified, all technical and scientific terms used in the embodiments of this specification have the same meanings as those commonly understood by those skilled in the technical field of this specification. The terms used in this specification are only for the purpose of describing specific embodiments, and are not intended to limit the scope of this specification. The term "and / or" used in this specification includes any and all combinations of one or more related listed items. The singular forms of "a", "above", and "the" used in the embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0137] It can be understood that the processor in the embodiments of this specification can be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method embodiments can be completed by the integrated logic circuit in the hardware of the processor or instructions in the form of software. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this specification. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of this specification can be directly embodied as being executed and completed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0138] It can be understood that the memory in the embodiments of this specification can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0139] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this specification.

[0140] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0141] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0142] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] In addition, in each embodiment of this specification, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0144] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification, in essence, or the part that contributes to the prior art or a part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of this specification. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0145] The above is only the specific embodiment of this specification, but the protection scope of this specification is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this specification and should be covered by the protection scope of this specification. Therefore, the protection scope of this specification should be subject to the protection scope of the claims.

Claims

1. A method for judging whether a sling cable exits a frame, characterized in that, the method includes: obtaining a to-be-processed image through a camera disposed on a trolley, where the to-be-processed image includes a sling cable and a cable groove; identifying the contour of the sling cable and the contour of the cable groove in the to-be-processed image to obtain the position of the contour of the sling cable and the position of the contour of the cable groove, including: for the to-be-processed image, establishing an image coordinate system, extracting the maximum contour and the minimum contour of the cable groove from the to-be-processed image, and determining the coordinates of the maximum contour and the minimum contour of the cable groove in the image coordinate system, extracting the contour of the sling cable from the to-be-processed image, and determining the coordinates of the contour of the sling cable in the image coordinate system; judging whether the sling cable exits the frame based on the position of the contour of the sling cable and the position of the contour of the cable groove, including: based on the coordinates of the maximum contour of the cable groove in the image coordinate system and the coordinates of the minimum contour of the cable groove in the image coordinate system, determining a target horizontal coordinate interval corresponding to each vertical coordinate within a target vertical coordinate interval, where the upper limit of the target vertical coordinate interval is the vertical coordinate of the maximum contour of the cable groove, and the lower limit of the target vertical coordinate interval is the vertical coordinate of the minimum contour of the cable groove; judging whether the sling cable exits the frame based on the coordinates of the contour of the sling cable and the target horizontal coordinate interval.

2. The method for judging whether a sling cable exits a frame according to claim 1, characterized in that, the number of the cameras is multiple, and the projections of the multiple cameras on a target plane surround the cable groove, and the target plane is the horizontal plane where the cable groove is located; the obtaining a to-be-processed image through a camera disposed on a trolley includes: performing image acquisition on the sling cable and the cable groove through each of the cameras to obtain multiple to-be-processed images.

3. The method for judging whether a sling cable exits a frame according to claim 1, characterized in that, determining a target horizontal coordinate interval corresponding to each vertical coordinate within a target vertical coordinate interval based on the coordinates of the maximum contour of the cable groove in the image coordinate system and the coordinates of the minimum contour of the cable groove in the image coordinate system, including: connecting a first coordinate and a second coordinate to obtain a first connection line, where the first coordinate is the coordinate when the horizontal coordinate of the maximum contour of the cable groove in the image coordinate system is the largest, and the second coordinate is the coordinate when the horizontal coordinate of the minimum contour of the cable groove in the image coordinate system is the largest; connecting a third coordinate and a fourth coordinate to obtain a second connection line, where the third coordinate is the coordinate when the horizontal coordinate of the maximum contour of the cable groove in the image coordinate system is the smallest, and the fourth coordinate is the coordinate when the horizontal coordinate of the minimum contour of the cable groove in the image coordinate system is the smallest; determining the horizontal coordinates of the first connection line and the second connection line at a target vertical coordinate as the upper and lower limits of the target horizontal coordinate interval corresponding to the target vertical coordinate, where the target vertical coordinate is any vertical coordinate within the target vertical coordinate interval.

4. The method for judging whether a sling cable exits a frame according to claim 1, Characterized in that, judging whether the spreader cable is out of the frame based on the coordinates of the profile of the spreader cable and the target abscissa interval, includes: if for each ordinate within the target ordinate interval, the abscissa of the profile of the spreader cable is located within the corresponding target abscissa interval, it is determined that the spreader cable is not out of the frame; if for any ordinate within the target ordinate interval, the abscissa of the profile of the spreader cable is located outside the corresponding target abscissa interval, it is determined that the spreader cable is out of the frame.

5. A judging device for whether a spreader cable is out of the frame, Characterized in that, the device includes: an acquisition module, configured to acquire a to-be-processed image through a camera disposed on a trolley, the to-be-processed image including a spreader cable and a cable trough; an identification module, configured to identify the profile of the spreader cable and the profile of the cable trough in the to-be-processed image, to obtain the position of the profile of the spreader cable and the position of the profile of the cable trough, including: for the to-be-processed image, establishing an image coordinate system, extracting the maximum profile and the minimum profile of the cable trough from the to-be-processed image, and determining the coordinates of the maximum profile and the minimum profile of the cable trough in the image coordinate system, extracting the profile of the spreader cable from the to-be-processed image, and determining the coordinates of the profile of the spreader cable in the image coordinate system; a judging module, configured to judge whether the spreader cable is out of the frame based on the position of the profile of the spreader cable and the position of the profile of the cable trough, including: based on the coordinates of the maximum profile of the cable trough in the image coordinate system and the coordinates of the minimum profile of the cable trough in the image coordinate system, determining the target abscissa interval corresponding to each ordinate within the target ordinate interval, the upper limit of the target ordinate interval being the ordinate of the maximum profile of the cable trough, and the lower limit of the target ordinate interval being the ordinate of the minimum profile of the cable trough; judging whether the spreader cable is out of the frame based on the coordinates of the profile of the spreader cable and the target abscissa interval.

6. An electronic device, Characterized in that, it includes a memory and a processor; the memory is connected to the processor and is used for storing programs; the processor is configured to implement the method for judging whether a spreader cable is out of the frame according to any one of claims 1 to 4 by running the programs in the memory.

7. A container crane, Characterized in that, the container crane is provided with the judging device for whether a spreader cable is out of the frame according to claim 5, or the electronic device according to claim 6.

Images