Container operation guiding method and gantry crane

Abstract

The invention provides a container operation guiding method and a field bridge, and relates to the technical field of automation. The method comprises the following steps: acquiring a container image acquired by image acquisition equipment; based on the container image, detecting and identifying actual lock hole coordinates of two lock holes in the width direction of the container in the container image; according to the actual lock hole coordinates, the actual lock hole distance between the two lock holes is determined; determining a pre-calibrated lock hole reference coordinate and a lock hole reference distance of the container; a compensation parameter is determined according to the ratio of the keyhole reference distance to the actual keyhole distance; according to the compensation parameters and the keyhole reference coordinates, determining target keyhole reference coordinates when the container is located at the reference position of operation; and guiding the container to a reference position of operation according to the target keyhole reference coordinate. According to the method, the keyhole positioning accuracy is improved, so that the container can be accurately guided to the working reference position according to the keyhole position.

Description

Technical Field

[0001] The present application relates to the field of automation technology, and in particular to a container operation guidance method and a container yard crane. Background Art

[0002] During automatic loading and unloading of containers, the industrial control machine will guide the container truck so that the container is in the reference position for the operation, so that the crane's lifting device can lift the container in the reference position for operation. Therefore, how to accurately guide the container to the reference position is crucial.

[0003] In the related technology, a camera installed on the horizontal beam of the yard bridge can be used to collect container image data of the container plane below, and the image data can be sent to the industrial computer. After receiving the image data, the industrial computer will use target detection technology or image segmentation technology to detect and identify the container keyhole in the image, and control the container truck to guide the container to the reference position of the operation according to the coordinates of the keyhole. However, the keyhole coordinates determined by the above method are prone to errors, and the container cannot be accurately guided to the reference position according to the keyhole coordinates, which makes the container operation easy to fail and the operation efficiency is low.

[0004] Therefore, there is a need for a solution that can improve the accuracy of keyhole positioning, thereby accurately guiding the container to a reference position for operation according to the keyhole position. Summary of the invention

[0005] The embodiments of the present application provide a method for guiding container operations and a yard crane, which can improve the accuracy of keyhole positioning, thereby accurately guiding the container to a reference position for operation according to the keyhole position.

[0006] In a first aspect, an embodiment of the present application provides a method for guiding container operation, comprising:

[0007] Acquire container images captured by image acquisition equipment;

[0008] Based on the container image, detecting and identifying actual keyhole coordinates of two keyholes in the container width direction in the container image;

[0009] Determining the actual keyhole distance between the two keyholes according to the actual keyhole coordinates;

[0010] Determining the pre-calibrated keyhole reference coordinates and keyhole reference distance of the container;

[0011] Determining a compensation parameter according to a ratio of the keyhole reference distance to the actual keyhole distance;

[0012] Determining the target keyhole reference coordinates when the container is in the reference position of the operation according to the compensation parameters and the keyhole reference coordinates;

[0013] The container is guided to a reference position for operation according to the target keyhole reference coordinates.

[0014] In a possible implementation, the detecting and identifying actual keyhole coordinates of two keyholes in the width direction of the container in the container image based on the container image includes:

[0015] Input the container image into a preset key point detection model to obtain the actual keyhole coordinates of the two keyholes in the width direction of the container;

[0016] The key point detection model is pre-trained using a container sample image including two keyholes in the width direction, and the container sample image includes a sample label, and the sample label is the keyhole coordinates of the two keyholes in the width direction.

[0017] In a possible implementation manner, determining the pre-calibrated keyhole reference coordinates and keyhole reference distance of the container includes:

[0018] Determining a target size of the container according to the container image;

[0019] According to the correspondence between the preset container size and the pre-calibration data, the target pre-calibration data corresponding to the target size is determined, and the target pre-calibration data includes a keyhole reference coordinate and a keyhole reference distance;

[0020] According to the target pre-calibration data, the keyhole reference coordinates and keyhole reference distance of the container pre-calibration are determined.

[0021] In a possible implementation manner, the correspondence between the preset container size and the pre-calibrated data is obtained in the following manner:

[0022] Obtaining reference container images corresponding to containers of various sizes, respectively. For each size of container, the reference container image is an image acquired when the image acquisition device is at a preset height from the container of the size and the container of the size is at a corresponding reference position;

[0023] For each size of container, calibrate the reference container image to obtain the keyhole reference coordinates of two keyholes in the width direction of the container, and determine the keyhole reference distance between the two keyholes according to the keyhole reference coordinates; determine the pre-calibration data corresponding to the container of the size according to the keyhole reference coordinates and the keyhole reference coordinates;

[0024] According to the pre-calibrated data respectively corresponding to the container of each size, the corresponding relationship between the preset container size and the pre-calibrated data is determined.

[0025] In a possible implementation manner, before determining the compensation parameter according to the ratio of the keyhole reference distance to the actual keyhole distance, the method further includes:

[0026] Determining whether the actual keyhole distance is consistent with the keyhole reference distance;

[0027] If not, it is determined that the container is not in the reference position for the operation, and the step of determining the compensation parameter according to the ratio of the keyhole reference distance to the actual keyhole distance is performed;

[0028] If they are consistent, it is determined that the container is already in the reference position for the operation, and the container is not guided.

[0029] In a possible implementation, guiding the container to a reference position for operation according to the target keyhole reference coordinates includes:

[0030] A guidance control instruction is generated according to the target keyhole reference coordinates and the actual keyhole coordinates, and the guidance control instruction is sent to a container truck corresponding to the container, wherein the guidance control instruction is used to instruct the container truck to guide the container to a reference position for the operation.

[0031] In a possible implementation, the image acquisition device includes a first image acquisition device and a second image acquisition device, the first image acquisition device and the second image acquisition device are arranged on the two side beams of the yard bridge relatively, the first image acquisition device is used to acquire a first container image including two keyholes in the width direction of one side of the container, and the second image acquisition device is used to acquire a second container image including two keyholes in the width direction of the other side of the container; the optical axes of the first image acquisition device and the second image acquisition device are both perpendicular to the plane of the container;

[0032] Accordingly, guiding the container to the reference position of the operation according to the target keyhole reference coordinates includes:

[0033] Determining, according to the target keyhole reference coordinates, first target keyhole reference coordinates corresponding to the first container image;

[0034] Determine a second container image corresponding to the first container image, and determine a second target keyhole reference coordinate corresponding to the second container image;

[0035] The container is guided to a reference position for operation according to the first target keyhole reference coordinates and the second target keyhole reference coordinates.

[0036] In a possible implementation manner, guiding the container to a reference position for operation according to the first target keyhole reference coordinates and the second target keyhole reference coordinates includes:

[0037] Determine a first actual keyhole coordinate corresponding to the first container image, and a second actual keyhole coordinate corresponding to the second container image;

[0038] Determining whether there is a keyhole positioning error on the container according to the first actual keyhole coordinate and the first target keyhole reference coordinate, and the second actual keyhole coordinate and the second target keyhole reference coordinate;

[0039] If there is a keyhole positioning error, a keyhole positioning error prompt message is output;

[0040] If there is no keyhole positioning error, a guidance control instruction is generated according to the first actual keyhole coordinate and the first target keyhole reference coordinate, or the second actual keyhole coordinate and the second target keyhole reference coordinate, and the guidance control instruction is sent to the container truck corresponding to the container, and the guidance control instruction is used to instruct the container truck to guide the container to the reference position of the operation.

[0041] In a possible implementation manner, determining whether there is a keyhole positioning error in the container according to the first actual keyhole coordinate and the first target keyhole reference coordinate, and the second actual keyhole coordinate and the second target keyhole reference coordinate includes:

[0042] Determine a first coordinate deviation value of each keyhole in the width direction of one side of the container according to the first actual keyhole coordinate and the first target keyhole reference coordinate;

[0043] Determine a second coordinate deviation value of each keyhole in the width direction of the other side of the container according to the second actual keyhole coordinate and the second target keyhole reference coordinate;

[0044] Determine the error values ​​between each first coordinate deviation value and each second coordinate deviation value respectively, and determine whether there is a target error value greater than a difference threshold value among the error values;

[0045] If there is a target error value, it is determined that there is a keyhole positioning error in the container;

[0046] If the target error value does not exist, it is determined that there is no keyhole positioning error in the container.

[0047] In a second aspect, an embodiment of the present application provides a computing device, including:

[0048] A processor, and a memory communicatively connected to the processor;

[0049] Memory is used to store computer executable instructions;

[0050] The processor is used to execute the computer-executable instructions stored in the memory to implement the container operation guidance method of the first aspect.

[0051] In a third aspect, an embodiment of the present application provides a field bridge, wherein a first image acquisition device and a second image acquisition device are respectively disposed on the cross beams on both sides of the field bridge, and the first image acquisition device and the second image acquisition device are respectively communicatively connected to the computing device described in the third aspect, and the optical axes of the first image acquisition device and the second image acquisition device are both perpendicular to the plane of the container;

[0052] The first image acquisition device is used to acquire a first container image including two keyholes in a width direction of one side of the container, and the second image acquisition device is used to acquire a second container image including two keyholes in a width direction of the other side of the container.

[0053] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the container operation guidance method of the first aspect.

[0054] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the container operation guidance method of the first aspect.

[0055] The embodiment of the present application provides a method and a yard bridge for guiding the operation of a container, which can identify and detect the acquired container image, obtain the actual keyhole coordinates of the two keyholes in the width direction of the container at the current height, and calculate the actual keyhole distance of the two keyholes accordingly. Since the height of the image acquisition device from the container may change, the keyhole positioning may be deviated, and the actual keyhole coordinates are not necessarily the keyhole coordinates when the container is in the reference position. Therefore, it is necessary to determine the compensation parameter according to the ratio between the pre-calibrated keyhole reference distance of the container and the actual keyhole distance, and use the compensation parameter to compensate the keyhole reference coordinates to obtain the target keyhole reference coordinates when the container is in the reference position of the operation at the current height, so as to accurately control the container truck to guide the container to the reference position of the operation according to the target keyhole reference coordinates. Through such a setting, the keyhole reference coordinates of the keyhole can be compensated according to the actual height of the image acquisition device from the container, and the keyhole reference coordinates of the container at different height positions can be adaptively adjusted, thereby improving the accuracy of keyhole positioning, so that the container can be accurately guided to the reference position of the operation according to the keyhole position. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0057] Figure 1 A system architecture diagram of an embodiment of the present application;

[0058] Figure 2 This is a flow chart of a method for guiding container operations according to an embodiment of the present application;

[0059] Figure 3 This is a schematic diagram of comparing container images at different heights according to an embodiment of the present application;

[0060] Figure 4 A schematic diagram of the structure of a computing device according to an embodiment of the present application;

[0061] Figure 5 A schematic diagram of the structure of a computing device according to another embodiment of the present application.

[0062] Figure numerals: 1. beam; 2. image acquisition device; 3. container; 4. container truck; 5. spreader.

[0063] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0064] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0065] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0066] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of information such as financial data or user data involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0067] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0068] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0069] The container operation guidance method and the crane yard of the present application can be used in the field of automation technology, and can also be used in any field other than the field of automation technology, such as the container lifting field, etc. The application field of the container operation guidance method and the crane yard of the present application is not limited.

[0070] The container operation guidance method and yard crane of the present application can be applied to scenarios where container trucks are used to transport containers. Any scenario involving loading and unloading operations of container trucks under a yard crane can be applied to the container operation guidance method and yard crane of the present application.

[0071] During automatic loading and unloading of containers, the industrial control machine will guide the container truck so that the container is in the reference position for the operation, so that the crane's lifting device can lift the container in the reference position for operation. Therefore, how to accurately guide the container to the reference position is crucial.

[0072] In the related technology, the camera installed on the horizontal beam of the yard bridge can be used to collect the container image data of the container plane below, and send the image data to the industrial computer. After receiving it, the industrial computer will use target detection technology or image segmentation technology to detect and identify the container keyhole in the image, and control the container truck to guide the container to the reference position of the operation according to the coordinates of the keyhole.

[0073] However, since the heights of the container and the container truck are not fixed, that is, the distance from the camera to the container is not fixed, the keyhole coordinates determined by the above method are prone to errors, and the container cannot be accurately guided to the reference position according to the keyhole coordinates, which leads to easy failure of container operations and low operation efficiency.

[0074] Based on the above technical problems, the inventive concept of the present application is to provide a solution that can improve the accuracy of keyhole positioning, thereby accurately guiding the container to the reference position of the operation according to the keyhole position.

[0075] The embodiment of the present application provides an operation guidance method and a yard bridge for containers, which can identify and detect the actual keyhole coordinates and actual keyhole distance of two keyholes in the width direction at the current height of the image acquisition device from the container, and compensate the keyhole reference coordinates according to the ratio of the pre-calibrated keyhole reference distance of the container to the actual keyhole distance to obtain the target keyhole reference coordinates, thereby adaptively adjusting the keyhole reference coordinates of the container at the current height, improving the accuracy of keyhole positioning, and being able to accurately guide the container to the reference position of the operation according to the keyhole position

[0076] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0077] Figure 1 This is a system architecture diagram of an embodiment of the present application, such as Figure 1As shown, 1 is the crossbeam of the field bridge, 2 is the image acquisition device, 3 is the container, 4 is the container truck, and 5 is the hoist of the field bridge. The image acquisition device 2 is respectively arranged on the crossbeam 1 on both sides of the field bridge. The image acquisition device 2 acquires the image of the container below and sends it to the computing device. The computing device obtains the container image acquired by the image acquisition device 2; based on the container image, detects and identifies the actual keyhole coordinates of the two keyholes in the width direction of the container in the container image; according to the actual keyhole coordinates, determines the actual keyhole distance between the two keyholes; determines the pre-calibrated keyhole reference coordinates and keyhole reference distance of the container; determines the compensation parameter according to the ratio of the keyhole reference distance to the actual keyhole distance; according to the compensation parameter and the keyhole reference coordinates, determines the target keyhole reference coordinates when the container is in the reference position of the operation; according to the target keyhole reference coordinates, controls the container truck 4 to guide the container 3 to the reference position of the operation so that the hoist 5 grabs the container 3 for operation.

[0078] Figure 2 This is a flow chart of a method for guiding container operations according to an embodiment of the present application. This embodiment uses a computing device as an execution subject to illustrate the method for guiding container operations. Figure 2 As shown, the container operation guidance method may include the following steps:

[0079] S201: Acquire container images captured by an image acquisition device.

[0080] In this embodiment, the computing device may be an industrial personal computer (IPC) or other types of computing devices, without any limitation.

[0081] In this embodiment, the image acquisition device can be set on the crossbeam of the yard bridge, and the specific setting position of the image acquisition device is not limited, as long as the image acquisition device can capture the two keyholes in the width direction of one side of the lower container. The image acquisition device can be a camera or other equipment, and there is no limitation here.

[0082] In this embodiment, the image acquisition device can acquire container images in real time. As long as a container truck carrying a container drives under the image acquisition device, the image acquisition device will acquire real-time container images.

[0083] In this embodiment, in order to improve the image resolution and clarity, the image acquisition device may include a first image acquisition device and a second image acquisition device, which may be respectively arranged on the two side beams of the yard bridge, and the first image acquisition device may capture two keyholes in the width direction of one side of the container, and the second image acquisition device may capture two keyholes in the width direction of the other side of the container. The first image acquisition device and the second image acquisition device may be arranged at the same or different positions.

[0084] In this embodiment, the optical axes of the image acquisition devices are perpendicular to the plane of the container, so as to acquire the container image of the container plane.

[0085] S202: Based on the container image, detect and identify actual keyhole coordinates of two keyholes in the container width direction in the container image.

[0086] In this embodiment, the keyholes of the container can be keyholes at the top corners of the container, and each corner has a keyhole. The lifting device of the yard crane can be connected to the keyholes at the corners of the container to complete the loading and unloading of the container.

[0087] In this embodiment, the actual keyhole coordinates may be the pixel coordinates of the center point of the keyhole at the current height of the image acquisition device from the container.

[0088] In this embodiment, the image key point detection technology can be used to detect and identify the actual keyhole coordinates of the container in the container image, thereby improving the keyhole detection accuracy and avoiding the problem of missed detection of small targets.

[0089] S203: Determine the actual keyhole distance between the two keyholes according to the actual keyhole coordinates.

[0090] In this embodiment, after the actual keyhole coordinates of the two keyholes are detected, the actual keyhole distance between the two keyholes can be calculated according to the coordinates.

[0091] S204: Determine the pre-calibrated keyhole reference coordinates and keyhole reference distance of the container.

[0092] In this embodiment, the keyhole reference coordinates and the keyhole reference distance may be obtained by calibrating the two keyholes in the width direction when the container (container of the size) is at the corresponding reference position.

[0093] In this embodiment, two keyholes in the width direction of containers of various sizes, such as 20 feet, 40 feet, and 45 feet, can be calibrated in advance to obtain the keyhole reference coordinates (the coordinates of the two calibrated keyholes) of the containers of various sizes, and the keyhole reference distance (the distance between the two calibrated keyholes) can be calculated through the keyhole reference coordinates. After the size of the container is determined, the keyhole reference coordinates and the keyhole reference distance of the container during pre-calibration can be determined from the pre-calibration data corresponding to the containers of various sizes.

[0094] S205: Determine a compensation parameter according to the ratio of the keyhole reference distance to the actual keyhole distance.

[0095] Exemplarily, if the actual keyhole distance is dc11c22 and the keyhole reference distance is dc1c2, the compensation parameter is dc11c22 / dc1c2.

[0096] S206: Determine the target keyhole reference coordinates when the container is at the reference position of the operation according to the compensation parameters and the keyhole reference coordinates.

[0097] In this embodiment, after the compensation parameters are determined, the keyhole reference coordinates can be adaptively adjusted according to the compensation parameters to obtain the target keyhole reference coordinates when the container is in the reference position for operation at the current height.

[0098] For example, if the height of the camera from the container is Zc, and the container is at the reference position (Xc, Yc, Zc), the pre-calibrated keyhole reference coordinates are (u1, v1), (u2, v2), and the keyhole reference distance is dc1c2. At the current height of the camera from the container, the actual keyhole distance between the two keyholes is measured using the image key point detection technology as dc11c22. Then, the target keyhole reference coordinates (u11, v11), (u22, v22) can be obtained using the following formulas (1)-(4):

[0099] u11=(dc11c22 / dc1c2)(u1-cx)+cx (1)

[0100] v11=(dc11c22 / dc1c2)(v1-cy)+cy(2)

[0101] u22=(dc11c22 / dc1c2)(u2-cx)+cx (3)

[0102] v22=(dc11c22 / dc1c2)(v2-cy)+cy (4)

[0103] Among them, cx and cy are the internal parameters of the camera, which can be obtained through camera calibration.

[0104] The target keyhole reference coordinates (u11, v11), (u22, v22) are the reference coordinates of the keyhole center point when the container is at the reference position (Xc, Yc, Zc+h) at the current height (Zc+h) of the camera from the container.

[0105] Specifically, Figure 3 FIG. 1 is a schematic diagram showing a comparison of container images at different heights according to an embodiment of the present application. Figure 3 As shown in the figure, (a) is the reference image of the container when the camera is 15 meters away from the container, and (b) is the image of the container when the height of the camera from the container changes by simulating the camera zoom when the container does not move. The distance between the two keyholes in (a) is 664.25pix, and the coordinates of the center point of the upper keyhole are (1574, 347). The distance between the two keyholes in (b) is 607.31pix. Using the above formula, it can be calculated that at the new height, the reference coordinates of this keyhole are (1521, 364), while the reference coordinates measured at this height in (b) are (1515, 362). The pixel deviation between the two is 6 pixels, which is in line with expectations. The reference coordinates calculated using the above formula are basically consistent with the actual measured reference coordinates.

[0106] S206: Guiding the container to the reference position of the operation according to the target keyhole reference coordinates.

[0107] In this embodiment, since the container is rigid, theoretically, the coordinate deviation of the two keyholes in the width direction of one side of the container is the same as the coordinate deviation of the two keyholes in the width direction of the other side of the container. Therefore, the first keyhole coordinates in the width direction of one side of the container and the first keyhole reference coordinates can be used to guide the container to the reference position for the operation.

[0108] In this embodiment, the actual keyhole coordinates may be the starting point of the container guidance and positioning, and the target keyhole reference coordinates may be the end point of the container guidance and positioning. By controlling the container truck to guide the keyhole on one side of the container from the actual keyhole coordinates to the target keyhole reference coordinates, the container can be guided to the reference position of the operation.

[0109] In this embodiment, the reference position may be the position of the container operation in the world coordinate system. The keyhole coordinates and the keyhole reference coordinates may be coordinates in the pixel coordinate system.

[0110] In this embodiment, the current height may be the height of the image acquisition device from the container when the image acquisition device acquires the image of the container below.

[0111] In this embodiment, the actual keyhole coordinates may be the coordinates at the current height of the image acquisition device from the container. Since the height of the image acquisition device from the container may change, resulting in deviations in keyhole positioning, the actual keyhole coordinates may be height compensated to obtain the target keyhole reference coordinates adaptively adjusted at the current height.

[0112] In this embodiment, the acquired container image can be identified and detected to obtain the actual keyhole coordinates of the two keyholes in the width direction of the container at the current height, and the actual keyhole distance of the two keyholes can be calculated accordingly. Since the height of the image acquisition device from the container may change, resulting in a deviation in the keyhole positioning, the actual keyhole coordinates are not necessarily the keyhole coordinates when the container is in the reference position. Therefore, it is necessary to determine the compensation parameter according to the ratio between the pre-calibrated keyhole reference distance of the container and the actual keyhole distance, and use the compensation parameter to compensate the keyhole reference coordinates to obtain the target keyhole reference coordinates when the container is in the reference position of the operation at the current height, so as to accurately control the container truck to guide the container to the reference position of the operation according to the target keyhole reference coordinates. Through such a setting, the keyhole reference coordinates of the keyhole can be compensated according to the actual height of the image acquisition device from the container, and the keyhole reference coordinates of the container at different height positions can be adaptively adjusted, thereby improving the accuracy of keyhole positioning, so that the container can be accurately guided to the reference position of the operation according to the keyhole position.

[0113] In a possible implementation, the above step S202 detects and identifies actual keyhole coordinates of two keyholes in the container width direction in the container image based on the container image, which may include:

[0114] The container image is input into the preset key point detection model to obtain the actual keyhole coordinates of the two keyholes in the width direction of the container.

[0115] The key point detection model is pre-trained using a container sample image including two keyholes in the width direction, and the container sample image includes a sample label, and the sample label is the keyhole coordinates of the two keyholes in the width direction.

[0116] In this embodiment, the length and height of the container may change, but the width is fixed, so the keyhole coordinates of the container can be determined by using two keyholes in the width direction of the container.

[0117] In this embodiment, after the coordinates of the keyhole in the camera coordinate system are detected, the coordinates of the keyhole in the pixel coordinate system can be obtained according to the projection formula.

[0118] For example, assuming that in the camera coordinate system, when the camera is at a height of Zc from the container, the center point coordinates of the two keyholes are detected to be Pc1=[Xc1, Yc1, Zc], Pc2=[Xc2, Yc2, Zc], and according to the projection formula, the corresponding pixel coordinates in the pixel coordinate system are (u1, v1), (u2, v2), u1=f×(Xc1 / Zc)×dx+cx, v1=f×(Yc1 / Zc)×dy+cy, u2=f×(Xc2 / Zc)×dx+cx, v2=f×(Yc2 / Zc)×dy+cy.

[0119] In general, dx=dy can be considered, where f, dx, dy, cx, and cy are the internal parameters of the camera and can be obtained through camera calibration.

[0120] In this embodiment, since the image acquisition device is far away from the container target, the container keyhole area captured in the image is very small. Therefore, a key point detection model based on the image key point detection algorithm can be used to identify the keyhole area in the container image and obtain the actual keyhole coordinates of the two keyholes in the width direction of the container, thereby avoiding the problem of missed detection of small targets and improving the accuracy and comprehensiveness of keyhole positioning.

[0121] In a possible implementation, the above step S204 of determining the pre-calibrated keyhole reference coordinates and keyhole reference distance of the container may include:

[0122] S11: Determine the target size of the container according to the container image.

[0123] S12: According to the correspondence between the preset container size and the pre-calibration data, the target pre-calibration data corresponding to the target size is determined, and the target pre-calibration data includes a keyhole reference coordinate and a keyhole reference distance.

[0124] S13: Determine the keyhole reference coordinates and keyhole reference distance of the container pre-calibration according to the target pre-calibration data.

[0125] In this embodiment, the specific implementation of determining the container size according to the container image can refer to the prior art and will not be described in detail here.

[0126] In this embodiment, the container size may be 20 feet, 40 feet, 45 feet, etc., and those skilled in the art may flexibly set the size according to actual conditions.

[0127] In this embodiment, the two keyholes in the width direction of containers of various sizes can be calibrated in advance to obtain the keyhole reference coordinates and keyhole reference distances when the containers of various sizes are in the corresponding reference positions. After the target size of the container is determined, the pre-calibrated keyhole reference coordinates and keyhole reference distance of the container can be simply and accurately determined according to the corresponding relationship between the target size and the preset container size and the pre-calibrated data.

[0128] In a possible implementation, the correspondence between the preset container size and the pre-calibrated data in the above step S12 may be obtained in the following manner:

[0129] S21: Obtain reference container images corresponding to containers of various sizes. For each size of container, the reference container image is an image captured by an image acquisition device at a preset height from the container of that size and when the container of that size is at a corresponding reference position.

[0130] S22: For each size of container, calibrate the reference container image to obtain the keyhole reference coordinates of the two keyholes in the width direction of the container, and determine the keyhole reference distance between the two keyholes according to the keyhole reference coordinates; determine the pre-calibration data corresponding to the container of the size according to the keyhole reference coordinates and the keyhole reference coordinates.

[0131] S23: Determine the correspondence between the preset container size and the pre-calibrated data according to the pre-calibrated data corresponding to each container size.

[0132] In this embodiment, the preset height can be flexibly set by those skilled in the art, and the preset heights of the image acquisition device from containers of different sizes can be the same or different, and no limitation is made here.

[0133] In this embodiment, when calibrating the reference container image, the key point detection technology of the image can be used to obtain the keyhole reference coordinates of the two keyholes in the width direction of the container, so as to avoid the error of keyhole positioning.

[0134] In this embodiment, the reference positions corresponding to containers of different sizes can be preset, and the reference container images of the containers of each size at the reference positions can be obtained. The reference container images can be calibrated to obtain the keyhole reference coordinates and keyhole reference coordinates corresponding to the containers of each size.

[0135] In a possible implementation manner, before determining the compensation parameter according to the ratio of the keyhole reference distance to the actual keyhole distance in step S205, the following may also be included:

[0136] S31: Determine whether the actual buttonhole distance is consistent with the buttonhole reference distance.

[0137] S32: If not consistent, it is determined that the container is not in the reference position of the operation, and a step of determining a compensation parameter according to the ratio of the keyhole reference distance to the actual keyhole distance is performed.

[0138] S33: If they are consistent, it is determined that the container is in the reference position for the operation, and the container is not guided.

[0139] In this embodiment, if the actual distance is consistent with the keyhole reference distance, it means that the height of the image acquisition device from the container has not changed, and is the same as the preset height during pre-calibration. The actual keyhole coordinates are the keyhole reference coordinates when the container is in the reference position at the current height. The container is already in the reference position for the operation and does not need to be guided.

[0140] In this embodiment, if the actual distance is inconsistent with the keyhole reference distance, it means that the height of the image acquisition device from the container has changed and is different from the preset height during pre-calibration. The actual keyhole coordinates need to be compensated and adjusted to obtain the keyhole reference coordinates of the container at the current height.

[0141] In this embodiment, by comparing the actual distance between the two keyholes at the current height with the keyhole reference distance obtained during pre-calibration, it is possible to determine whether the height of the image acquisition device from the container has changed compared to the pre-calibration time, thereby accurately judging whether the actual keyhole distance needs to be compensated and adjusted.

[0142] In a possible implementation, the above step S207 guides the container to the reference position of the operation according to the target keyhole reference coordinates, which may include:

[0143] A guidance control instruction is generated according to the target keyhole reference coordinates and the actual keyhole coordinates, and the guidance control instruction is sent to the container truck corresponding to the container. The guidance control instruction is used to instruct the container truck to guide the container to the reference position of the operation.

[0144] In this embodiment, the actual keyhole coordinates may be the starting point of the container guidance and positioning, and the target keyhole reference coordinates may be the end point of the container guidance and positioning. By controlling the container truck to guide the keyhole on one side of the container from the actual keyhole coordinates to the target keyhole reference coordinates, the container can be guided to the reference position of the operation.

[0145] In this embodiment, since the container is rigid, theoretically, the container position determined based on the target keyhole reference coordinates of the two keyholes in the width direction of one side of the container is the same as the container position determined based on the target keyhole reference coordinates of the two keyholes in the width direction of the other side of the container. Therefore, the container can be accurately guided to the reference position for the operation by using the actual keyhole coordinates and the target keyhole reference coordinates in the width direction of one side of the container.

[0146] In a possible implementation, the image acquisition device may include a first image acquisition device and a second image acquisition device. Accordingly, the above step S207 guides the container to the reference position of the operation according to the target keyhole reference coordinates, and may also include:

[0147] S41: Determine the first target keyhole reference coordinates corresponding to the first container image according to the target keyhole reference coordinates.

[0148] S42: Determine a second container image corresponding to the first container image, and determine a second target keyhole reference coordinate corresponding to the second container image.

[0149] S43: Guiding the container to a reference position for the operation according to the first target keyhole reference coordinates and the second target keyhole reference coordinates.

[0150] Among them, the first image acquisition device and the second image acquisition device are relatively arranged on the two side beams of the yard bridge, the first image acquisition device is used to acquire a first container image including two keyholes in the width direction of one side of the container, and the second image acquisition device is used to acquire a second container image including two keyholes in the width direction of the other side of the container; the optical axes of the first image acquisition device and the second image acquisition device are both perpendicular to the plane of the container.

[0151] In this embodiment, the width direction of the other side of the container may be the width direction opposite to the width direction of one side of the container. For example, if the width direction of one side of the container is the left width direction, the width direction of the other side of the container is the right width direction.

[0152] In this embodiment, the specific process of determining the second target keyhole reference coordinates corresponding to the second container image is the same as the target keyhole reference coordinates (first target keyhole reference coordinates), and can refer to the above steps S201-S206, which will not be described in detail.

[0153] In this embodiment, in order to improve image resolution and clarity, the image acquisition device may include a first image acquisition device and a second image acquisition device. The first image acquisition device and the second image acquisition device may be respectively arranged on the two side beams of the yard bridge relative to each other. The first image acquisition device and the second image acquisition device respectively capture two keyholes in the width direction of one side of the container, thereby further improving the accuracy of keyhole positioning.

[0154] In this embodiment, the first image acquisition device and the second image acquisition device can be set at the same or different positions on the two side beams of the field bridge. There is no limitation here, as long as they can respectively capture the two keyholes in the width direction of one side of the lower container.

[0155] In this embodiment, since the cross beams on both sides of the field crane and the container are rigid, theoretically, the first image acquisition device and the second image acquisition device are at the same height from the container.

[0156] In this embodiment, after determining the first target keyhole reference coordinates of the two keyholes in the width direction of one side of the container at the current height, the second target keyhole reference coordinates of the two keyholes in the width direction of the other side of the container at the current height can also be determined. The keyhole reference coordinates positioned on both sides are used to guide the container to the reference position of the operation, further improving the accuracy of container guidance.

[0157] In a possible implementation, the above step S43 guides the container to the reference position of the operation according to the first target keyhole reference coordinates and the second target keyhole reference coordinates, and may include:

[0158] S51: Determine a first actual keyhole coordinate corresponding to the first container image and a second actual keyhole coordinate corresponding to the second container image.

[0159] S52: Determine whether there is a keyhole positioning error in the container according to the first actual keyhole coordinate and the first target keyhole reference coordinate, and the second actual keyhole coordinate and the second target keyhole reference coordinate.

[0160] S53: If there is a buttonhole positioning error, output a buttonhole positioning error prompt message.

[0161] S54: If there is no keyhole positioning error, a guidance control instruction is generated according to the first actual keyhole coordinate and the first target keyhole reference coordinate, or the second actual keyhole coordinate and the second target keyhole reference coordinate, and the guidance control instruction is sent to the container truck corresponding to the container. The guidance control instruction is used to instruct the container truck to guide the container to the reference position of the operation.

[0162] In this embodiment, if there is no positioning error, the effects of guiding according to the first actual keyhole coordinates, the first target keyhole reference coordinates, and the second keyhole actual coordinates and the second target keyhole reference coordinates are the same, and all can guide the container to the reference position. Therefore, only one set of them is used to generate the guidance control instruction.

[0163] In this embodiment, since the container is rigid, theoretically, the first container position determined according to the first target keyhole reference coordinates of the two keyholes in the width direction of one side of the container should be the same as the second container position determined according to the second target keyhole reference coordinates of the two keyholes in the width direction of the other side of the container. If the first container position and the second container position differ greatly, it means that the previous keyhole positioning process has an error, and it is necessary to output a prompt message to prompt the staff to take corresponding measures to further improve the accuracy of container guidance.

[0164] In a possible implementation, the above step S52 determines whether there is a keyhole positioning error in the container according to the first actual keyhole coordinate and the first target keyhole reference coordinate, and the second actual keyhole coordinate and the second target keyhole reference coordinate, and may include:

[0165] S61: Determine a first coordinate deviation value of each keyhole in the width direction of one side of the container according to the first actual keyhole coordinate and the first target keyhole reference coordinate.

[0166] S62: Determine a second coordinate deviation value of each keyhole in the width direction of the other side of the container according to the second actual keyhole coordinate and the second target keyhole reference coordinate.

[0167] S63: respectively determining the error values ​​between each first coordinate deviation value and each second coordinate deviation value, and determining whether there is a target error value greater than a difference threshold value among the error values.

[0168] S64: If there is a target error value, it is determined that there is a keyhole positioning error in the container.

[0169] S65: If the target error value does not exist, it is determined that there is no keyhole positioning error in the container.

[0170] In this implementation, those skilled in the art may flexibly set the difference threshold. For example, the difference threshold may be 5 pixels or 10 pixels, and no limitation is made herein.

[0171] Exemplarily, the first keyhole coordinates are (u1, v1), (u2, v2), the first keyhole reference coordinates are (u11, v11), (u22, v22); the second keyhole coordinates are (u3, v3), (u4, v4), and the second keyhole reference coordinates are (u33, v33), (u44, v44). The first coordinate deviation value is a1=(u11, v11)-(u1, v1), a2=(u22, v22)-(u2, v2), and the second coordinate deviation value is a3=(u33, v33)-(u3, v3), a4=(u44, v44)-(u4, v4).

[0172] Error values ​​A1=|a1-a2|, A2=|a1-a3|, A3=|a1-a4|, A4=|a2-a3|, A5=|a2-a4|, A6=|a3-a4|.

[0173] If the error values ​​are all less than the difference threshold, it means that there is no positioning error and the buttonhole is positioned accurately.

[0174] If a certain error value is greater than the difference threshold, it indicates that an error has occurred in the keyhole positioning.

[0175] In this embodiment, according to the difference between the coordinate deviation values ​​of each keyhole in the width direction of one side of the container and the coordinate deviation values ​​of each keyhole in the width direction of the other side, it can be accurately determined whether the target keyhole reference coordinates calculated in the width directions on both sides of the container are within the allowable error range. If they are all within the allowable error range, there is no positioning error, the keyhole positioning is accurate, and the container can be accurately guided to the reference position of the operation according to the keyhole position.

[0176] The container operation guidance method of the present application is described below with reference to a specific embodiment.

[0177] In a specific embodiment, a container truck drives to a certain position under the field bridge, and the first image acquisition device and the second image acquisition device arranged on the beams on both sides of the field bridge collect the image of the container below and send it to the industrial computer. The industrial computer determines the keyhole reference coordinates of the container and guides the operation of the container. The specific process is as follows:

[0178] In the first step, the first image acquisition device acquires a first container image including two keyholes on the left side of the container in the width direction, and sends it to the industrial computer. The second image acquisition device acquires a second container image including two keyholes on the right side of the container in the width direction, and sends it to the industrial computer.

[0179] In the second step, the industrial computer uses the key point detection model to determine the first actual keyhole coordinates of the two keyholes in the width direction of the left side of the container in the first container image.

[0180] In the third step, the industrial computer determines the first actual keyhole distance between the two keyholes at the current height according to the first actual keyhole coordinates of the two keyholes.

[0181] In the fourth step, the industrial computer determines the size of the container based on the first container image, and determines the keyhole reference coordinates of the two keyholes in the width direction and the keyhole reference distance between the two keyholes when the container of this size is in the corresponding reference position based on the correspondence between the preset container size and the pre-calibrated data.

[0182] In the fifth step, the industrial computer determines that the first actual keyhole distance is inconsistent with the keyhole reference distance, then determines a compensation parameter according to the ratio of the keyhole reference distance to the first actual keyhole distance, and determines the first target keyhole reference coordinate when the container is in the reference position of the operation according to the compensation parameter and the keyhole reference coordinate.

[0183] In the sixth step, the industrial computer repeats the above steps 2 to 5 to obtain the second actual keyhole coordinates of the two keyholes in the width direction of the right side of the container in the second container image, and the second target keyhole reference coordinates when the container is in the reference position at the current height.

[0184] In the seventh step, the industrial computer determines the first coordinate deviation value according to the first actual keyhole coordinate and the first target keyhole reference coordinate; determines the second coordinate deviation value according to the second actual keyhole coordinate and the second target keyhole reference coordinate; and determines, according to the error value between the first coordinate deviation value and the second coordinate deviation value, that the target keyhole reference coordinates calculated in the width directions on both sides are within the allowable error range, thereby determining that there is no positioning error in the keyhole.

[0185] In the eighth step, the industrial computer generates a guidance control instruction according to the first actual keyhole coordinate and the first target keyhole reference coordinate, and sends the guidance control instruction to the container truck corresponding to the container. The guidance control instruction is used to instruct the container truck to guide the container to the reference position of the operation.

[0186] Figure 4 This is a schematic diagram of the structure of a computing device according to an embodiment of the present application. Figure 4 As shown, the computing device includes: an acquisition module 41, used to acquire the container image acquired by the image acquisition device; a processing module 42, used to detect and identify the actual keyhole coordinates of two keyholes in the container width direction in the container image based on the container image; determine the actual keyhole distance between the two keyholes according to the actual keyhole coordinates; determine the pre-calibrated keyhole reference coordinates and the keyhole reference distance of the container; determine the compensation parameter according to the ratio of the keyhole reference distance to the actual keyhole distance; determine the target keyhole reference coordinates when the container is in the reference position of the operation according to the compensation parameter and the keyhole reference coordinates; and guide the container to the reference position of the operation according to the target keyhole reference coordinates.

[0187] The computing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0188] In a possible implementation, the processing module 42 may also be used to:

[0189] Input the container image into the preset key point detection model to obtain the actual keyhole coordinates of the two keyholes in the width direction of the container;

[0190] The key point detection model is pre-trained using a container sample image including two keyholes in the width direction, and the container sample image includes a sample label, and the sample label is the keyhole coordinates of the two keyholes in the width direction.

[0191] The computing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0192] In a possible implementation, the processing module 42 may also be used to:

[0193] Determine the target size of the container based on the container image;

[0194] According to the correspondence between the preset container size and the pre-calibration data, the target pre-calibration data corresponding to the target size is determined, and the target pre-calibration data includes a keyhole reference coordinate and a keyhole reference distance;

[0195] According to the target pre-calibration data, the keyhole reference coordinates and keyhole reference distance of the container pre-calibration are determined.

[0196] The computing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0197] In a possible implementation, the processing module 42 may also be used to:

[0198] Obtain reference container images corresponding to containers of various sizes. For each size of container, the reference container image is an image captured by an image acquisition device at a preset height from the container of that size and when the container of that size is at a corresponding reference position;

[0199] For each size of container, calibrate the reference container image to obtain the keyhole reference coordinates of the two keyholes in the width direction of the container, and determine the keyhole reference distance of the two keyholes according to the keyhole reference coordinates; determine the pre-calibration data corresponding to the container of the size according to the keyhole reference coordinates and the keyhole reference coordinates;

[0200] According to the pre-calibrated data corresponding to each container size, the corresponding relationship between the preset container size and the pre-calibrated data is determined.

[0201] The computing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0202] In a possible implementation, before determining the compensation parameter according to the ratio of the keyhole reference distance to the actual keyhole distance, the processing module 42 may also be used to:

[0203] Determine whether the actual keyhole distance is consistent with the keyhole reference distance;

[0204] If not, it is determined that the container is not in the reference position of the operation, and a step of determining a compensation parameter according to a ratio of the keyhole reference distance to the actual keyhole distance is performed;

[0205] If they are consistent, it is determined that the container is in the reference position for the operation and the container is not guided.

[0206] The computing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0207] In a possible implementation, the processing module 42 may also be used to:

[0208] A guidance control instruction is generated according to the target keyhole reference coordinates and the actual keyhole coordinates, and the guidance control instruction is sent to the container truck corresponding to the container. The guidance control instruction is used to instruct the container truck to guide the container to the reference position of the operation.

[0209] The computing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0210] In a possible implementation, the image acquisition device includes a first image acquisition device and a second image acquisition device, the first image acquisition device and the second image acquisition device are arranged on the two side beams of the yard bridge relatively, the first image acquisition device is used to acquire a first container image including two keyholes in the width direction of one side of the container, and the second image acquisition device is used to acquire a second container image including two keyholes in the width direction of the other side of the container; the optical axes of the first image acquisition device and the second image acquisition device are both perpendicular to the plane of the container; the processing module 42 can also be used to:

[0211] Determine the first target keyhole reference coordinates corresponding to the first container image according to the target keyhole reference coordinates;

[0212] Determine a second container image corresponding to the first container image, and determine a second target keyhole reference coordinate corresponding to the second container image;

[0213] The container is guided to a reference position for the operation according to the first target keyhole reference coordinates and the second target keyhole reference coordinates.

[0214] The computing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0215] In a possible implementation, the processing module 42 may also be used to:

[0216] Determine a first actual keyhole coordinate corresponding to the first container image, and a second actual keyhole coordinate corresponding to the second container image;

[0217] Determining whether there is a keyhole positioning error in the container according to the first actual keyhole coordinate and the first target keyhole reference coordinate, and the second actual keyhole coordinate and the second target keyhole reference coordinate;

[0218] If there is a keyhole positioning error, a keyhole positioning error prompt message is output;

[0219] If there is no keyhole positioning error, a guidance control instruction is generated according to the first actual keyhole coordinate and the first target keyhole reference coordinate, or the second actual keyhole coordinate and the second target keyhole reference coordinate, and the guidance control instruction is sent to the container truck corresponding to the container. The guidance control instruction is used to instruct the container truck to guide the container to the reference position of the operation.

[0220] The computing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0221] In a possible implementation, the processing module 42 may also be used to:

[0222] Determine a first coordinate deviation value of each keyhole in a width direction of one side of the container according to the first actual keyhole coordinate and the first target keyhole reference coordinate;

[0223] Determine the second coordinate deviation value of each keyhole in the width direction of the other side of the container according to the second actual keyhole coordinate and the second target keyhole reference coordinate;

[0224] Determine the error values ​​between each first coordinate deviation value and each second coordinate deviation value respectively, and determine whether there is a target error value greater than a difference threshold value among the error values;

[0225] If there is a target error value, it is determined that there is a keyhole positioning error in the container;

[0226] If there is no target error value, it is determined that there is no keyhole positioning error in the container.

[0227] The computing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0228] Figure 5 This is a schematic diagram of the structure of a computing device according to an embodiment of the present application. Figure 5 As shown, the computing device includes: a processor 501, and a memory 502 that is communicatively connected to the processor 501; the memory 502 stores computer execution instructions; the processor 501 executes the computer execution instructions stored in the memory 502 to implement the steps of the container operation guidance method in the above-mentioned method embodiments.

[0229] In the above computing device, the memory 502 and the processor 501 are electrically connected directly or indirectly to realize data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines, such as through a bus connection. The memory 502 stores computer-executable instructions for implementing the data access control method, including at least one software function module that can be stored in the memory 502 in the form of software or firmware. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502.

[0230] The memory 502 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electric erasable programmable read-only memory (EEPROM), etc. The memory 502 is used to store programs, and the processor 501 executes the programs after receiving the execution instruction. Furthermore, the software programs and modules in the above-mentioned memory 502 may also include an operating system, which may include various software components and / or drivers for managing system tasks (such as memory management, storage device control, power management, etc.), and may communicate with various hardware or software components to provide an operating environment for other software components.

[0231] The processor 501 may be an integrated circuit chip having the ability to process signals. The processor 501 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. The methods, steps, and logic diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or any conventional processor.

[0232] An embodiment of the present application also provides a field bridge, such as Figure 2 As shown, the first image acquisition device and the second image acquisition device are respectively arranged on the two side beams of the field bridge, and the first image acquisition device and the second image acquisition device are respectively arranged with Figure 5 The computing devices shown are communicatively connected, and the optical axes of the first image acquisition device and the second image acquisition device are both perpendicular to the plane of the container; the first image acquisition device is used to acquire a first container image including two keyholes in the width direction of one side of the container, and the second image acquisition device is used to acquire a second container image including two keyholes in the width direction of the other side of the container.

[0233] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the steps of each method embodiment of the present application.

[0234] An embodiment of the present application further provides a computer program product, including a computer program, which implements the steps of each method embodiment of the present application when executed by a processor.

[0235] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0236] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0237] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0238] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0239] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0240] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the appended claims.

[0241] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for guiding container operation, characterized in that: include: Acquire container images captured by an image acquisition device; Based on the container image, detecting and identifying actual keyhole coordinates of two keyholes in the container width direction in the container image; Determining the actual keyhole distance between the two keyholes according to the actual keyhole coordinates; Determining the pre-calibrated keyhole reference coordinates and keyhole reference distance of the container; Determining a compensation parameter according to a ratio of the keyhole reference distance to the actual keyhole distance; Determining the target keyhole reference coordinates when the container is in the reference position of the operation according to the compensation parameters and the keyhole reference coordinates; The container is guided to a reference position for operation according to the target keyhole reference coordinates.

2. The container operation guidance method according to claim 1, characterized in that: The detecting and identifying actual keyhole coordinates of two keyholes in the container width direction in the container image based on the container image includes: Input the container image into a preset key point detection model to obtain the actual keyhole coordinates of the two keyholes in the width direction of the container; The key point detection model is pre-trained using a container sample image including two keyholes in the width direction, and the container sample image includes a sample label, and the sample label is the keyhole coordinates of the two keyholes in the width direction.

3. The container operation guidance method according to claim 2, characterized in that: The determining of the pre-calibrated keyhole reference coordinates and keyhole reference distance of the container includes: Determining a target size of the container according to the container image; According to the correspondence between the preset container size and the pre-calibration data, the target pre-calibration data corresponding to the target size is determined, and the target pre-calibration data includes a keyhole reference coordinate and a keyhole reference distance; According to the target pre-calibration data, the keyhole reference coordinates and keyhole reference distance of the container pre-calibration are determined.

4. The container operation guidance method according to claim 3, characterized in that: The correspondence between the preset container size and the pre-calibrated data is obtained in the following manner: Obtaining reference container images corresponding to containers of various sizes, respectively. For each size of container, the reference container image is an image acquired when the image acquisition device is at a preset height from the container of the size and the container of the size is at a corresponding reference position; For each size of container, calibrate the reference container image to obtain keyhole reference coordinates of two keyholes in the width direction of the container, and determine the keyhole reference distance between the two keyholes according to the keyhole reference coordinates; Determining pre-calibrated data corresponding to a container of said size according to said keyhole reference coordinates and said keyhole reference coordinates; According to the pre-calibrated data respectively corresponding to the container of each size, the corresponding relationship between the preset container size and the pre-calibrated data is determined.

5. The method for guiding container operation according to any one of claims 1 to 4, characterized in that: Before determining the compensation parameter according to the ratio of the keyhole reference distance to the actual keyhole distance, the method further includes: Determining whether the actual keyhole distance is consistent with the keyhole reference distance; If not, it is determined that the container is not in the reference position for the operation, and the step of determining the compensation parameter according to the ratio of the keyhole reference distance to the actual keyhole distance is performed; If they are consistent, it is determined that the container is already in the reference position for the operation, and the container is not guided.

6. The method for guiding container operation according to any one of claims 1 to 4, characterized in that: The step of guiding the container to a reference position for operation according to the target keyhole reference coordinates comprises: A guidance control instruction is generated according to the target keyhole reference coordinates and the actual keyhole coordinates, and the guidance control instruction is sent to a container truck corresponding to the container, wherein the guidance control instruction is used to instruct the container truck to guide the container to a reference position for the operation.

7. The method for guiding container operation according to any one of claims 1 to 4, characterized in that: The image acquisition device comprises a first image acquisition device and a second image acquisition device, the first image acquisition device and the second image acquisition device are arranged on the two side beams of the yard bridge relatively, the first image acquisition device is used to acquire a first container image including two keyholes in the width direction of one side of the container, and the second image acquisition device is used to acquire a second container image including two keyholes in the width direction of the other side of the container; the optical axes of the first image acquisition device and the second image acquisition device are both perpendicular to the plane of the container; Accordingly, guiding the container to the reference position of the operation according to the target keyhole reference coordinates includes: Determining, according to the target keyhole reference coordinates, first target keyhole reference coordinates corresponding to the first container image; Determine a second container image corresponding to the first container image, and determine a second target keyhole reference coordinate corresponding to the second container image; The container is guided to a reference position for operation according to the first target keyhole reference coordinates and the second target keyhole reference coordinates.

8. The method for guiding container operation according to claim 7, characterized in that: The step of guiding the container to a reference position for operation according to the first target keyhole reference coordinates and the second target keyhole reference coordinates comprises: Determining a first actual keyhole coordinate corresponding to the first container image and a second actual keyhole coordinate corresponding to the second container image; Determining whether there is a keyhole positioning error on the container according to the first actual keyhole coordinate and the first target keyhole reference coordinate, and the second actual keyhole coordinate and the second target keyhole reference coordinate; If there is a keyhole positioning error, a keyhole positioning error prompt message is output; If there is no keyhole positioning error, a guidance control instruction is generated according to the first actual keyhole coordinate and the first target keyhole reference coordinate, or the second actual keyhole coordinate and the second target keyhole reference coordinate, and the guidance control instruction is sent to the container truck corresponding to the container, and the guidance control instruction is used to instruct the container truck to guide the container to the reference position of the operation.

9. The method for guiding container operation according to claim 8, characterized in that: The determining whether there is a keyhole positioning error in the container according to the first actual keyhole coordinate and the first target keyhole reference coordinate, and the second actual keyhole coordinate and the second target keyhole reference coordinate, comprises: Determine a first coordinate deviation value of each keyhole in the width direction of one side of the container according to the first actual keyhole coordinate and the first target keyhole reference coordinate; Determine a second coordinate deviation value of each keyhole in the width direction of the other side of the container according to the second actual keyhole coordinate and the second target keyhole reference coordinate; Determine the error values ​​between each first coordinate deviation value and each second coordinate deviation value respectively, and determine whether there is a target error value greater than a difference threshold value among the error values; If there is a target error value, it is determined that there is a keyhole positioning error in the container; If the target error value does not exist, it is determined that there is no keyhole positioning error in the container.

10. A field bridge, characterized in that: A first image acquisition device and a second image acquisition device are respectively arranged on the cross beams on both sides of the field bridge, the first image acquisition device and the second image acquisition device are respectively connected to the computing device for communication, and the optical axes of the first image acquisition device and the second image acquisition device are both perpendicular to the plane of the container; The first image acquisition device is used to acquire a first container image including two keyholes in a width direction of one side of the container, and the second image acquisition device is used to acquire a second container image including two keyholes in a width direction of the other side of the container; The computing device comprises: A processor, and a memory communicatively connected to the processor; The memory is used to store computer-executable instructions; The processor is used to execute the computer-executable instructions stored in the memory to implement the container operation guidance method according to any one of claims 1 to 9.

Images