Container positioning method, container positioning device, container positioning equipment and field bridge

Abstract

The invention provides a container positioning method, device and equipment and a field bridge. The method comprises the following steps: acquiring an initial image of the top surface of a container loaded by a container truck; according to the container position of the initial image and a preset reference position, whether the initial image meets an initial positioning condition is determined, and the reference position is the position of the container in an image shot by a pre-calibrated field bridge camera; if the initial positioning condition is met, controlling the camera to carry out zooming processing, and obtaining a zoomed image after zooming; the zoom image is cut to obtain a cut image, and the cut image comprises a container hole of the container; and determining a target deviation of the container according to the container hole area in the cut image and a preset reference container hole area. According to the method, initial positioning is carried out through few computing resources, after initial positioning succeeds, the effective pixel size of a container hole area is increased through zooming processing, the zoomed image is cut, interference of surrounding objects is effectively removed, and the positioning accuracy of the container is improved.

Description

Technical Field

[0001] The present application relates to the field of container loading and unloading, and in particular to a method, device, equipment and yard crane for positioning a container. Background Art

[0002] With the continuous growth of trade and the rapid development of container transportation, the automation of container loading and unloading operations is becoming increasingly higher. In the truck guidance system, the camera is installed on the main beam of the yard bridge to capture images of the truck and container from top to bottom, obtain the location of the truck and container based on the captured images, and provide path guidance for the truck to ensure that the truck can reach the preset location.

[0003] However, due to the insufficient accuracy of the collected images, the accuracy of container positioning is poor, which affects the guidance of the container truck and makes it impossible to accurately reach the predetermined location. Therefore, how to improve the positioning accuracy of the container is an urgent problem to be solved. Summary of the invention

[0004] The present application provides a method, device, equipment and yard bridge for positioning a container, so as to improve the positioning accuracy of a container loaded on a container truck.

[0005] In a first aspect, the present application provides a method for positioning a container, the method comprising:

[0006] Acquire an initial image of the top surface of the container loaded on the container truck;

[0007] Determining whether the initial image meets the initial positioning condition according to the container position in the initial image and a preset reference position, wherein the reference position is the position of the container in the image taken by the pre-calibrated yard bridge camera;

[0008] If the initial positioning condition is met, the camera is controlled to perform zoom processing, and a zoom image captured after zooming is obtained;

[0009] Cropping the zoomed image to obtain a cropped image, wherein the cropped image includes the container hole;

[0010] The target deviation of the container is determined according to the container hole area in the cropped image and a preset reference container hole area.

[0011] Optionally, determining the target deviation of the container according to the container hole area in the cropped image and a preset reference container hole area includes:

[0012] Obtain the target pixel coordinates of the center point of the box hole area;

[0013] Obtaining the reference pixel coordinates of the center point of the reference box hole area;

[0014] The target deviation of the container in the advancing direction of the container truck is determined according to the target pixel coordinates and the reference pixel coordinates.

[0015] Optionally, determining whether the initial image satisfies an initial positioning condition according to the container position of the initial image and a preset reference position includes:

[0016] determining an initial position of the container from the initial image;

[0017] Determining whether a deviation between the initial position and the reference position is within a preset range;

[0018] If it is within the preset range, the initial positioning condition is met; otherwise, the initial positioning condition is not met.

[0019] Optionally, the method further includes:

[0020] If the target deviation exceeds the preset deviation range, a first guiding signal is output, wherein the first guiding signal is used to guide the container truck to move until the new target deviation is within the deviation range;

[0021] If the initial positioning condition is not met, a second guiding signal is output, wherein the second guiding signal is used to guide the container truck to move until the initial positioning condition is met.

[0022] Optionally, controlling the camera to perform zoom processing includes:

[0023] Acquire a pre-stored fixed zoom focal length, where the fixed zoom focal length is greater than the focal length of the image captured during initial positioning;

[0024] The fixed zoom focal length is sent to the camera.

[0025] Optionally, the step of cropping the zoomed image to obtain a cropped image includes:

[0026] The image of a fixed size area is cropped from a preset fixed position of the zoomed image as a starting point to obtain the cropped image.

[0027] Optionally, before determining the deviation of the container according to the container hole area in the cropped image and a preset reference container hole area, the method further includes:

[0028] The box hole center point in the cropped image is determined by a deep learning model, wherein the deep learning model is a pre-trained neural network model capable of identifying the box hole center point.

[0029] In a second aspect, the present application provides a container positioning device, the device comprising:

[0030] An acquisition module, used to acquire an initial image of the top surface of the container loaded on the container truck;

[0031] An initial positioning module, used to determine whether the initial image meets the initial positioning conditions according to the container position in the initial image and a preset reference position, wherein the reference position is the position of the container in the image taken by the pre-calibrated yard bridge camera;

[0032] A zoom control module, used to control the camera to perform zoom processing and obtain a zoomed image taken after zooming if the initial positioning condition is met;

[0033] A cropping module, used for cropping the zoomed image to obtain a cropped image, wherein the cropped image includes the container hole;

[0034] The determination module is used to determine the target deviation of the container according to the container hole area in the cropped image and a preset reference container hole area.

[0035] Optionally, the determining module is used to:

[0036] Obtain the target pixel coordinates of the center point of the box hole area;

[0037] Obtaining the reference pixel coordinates of the center point of the reference box hole area;

[0038] The target deviation of the container in the advancing direction of the container truck is determined according to the target pixel coordinates and the reference pixel coordinates.

[0039] Optionally, the initial positioning module is specifically used for:

[0040] determining an initial position of the container from the initial image;

[0041] Determining whether a deviation between the initial position and the reference position is within a preset range;

[0042] If it is within the preset range, the initial positioning condition is met; otherwise, the initial positioning condition is not met.

[0043] Optionally, the device further includes a control module, wherein the control module is used to:

[0044] If the target deviation exceeds a preset deviation range, a first guiding signal is output, wherein the first guiding signal is used to guide the container truck to move until the deviation is within the deviation range;

[0045] If the initial positioning condition is not met, a second guiding signal is output, wherein the second guiding signal is used to guide the container truck to move until the initial positioning condition is met.

[0046] Optionally, the zoom control module is specifically used for:

[0047] Acquire a pre-stored fixed zoom focal length, where the fixed zoom focal length is greater than the focal length of the image captured during initial positioning;

[0048] The fixed zoom focal length is sent to the camera.

[0049] Optionally, the cutting module is specifically used for:

[0050] The image of a fixed size area is cropped from a preset fixed position of the zoomed image as a starting point to obtain the cropped image.

[0051] Optionally, the determining module is further used for:

[0052] The box hole center point in the cropped image is determined by a deep learning model, wherein the deep learning model is a pre-trained neural network model capable of identifying the box hole center point.

[0053] In a third aspect, the present application provides an electronic device, including: a memory, a processor;

[0054] The memory stores computer-executable instructions;

[0055] The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method as described in any one of the first aspects.

[0056] In a fourth aspect, the present application provides a field bridge, comprising a controller and a zoom camera, wherein the controller is used to execute the method as described in any one of the first aspects.

[0057] In a fifth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the above first aspect and / or various possible implementations of the first aspect.

[0058] In a sixth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementations of the first aspect.

[0059] The present application provides a method, device, equipment and yard bridge for container positioning, the method comprising: obtaining an initial image of the top surface of a container loaded on a container truck; determining whether the initial image meets the initial positioning condition based on the container position in the initial image and a preset reference position, the reference position being the position of the container in the image taken by a pre-calibrated yard bridge camera; if the initial positioning condition is met, controlling the camera to perform zoom processing, and obtaining a zoomed image taken after zooming; cropping the zoomed image to obtain a cropped image, the cropped image including the container hole; determining the target deviation of the container based on the box hole area in the cropped image and the preset reference box hole area. First, determine whether the initial positioning is successful by using less computing resources, after the initial positioning is successful, zoom the camera to increase the effective pixel size of the box hole area, which can improve the positioning accuracy, crop the zoomed image to effectively remove the interference of surrounding objects, and accurately locate the box hole corners of the cropped image. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0061] Figure 1 Schematic diagram of the container positioning method provided in this application Figure 1 ;

[0062] Figure 2 A schematic diagram of a container truck loaded with a container provided in this application;

[0063] Figure 3 Schematic diagram of the process of container positioning method provided in this application Figure 2 ;

[0064] Figure 4 A schematic diagram of a zoomed image provided for this application;

[0065] Figure 5 Schematic diagram of the cropped image provided for this application;

[0066] Figure 6 A schematic diagram of a container positioning device provided in this application;

[0067] Figure 7 A schematic diagram of the structure of the electronic device provided in this application.

[0068] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0069] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0070] First, the terms involved in this application are explained:

[0071] Crane equipment refers to heavy machinery used to unload containers from ships or trucks and transfer them to designated locations at container terminals or logistics centers. Crane equipment usually consists of a crane arm, tracks, drive systems, etc. It can move horizontally and lift containers through the crane arm and place them on trucks, yards or other means of transportation.

[0072] A container truck is a truck used for container transportation, usually consisting of a trailer head and a trailer. Container trucks are used to transfer containers between container terminals and inland transportation networks.

[0073] Container holes refer to the openings on the container used for loading and unloading goods, usually located on the top or side of the container. Holes are used to provide passages during container loading and unloading, allowing lifting equipment, robotic arms and other tools to extract or put goods from the container through these openings. The location and size of the holes are usually regulated according to the standard design of the container to ensure the efficiency and safety of loading and unloading operations. In automated container operations, accurately locating the location of the container holes is the key to accurate operation.

[0074] At present, the position of the container truck is mainly adjusted by manual observation and guidance so that the spreader can grab and put the container on the container truck, which is very time-consuming and cumbersome. In some methods, visual methods are used to take pictures for identification and positioning, but the accuracy is not high, which may lead to grabbing failure.

[0075] In view of this, the application pre-calibrates the position of the spreader and the camera. After the container truck arrives at the field of view, it performs two positioning operations based on the zoom camera. First, it performs initial positioning to preliminarily determine whether the container meets the requirements. If it meets the requirements, it improves the pixel quality of the keyhole area of ​​the image after zooming, and performs fine positioning after cropping the keyhole area to finally determine whether the position of the container meets the requirements. In this way, the positioning accuracy of the container is improved and multiple grab failures are avoided.

[0076] The execution subject of this application is the controller of the field bridge equipment, or the processor in the controller of the field bridge equipment.

[0077] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0078] Figure 1 Schematic diagram of the container positioning method provided in this application Figure 1 ,like Figure 1 As shown, the method comprises the following steps:

[0079] S101. Acquire an initial image of the top surface of a container loaded on a container truck.

[0080] In this step, the camera installed on the yard crane equipment takes the initial image of the top surface of the container and serves as the basic image for the subsequent positioning and recognition process. The initial image is the starting point for the system to perform subsequent operations and contains the approximate location and shape of the container.

[0081] S102: determining whether the initial image meets an initial positioning condition according to the container position in the initial image and a preset reference position, where the reference position is the position of the container in the image taken by the pre-calibrated yard bridge camera.

[0082] In this step, the position of the container in the initial image is analyzed by the image processing algorithm and compared with the pre-calibrated reference position. The reference position is the standard position of the container in the field of view determined by the image data of the yard bridge camera in the previous calibration process. By judging the degree of deviation of the container in the initial image, it is confirmed whether the image meets the initial positioning conditions for subsequent operations. If the initial positioning conditions are met, the process proceeds to the next step, otherwise it is necessary to re-acquire the initial image or guide the container truck to adjust the position.

[0083] In a specific implementation, the reference rectangular area of ​​the container on the truck under the camera's view angle is pre-calibrated as the reference position, and then the target rectangular area of ​​the current container in the camera's view angle is determined from the initial image. Finally, the overlapping area of ​​the reference rectangular area and the target rectangular area is determined. If the overlapping area is greater than a preset value, it is determined that the initial positioning condition is met. The preset value can be set to 99%, 98%, and 95%.

[0084] In a specific implementation method, the initial position of the container is identified in the initial image. The initial position may refer to the position of each edge of the top of the container in the initial image. The reference position is also the position of each edge in the reference image. By position comparison, it is determined whether the deviation between the position of each edge and the corresponding reference position is within a preset range. If it is within the preset range, the initial positioning condition is met; otherwise, the initial positioning condition is not met.

[0085] S103: If the initial positioning condition is met, control the camera to perform zoom processing and obtain a zoomed image captured after zooming.

[0086] If the initial positioning conditions are met, indicating that the location of the container meets the requirements, the camera is instructed to zoom. Zooming is to enlarge the camera focal length to make the container hole area clearer for more accurate analysis. The purpose of this step is to increase the details of the image to ensure accurate cropping and positioning of the hole area in the future.

[0087] The zoom processing is performed according to a fixed zoom focal length. The change in zoom focal length is the same as that in the calibration process. Based on this, it provides a basis for determining the deviation between the current container and the calibrated reference position and maintaining consistency between the two.

[0088] S104: Crop the zoomed image to obtain a cropped image, where the cropped image includes the container holes.

[0089] The zoomed image is cropped to extract the container hole area and obtain a sub-image containing the hole details. The cropped image usually has a fixed size, which can remove unnecessary background and cluttered areas and focus on the hole and its surrounding areas, thereby reducing the computational complexity and improving the accuracy of subsequent image analysis.

[0090] In order to improve the accuracy of subsequent positioning, each cropping is performed with a fixed size and area, and the size and area are the same as those cropped during calibration. For example, during calibration, a standard square size is used for cropping in the initial image to ensure that the center point of the box hole is located at the center of the cropped square. The preset position of the square used for cropping in the zoom image during calibration is recorded. During normal execution, a standard square size is used for cropping at a fixed position in the zoom image to obtain a cropped image. By cropping the image of the box hole area according to a fixed position and size, the computing resource overhead is reduced.

[0091] S105: Determine a target deviation of the container according to the container hole area in the cropped image and a preset reference container hole area.

[0092] In the cropped image, the target deviation of the container is calculated by comparing the actual position of the hole area with the preset reference hole area. The reference hole area is pre-calibrated according to the standard size and position of the container. The comparison method is the same as the comparison method of the initial positioning. The difference from the initial positioning is that the initial positioning compares the position deviation of the container outline, while the comparison after zooming is the position deviation of the hole area.

[0093] In one implementation, the horizontal coordinates of the center points of the two box hole areas are obtained and the average value is calculated, where the center point is the intersection point of the diagonal of each rectangular box hole area. Then, the horizontal coordinate of the center point of the reference box hole is obtained. The target deviation of the container in the forward direction (horizontally) is determined by subtracting the horizontal coordinate of the center point of the reference box hole from the average value of the horizontal coordinate.

[0094] In one implementation, the horizontal coordinates of the left lines of the two box hole areas are obtained and the average value is calculated. The horizontal coordinate of the left line of the reference box hole area is obtained. The target deviation of the container in the forward direction (horizontally) is determined by subtracting the horizontal coordinate of the left line of the reference box hole area from the average value of the horizontal coordinate of the left line of the box hole area.

[0095] In one implementation, the target pixel coordinates of the center point of the box hole area are obtained, and the reference pixel coordinates of the center point of the reference box hole area are obtained, where the center point is the intersection of the diagonals of the rectangular box hole area. The average value of the two target pixel coordinates is determined, and the target deviation of the container in the forward direction (horizontally) is determined by subtracting the reference pixel horizontal coordinate from the average value of the target pixel coordinate horizontal coordinate. In the same way, the target deviation of the container in the vertical direction can be determined. If the target deviation is within the preset error range, no adjustment is required. If the deviation exceeds the preset error range, the container truck or container needs to be adjusted before it can be grabbed.

[0096] If the target center point of the box hole area is in the center area after zooming, the deviation can be determined by the center point of the area, which can avoid the error caused by the difference in container height and improve the accuracy of error judgment. Among them, the target center point is the midpoint of the line connecting the center points of the two box holes of the container, and the center area is an area within the preset range of the center position of the zoom image. Specifically, different tire pressures of container trucks will cause different heights of containers. When the target center point is in the center area, the box hole area can be regarded as on the optical axis of the camera. In this way, when collecting zoomed images of container trucks at different heights, it is equivalent to a slight zoom of the image. After zooming, the edge lines of the box hole area will change, but the center point position is the same at different heights. Therefore, in this case, the accuracy of judging the deviation by the center point is higher.

[0097] This embodiment provides a method for positioning a container, which includes: obtaining an initial image of the top surface of a container loaded on a container truck; determining whether the initial image meets the initial positioning condition based on the container position in the initial image and a preset reference position, the reference position being the position of the container in the image taken by a pre-calibrated yard bridge camera; if the initial positioning condition is met, controlling the camera to perform zoom processing, and obtaining a zoomed image taken after zooming; cropping the zoomed image to obtain a cropped image, the cropped image including the container hole; determining the target deviation of the container based on the container hole area in the cropped image and the preset reference container hole area. First, determine whether the initial positioning is successful by using less computing resources. After the initial positioning is successful, zoom the camera to increase the effective pixel size of the container hole area, which can improve the positioning accuracy, crop the zoomed image to effectively remove the interference of surrounding objects, and accurately locate the corner points of the container hole in the cropped image.

[0098] Through the previous positioning process, the deviation between the container and the reference position has been obtained. Deviations may include position deviations (lateral, longitudinal, and vertical offsets) and angle deviations (rotation angle errors). These deviations will affect whether the spreader (such as a crane hook or spreader arm) can be accurately placed in the container's box hole. According to the size of the container, the size and structure of the spreader, a reasonable deviation range is preset. The deviation range is set based on the safety and accuracy of the operation to ensure that the spreader can be smoothly docked to the box hole.

[0099] Exemplarily, the deviation range may be set to ±5 cm for lateral deviation and ±2° for angular deviation.

[0100] If the actual deviation of the container exceeds the preset deviation range, it means that the spreader cannot accurately connect to the container hole. At this time, the controller will send a signal to prompt that the container truck needs to be adjusted.

[0101] If the deviation is a horizontal offset (for example, the container is misaligned front to back), the truck needs to move forward or backward along the road to ensure that the container is aligned with the placement position of the spreader.

[0102] Angle adjustment: If the deviation is an angle (for example, the rotation angle deviation of the container), on the one hand, the alignment can be achieved by adjusting the rotation angle of the spreader to ensure that the spreader can be correctly inserted into the container hole; on the other hand, the angle of the container truck can also be adjusted for alignment to ensure that the spreader can be correctly inserted into the container hole.

[0103] Optionally, to facilitate the adjustment of the container truck, a movable conveyor belt is set at the bottom of the container truck, and the yard bridge controller can control the forward and backward movement of the conveyor belt, which can realize automatic fine-tuning of the container truck without the need for the driver to manually start the vehicle for adjustment. This method can increase the adjustment speed of the container truck, and can also achieve precise adjustment by calculating and determining the adjustment distance of the container truck, which significantly improves the positioning accuracy compared to the method where the user manually starts the vehicle to move forward and backward.

[0104] After the container truck has adjusted its position, the system will retake the initial image, recalculate the deviation of the container, and determine whether it meets the preset deviation range.

[0105] The above embodiment introduces a method for performing precise positioning. The following describes the calibration process with a specific embodiment.

[0106] The calibration process includes the following steps:

[0107] (1) Adjust the angle of the zoom camera.

[0108] The image captured from the perspective of the zoom camera contains lane lines that guide the vehicle. The camera angle is adjusted so that the two lane lines are horizontal and parallel in the image.

[0109] (2) When the spreader is in the center position, move the spreader to the top of the container truck operation lane and record the spreader's position (Y0, Z0, theta0) at this time. That is, the spreader's position is fixed and waits for a container truck loaded with containers to enter the site. Under the premise that the spreader is kept stationary, adjust the position and posture of the container truck to ensure that when it is adjusted to a certain position, the spreader only needs to be lowered vertically to enter the container hole (Y0 and theta0 cannot be adjusted).

[0110] (3) Ensure that the container truck does not move and move the spreader away to avoid blocking the container image. Save the reference image at this time, which is the image before zooming and the image after zooming. Use these two images to determine the reference position and the reference box hole area.

[0111] The reference position is a region where the container outline is determined based on the image before zooming, or a left line of the container top surface outline is used as the reference position. The reference position is used for initial positioning of the container during the execution of the method.

[0112] Based on the zoomed image, the zoomed box holes are identified. At least two box holes are identified in the image. For each box hole, a reference box hole area can be determined. The reference box hole area can be a coordinate range or the coordinates of the center point of the box hole area.

[0113] The above calibration process can provide accuracy guarantee for subsequent container positioning, so that the spreader can be placed vertically in the container hole.

[0114] The actual operation process of container positioning is introduced below with a specific example.

[0115] Figure 2 A schematic diagram of a container truck loaded with a container provided in this application, such as Figure 2 As shown, the container drives to the preset position and stops moving forward. Figure 2 It is also the initial image captured by the camera.

[0116] Figure 3 Schematic diagram of the process of container positioning method provided in this application Figure 2 ,like Figure 3 As shown, the following steps are included:

[0117] S301, obtaining an initial image.

[0118] S302: Identify the target rectangular area of ​​the container using target detection or image segmentation method.

[0119] In one implementation, an object detection algorithm (such as YOLO, Faster R-CNN, etc.) is used to identify the container area in the image. This method uses a pre-trained model to identify whether there is a container target in the image and calibrate its position in the image.

[0120] In another implementation, image segmentation techniques (such as U-Net, Mask R-CNN, etc.) are used to identify the outline and boundaries of the container. The identified container area will be framed into a rectangular area, usually the outer boundary of the container, for subsequent position and deviation analysis.

[0121] S303: Compare the target rectangular area with the reference target rectangular area to determine a first deviation.

[0122] The reference target rectangular area is a pre-calibrated reference position, which refers to the expected position of the container in a specific parking space.

[0123] The deviation between the target rectangular area and the reference target rectangular area is determined by calculating the difference (eg, position deviation, rotation angle deviation) between the target rectangular area and the reference target rectangular area.

[0124] S304: Determine whether the first deviation is less than a threshold.

[0125] If it is less than the threshold, it means the initial positioning is successful, and step S305 is executed.

[0126] If it is greater than or equal to the threshold, it indicates that the initial positioning has failed, and step S312 is executed.

[0127] S305: Control the camera to perform zoom processing.

[0128] The zoom operation adjusts the camera focal length to make the target area of ​​the container clearer, facilitating subsequent precise positioning.

[0129] S306: Acquire a zoomed image after zooming.

[0130] Figure 4 A schematic diagram of a zoomed image provided for this application, in Figure 2 Based on this, we can zoom in and get Figure 4 Image shown.

[0131] S307 , performing fixed-size cropping of the box hole area on the zoomed image.

[0132] The zoomed image is cropped in a fixed area to crop out the hole area on the container. The hole area is the opening on the container used for lifting. Cropping can focus on this area.

[0133] Figure 5 A schematic diagram of a cropped image provided for this application, Figure 5 The graph a shown in Figure 4 Schematic diagram of the cutting of the box hole in the middle and upper part, and Figure b is Figure 4 Schematic diagram of the cutting of the box hole in the upper middle.

[0134] S308. Calculate the center point of the box hole using a deep learning model.

[0135] The cropped image is input into the deep learning model to output the center point of the box hole, wherein the deep learning model is a pre-trained neural network model capable of identifying the center point of the box hole. Specifically, the coordinates of the four box hole corner points in a box hole area are first determined, and then the coordinates of the center point are calculated.

[0136] like Figure 5 As shown in the figure, the four corner points of each box hole area are marked, and the position coordinates of the center point can be determined. The 0.96 and 0.97 marked in the figure are the horizontal coordinates of the center points of the two box hole areas in the image. The average value of the horizontal coordinates of the center points of the two box hole areas is 0.965, which is the horizontal coordinate of the center point of the box hole.

[0137] S309, comparing the center point of the box hole with the center point of the reference box hole to determine a second deviation.

[0138] The difference between the calculated box hole center point and the reference box hole center point is compared to obtain the second deviation. The reference box hole center point indicates the position where the container should be aligned during hoisting during the calibration process.

[0139] Exemplarily, the position of the center point of the reference box hole area is 1, and when the center point of the box hole area is 0.965, it means that the deviation between the two is 0.035.

[0140] S310: Determine whether the second deviation is less than a threshold.

[0141] The threshold is a preset error range, for example, it can be set to 0.1, 0.05, 0.08, etc.

[0142] If yes, the final positioning is successful and step S311 is executed.

[0143] If not, execute step S312.

[0144] S311. Place the spreader and grab the container.

[0145] S312, adjusting the position of the collection card.

[0146] If the first deviation of the container is large, resulting in a failure in the initial positioning, the truck position needs to be adjusted until the container is aligned with the ideal position and positioned again. Specifically, the truck guidance system outputs a second guidance signal (such as a sound prompt or a route diagram) to prompt the truck to adjust until the initial positioning condition is met.

[0147] If the initial positioning is successful, but the second deviation is large, the container truck position also needs to be fine-tuned. Specifically, the container truck guidance system outputs a first guidance signal (such as a sound prompt or a route diagram) to prompt the container truck to make fine adjustments until the second positioning requirement is met.

[0148] It should be noted that the actual size of the container hole area is fixed, so the actual distance and angle to be adjusted can be determined according to the deviation value. For example, the actual distance in the horizontal direction of the hole area is divided by the horizontal width of the hole area in the current pixel to obtain the actual distance per unit image width, and then multiplied by the horizontal deviation value to obtain the horizontal distance that the container truck needs to move. Similarly, the vertical distance can be determined.

[0149] The process of this example combines target detection, deep learning, image cropping and geometric calculation to ensure that the container can be accurately docked with the spreader, ensuring the safety and efficiency of the lifting process.

[0150] Figure 6 A schematic diagram of a container positioning device provided in this application, such as Figure 6 As shown, the container positioning device 60 includes:

[0151] An acquisition module 601 is used to acquire an initial image of the top surface of the container loaded on the container truck;

[0152] An initial positioning module 602 is used to determine whether the initial image meets the initial positioning condition according to the container position in the initial image and a preset reference position, wherein the reference position is the position of the container in the image taken by the pre-calibrated yard bridge camera;

[0153] A zoom control module 603 is used to control the camera to perform zoom processing and obtain a zoomed image taken after zooming if the initial positioning condition is met;

[0154] A cropping module 604, configured to crop the zoomed image to obtain a cropped image, wherein the cropped image includes a container hole;

[0155] The determination module 605 is used to determine the target deviation of the container according to the container hole area in the cropped image and a preset reference container hole area.

[0156] Optionally, the determining module 605 is used to:

[0157] Obtain the target pixel coordinates of the center point of the box hole area;

[0158] Obtaining the reference pixel coordinates of the center point of the reference box hole area;

[0159] The target deviation in the moving direction of the container truck is determined according to the target pixel coordinates and the reference pixel coordinates.

[0160] Optionally, the initial positioning module 602 is specifically used for:

[0161] determining an initial position of the container from the initial image;

[0162] Determining whether a deviation between the initial position and the reference position is within a preset range;

[0163] If it is within the preset range, the initial positioning condition is met; otherwise, the initial positioning condition is not met.

[0164] Optionally, the device further includes a control module 606, and the control module 606 is used to:

[0165] If the target deviation exceeds the preset deviation range, a first guiding signal is output, wherein the first guiding signal is used to guide the container truck to move until the new target deviation is within the deviation range;

[0166] If the initial positioning condition is not met, a second guiding signal is output, wherein the second guiding signal is used to guide the container truck to move until the initial positioning condition is met.

[0167] Optionally, the zoom control module 603 is specifically used for:

[0168] Acquire a pre-stored fixed zoom focal length, where the fixed zoom focal length is greater than the focal length of the image captured during initial positioning;

[0169] The fixed zoom focal length is sent to the camera.

[0170] Optionally, the cutting module 604 is specifically used for:

[0171] The image of a fixed size area is cropped from a preset fixed position of the zoomed image as a starting point to obtain the cropped image.

[0172] Optionally, the determining module 605 is further configured to:

[0173] The box hole center point in the cropped image is determined by a deep learning model, wherein the deep learning model is a pre-trained neural network model capable of identifying the box hole center point.

[0174] The container positioning device provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be described in detail here.

[0175] Figure 7 This is a schematic diagram of the structure of the electronic device provided in this application, which may be a controller of a field bridge device. Figure 7 As shown, the electronic device 50 provided in this embodiment includes: at least one processor 501 and a memory 502. Optionally, the device 50 also includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected via a bus 504.

[0176] In a specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502, so that at least one processor 501 executes the above method.

[0177] The specific implementation process of the processor 501 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0178] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0179] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0180] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0181] The present application also provides a field bridge device, which includes a controller and a zoom camera, and the controller is used to implement the method in the above embodiment.

[0182] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0183] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0184] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0185] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0186] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0187] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0188] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0189] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0190] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0191] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for positioning a container, characterized in that: The method comprises: Acquire an initial image of the top surface of the container loaded on the container truck; Determining whether the initial image meets the initial positioning condition according to the container position in the initial image and a preset reference position, wherein the reference position is the position of the container in the image taken by the pre-calibrated yard bridge camera; If the initial positioning condition is met, the camera is controlled to perform zoom processing, and a zoom image captured after zooming is obtained; Cropping the zoomed image to obtain a cropped image, wherein the cropped image includes the container hole; The target deviation of the container is determined according to the container hole area in the cropped image and a preset reference container hole area.

2. The method according to claim 1, characterized in that The step of determining the target deviation of the container according to the container hole area in the cropped image and the preset reference container hole area includes: Obtain the target pixel coordinates of the center point of the box hole area; Obtaining the reference pixel coordinates of the center point of the reference box hole area; The target deviation of the container in the advancing direction of the container truck is determined according to the target pixel coordinates and the reference pixel coordinates.

3. The method according to claim 1, characterized in that The determining whether the initial image satisfies the initial positioning condition according to the container position of the initial image and the preset reference position includes: determining an initial position of the container from the initial image; Determining whether a deviation between the initial position and the reference position is within a preset range; If it is within the preset range, the initial positioning condition is met; otherwise, the initial positioning condition is not met.

4. The method according to claim 1, characterized in that: The method further comprises: If the target deviation exceeds the preset deviation range, a first guidance signal is output, wherein the first guidance signal is used to guide the container truck to move until the new target deviation is within the deviation range; If the initial positioning condition is not met, a second guiding signal is output, wherein the second guiding signal is used to guide the container truck to move until the initial positioning condition is met.

5. The method according to any one of claims 1 to 4, characterized in that: The controlling the camera to perform zoom processing comprises: Acquire a pre-stored fixed zoom focal length, where the fixed zoom focal length is greater than the focal length of the image captured during initial positioning; The fixed zoom focal length is sent to the camera.

6. The method according to any one of claims 1 to 4, characterized in that: The step of cropping the zoomed image to obtain a cropped image includes: The image of a fixed size area is cropped from a preset fixed position of the zoomed image as a starting point to obtain the cropped image.

7. The method according to any one of claims 1 to 4, characterized in that: Before determining the deviation of the container according to the container hole area in the cropped image and the preset reference container hole area, the method further includes: The box hole center point in the cropped image is determined by a deep learning model, wherein the deep learning model is a pre-trained neural network model capable of identifying the box hole center point.

8. A container positioning device, characterized in that: The device comprises: An acquisition module, used to acquire an initial image of the top surface of the container loaded on the container truck; An initial positioning module, used to determine whether the initial image meets the initial positioning conditions according to the container position in the initial image and a preset reference position, wherein the reference position is the position of the container in the image taken by the pre-calibrated yard bridge camera; A zoom control module, used to control the camera to perform zoom processing and obtain a zoomed image taken after zooming if the initial positioning condition is met; A cropping module, used for cropping the zoomed image to obtain a cropped image, wherein the cropped image includes the container hole; The determination module is used to determine the target deviation of the container according to the container hole area in the cropped image and a preset reference container hole area.

9. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A field bridge, characterized in that: The invention comprises a controller and a zoom camera, wherein the controller is used to execute the method according to any one of claims 1 to 7.

Images