Automatic box placing method, device, electronic equipment and engineering machinery

Abstract

The present application provides an automatic container placement method, device, electronic equipment and engineering machinery, which obtains the position of the target container corner and the position of the corresponding target component by means of a camera arranged on the spreader, and adjusts the spreader to the projection of the target container on the plane where the free area is located to be located in the free area based on the obtained position, and obtains the position of the target container component and the position of the target truck component by means of a camera arranged on one side of the container truck. Since the target container component and the target truck component are in contact or overlap after the spreader places the container, the position of the target container component and the position of the target truck component can more accurately reflect the positional relationship between the container and the truck deck, therefore, based on the position of the target container component and the position of the target truck, the spreader is controlled to place the container to the free area, which can achieve the effect of placing the container at the appropriate position, improve the reliability and success rate of placing the container, and effectively ensure the success of placing the container.

Description

Technical Field

[0001] The present application relates to the technical field of container lifting, and in particular to an automatic container placing method, device, electronic equipment and engineering machinery. Background Art

[0002] At present, laser radar is usually used to obtain the spatial position information of containers and container trucks, and the container release is controlled based on the spatial position information of the containers and container trucks. Alternatively, a camera set under the spreader is used to identify the position of the lock hole on the container and the position of the lock on the truck's deck, and the container release is controlled based on the lock hole position and the lock head position.

[0003] Among them, when the camera installed under the spreader is used to identify the position of the lock hole on the container and the position of the lock on the truck's deck, if the container is located above the deck and drops to a certain height, the camera installed under the spreader may be blocked by the container, and the positional relationship between the container and the deck cannot be observed, making it difficult to determine the appropriate container placement position, resulting in container placement failure. Summary of the invention

[0004] Based on the defects and shortcomings of the above-mentioned prior art, the present application proposes an automatic container placement method, device, electronic equipment and engineering machinery, which can control the spreader to place the container to the idle area when the target container is above the idle area of ​​the vehicle plate based on the positions of the target container components and the target truck components obtained by the camera on one side of the truck, thereby solving the problem of container placement failure caused by the difficulty in determining the appropriate container placement position in the prior art.

[0005] According to a first aspect of an embodiment of the present application, there is provided an automatic box placing method, comprising:

[0006] The position of the target container corner and the position of the target component are obtained by a camera arranged on the spreader, wherein the target container corner is the container corner of the target container grabbed by the spreader, and the target component is a component on the free area of ​​the truck plate or a container corner of an adjacent container to the target container, and the target container corner has a corresponding relationship with the target component;

[0007] Based on the position of the target container corner and the position of the corresponding target component, adjusting the spreader until the projection of the target container on the plane where the free area is located is within the free area;

[0008] The position of the target container component and the position of the target truck component are obtained by a camera arranged on one side of the truck, the target container component and the target truck component are in contact with or overlapped with each other after the spreader places the container in the free area, the target container component is a component on the target container whose position remains unchanged relative to the target container, and the target truck component is a component on the truck whose position remains unchanged relative to the truck;

[0009] Based on the position of the target container component and the position of the target truck component, the spreader is controlled to place the container into the idle area.

[0010] According to a second aspect of an embodiment of the present application, there is provided an automatic box placing device, comprising:

[0011] An acquisition module is used to acquire the position of a target container corner and a target component through a camera arranged on the spreader, wherein the target container corner is a container corner of a target container grabbed by the spreader, and the target component is a component on an idle area of ​​a truck plate or a container corner of an adjacent container to the target container, and the target container corner has a corresponding relationship with the target component;

[0012] an adjustment module, configured to adjust the spreader based on the position of the target container corner and the corresponding position of the target component until the projection of the target container on the plane where the idle area is located is within the idle area;

[0013] The acquisition module is further used to acquire the position of the target container component and the position of the target truck component through a camera arranged on one side of the truck, the target container component and the target truck component contacting or overlapping each other after the spreader places the container in the idle area, the target container component is a component on the target container whose position remains unchanged relative to the target container, and the target truck component is a component on the truck whose position remains unchanged relative to the truck;

[0014] The control module is used to control the spreader to place the container to the idle area based on the position of the target container component and the position of the target truck component.

[0015] According to a third aspect of an embodiment of the present application, there is provided an electronic device, including a memory and a processor;

[0016] The memory is connected to the processor and is used to store programs;

[0017] The processor is used to implement the automatic box placement method as described in the first aspect by running the program in the memory.

[0018] According to a fourth aspect of an embodiment of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the automatic box placement method as described in the first aspect is implemented.

[0019] According to a fifth aspect of an embodiment of the present application, there is provided a construction machine, in which the automatic box placing device as described in the second aspect or the electronic device as described in the third aspect is installed.

[0020] In the above-mentioned automatic container placement method, device, electronic equipment and engineering machinery, the position of the target corner, i.e., the corner of the target container grabbed by the spreader, and the position of the target component corresponding to the target corner, i.e., the component on the free area of ​​the truck board or the position of the corner of the container adjacent to the target container, are obtained by a camera arranged on the spreader, and the spreader is adjusted based on the position of the target corner and the position of the corresponding target component until the target container is located above the free area, i.e., the projection of the target container on the plane where the free area is located is located in the free area. At this time, the position of the target container component and the position of the target truck component corresponding to the target container component located on the same side of the truck are obtained by a camera arranged on one side of the truck, so that the positional relationship between the container and the truck board can be continuously observed, and based on the position of the target container component and the position of the target truck component, the spreader is controlled to place the container to the free area of ​​the truck board. Since the target container component and the target truck component overlap or touch each other after the spreader places the target container in the idle area, after the projection of the target container on the plane where the idle area is located is within the idle area, the positional relationship between the target container and the truck deck can continue to be observed through the camera arranged on one side of the truck. Based on the positions of the target container component and the target truck component, the spreader can be controlled to place the container at a suitable position, which better ensures successful container placement and solves the problem in the prior art that when the container is located above the truck, it is difficult to observe the positional relationship between the container and the truck, and a suitable container placement position cannot be selected, resulting in container placement failure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0022] Figure 1 A schematic diagram of a process flow of an automatic box placing method provided in an embodiment of the present application;

[0023] Figure 2A schematic diagram of a target component provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of the direction of a large vehicle and a small vehicle provided in an embodiment of the present application;

[0025] Figure 4 (a) is a schematic diagram of the position of a lock head and a lock hole proposed in an embodiment of the present application Figure 1 ;

[0026] Figure 4 (b) is a schematic diagram of the position of a lock head and a lock hole proposed in an embodiment of the present application Figure 2 ;

[0027] Figure 5 A schematic diagram of an automatic box placement process proposed in an embodiment of the present application;

[0028] Figure 6 A schematic diagram of the structure of an automatic box placing device proposed in an embodiment of the present application;

[0029] Figure 7 A schematic diagram of the structure of an automatic box placing system proposed in an embodiment of the present application;

[0030] Figure 8 A schematic diagram of the structure of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] Overview

[0033] As described in the background technology, when the position of the lock hole on the container and the position of the lock on the truck's deck are identified by the camera provided under the spreader, if the container is located above the deck, the camera installed under the spreader may be blocked by the container, and the positional relationship between the container and the deck cannot be observed, making it difficult to determine the appropriate container placement position, resulting in container placement failure.

[0034] On this basis, the inventor found through further research that when the container is located above the deck of the container truck, by setting a camera on one side of the container truck, the positional relationship between the container and the container truck can continue to be observed through the camera on the side of the container truck. Therefore, after obtaining the position of the target box corner and the position of the target component through the camera set on the spreader, and adjusting the spreader to the projection of the target container on the plane where the idle area is located is located in the idle area based on the position of the target box corner and the position of the corresponding target component, the position of the target container component and the position of the target container truck component are obtained through the camera set on the side of the container truck. Since the target container component and the target container truck component are in contact or overlap after the spreader releases the container, based on the position of the target container component and the position of the target container truck component, the spreader is controlled to release the container to the idle area, which can achieve the effect of placing the container at the appropriate position, effectively ensuring the successful release of the container, and solving the problem in the prior art that the camera installed under the spreader may be blocked by the container because the container is located above the deck, and the positional relationship between the container and the deck cannot be observed, and it is difficult to determine the appropriate position for placing the container, resulting in the failure of placing the container.

[0035] Based on the above concept, an embodiment of the present specification provides an automatic box placing method, which will be described exemplarily in conjunction with the accompanying drawings.

[0036] Exemplary Methods

[0037] See also Figure 1 In an exemplary embodiment, an automatic container placement method is provided, which is applied to any electronic device that is communicatively connected with a camera disposed on a spreader and a camera disposed on one side of a container truck.

[0038] like Figure 1 As shown, the automatic box placing method includes steps S101-S104:

[0039] S101: Obtain the position of the target box corner and the position of the target component through a camera arranged on the lifting device.

[0040] The target corner is a corner of a target container grabbed by the spreader, the target component is a component on a free area of ​​a truck plate or a corner of an adjacent container to the target container, and the target corner has a corresponding relationship with the target component.

[0041] The target container grabbed by the spreader is the container to be placed, and the target container corners are all the container corners of the target container.

[0042] It is understandable that under the viewing angle of the camera set on the spreader, the upper and lower corners of the container overlap. Therefore, the positions of the four target corners, that is, the positions of the four corners of the container, can be obtained through the camera set on the spreader.

[0043] In addition, the free area of ​​the vehicle plate is the area on the vehicle plate where no container is loaded. If no container is loaded on the vehicle plate, the free area of ​​the vehicle plate is the entire vehicle plate, and if a container is loaded on the vehicle plate, the free area of ​​the vehicle plate is a partial area of ​​the vehicle plate.

[0044] The adjacent container of the target container is the container adjacent to the target container in the view of the camera installed on the spreader. Generally, the adjacent container is already on the vehicle deck, and the specifications of the adjacent container are the same as those of the target container. It can also be said that the adjacent container of the target container is the container adjacent to the target container on the vehicle deck after the target container is placed.

[0045] Generally, parts of the same type are selected as target parts. When the positions of parts of the same type cannot be obtained or there are parts of the same type that are far from the corners of the target container, parts of different types may be selected as target parts.

[0046] Exemplarily, if the components on the free area of ​​the vehicle plate are the corners of the vehicle plate or the lock on the vehicle plate, the four corners of the vehicle plate can be selected as target components. If a container has been placed on the corner side at the front or rear of the vehicle, resulting in the front or rear corners being far away, the lock on the vehicle plate and the corners of the front or rear of the vehicle can be selected as target components.

[0047] In addition, the cameras on the spreader can be arranged at the four corners of the spreader, so that it can be ensured that the corners of the target container and the target components can be observed in real time through the cameras arranged on the spreader.

[0048] Exemplarily, the camera disposed on the sling may be a gun camera.

[0049] Specifically, the corresponding relationship between the target box corner and the target component is that the target box corner and the target component have a one-to-one correspondence in position and quantity.

[0050] The target box corner corresponds to the target component that is closest to it (referring to the target box corner) among the target components.

[0051] Exemplarily, the target box angles include box angles 1-4, and the target parts include parts ad, wherein box angle 1 corresponds to part a, box angle 2 corresponds to part b, box angle 3 corresponds to part c, and box angle 4 corresponds to target part d.

[0052] Specifically, the target box corners include box corners 1-4, and the target components include components ad, wherein box corner 1 corresponds to component a that is closest to box corner 1 in the target component, box corner 2 corresponds to component b that is closest to box corner 2 in the target component, box corner 3 corresponds to component c that is closest to box corner 3 in the target component, and box corner 4 corresponds to component d that is closest to box corner 4 in the target component.

[0053] S102: adjusting the spreader based on the position of the target container corner and the position of the corresponding target component until the projection of the target container on the plane where the free area is located is within the free area.

[0054] Among them, adjusting the spreader means adjusting the position and posture of the spreader, that is, adjusting the positions of the four corners of the spreader. Since the spreader grabs the target container, the position and posture of the target container changes with the adjustment of the spreader.

[0055] S103: Obtain the position of the target container component and the position of the target truck component through a camera disposed on one side of the truck.

[0056] The target container component and the target truck component are in contact or overlapped with each other after the spreader releases the container.

[0057] The target container component is a component on the target container whose position remains unchanged relative to the target container, such as a lock hole. The target container truck component is a component on the container truck whose position remains unchanged relative to the container truck, such as a lock head.

[0058] Exemplarily, the target container component is the lock hole of the container, and the target truck component is the lock on the truck. After the target container is successfully placed, the target truck component is hidden by the target container component, that is, the target truck component overlaps with the target container component.

[0059] Exemplarily, the target container component is the center point of the side bottom of the target container, and correspondingly, the target truck component can be the center point of the side of the free area of ​​the vehicle board. Of course, the target container component and the target truck component are located on the same side of the truck.

[0060] In addition, the camera on the side of the container truck can be installed on the door leg of the yard bridge, so that it can be ensured that the target container parts and the target container truck parts can be observed in real time through the camera installed on the side of the container truck.

[0061] Generally, the door leg where the camera is located is the door leg close to the sea side of the yard bridge. At this time, the container corner on the side of the container observed by the camera is the sea side container corner, and the container corner on the opposite side of the side is the land side container corner. Similarly, the car plate corner or lock head on the side of the container truck observed by the camera is the sea side corner or lock head, and the car plate corner or lock head on the opposite side of the side is the land side corner or lock head.

[0062] S104: Based on the position of the target container component and the position of the target truck component, control the spreader to place the container to an empty area.

[0063] Specifically, based on the position of the target container component and the position of the target truck component, the position deviation between the target container component and the target truck component is calculated, and based on the position deviation, the spreader is controlled to place the container to an empty area, or the spreader is first adjusted, that is, the spreader is controlled to adjust the position of the target container component, and then the spreader is controlled to place the container to an empty area.

[0064] When the spreader is controlled to release the container to the idle area, a release signal is sent to the spreader, and the spreader moves downward in response to the release signal, driving the target container to move downward and start releasing the container.

[0065] In this embodiment, after obtaining the position of the target box corner and the position of the target component through the camera set on the spreader, and adjusting the spreader to the target container based on the position of the target box corner and the position of the corresponding target component, the projection of the target container on the plane where the free area is located is located in the free area, and then obtaining the position of the target container component and the position of the target truck component through the camera set on the side of the truck. Since the target container component and the target truck component are in contact or overlap after the spreader releases the container, the position relationship between the container and the truck board can be more accurately reflected through the position of the target container component and the position of the target truck component. Therefore, based on the position of the target container component and the position of the target truck, the spreader is controlled to release the container to the free area, which can achieve the effect of releasing the container at the appropriate position, improve the reliability and success rate of the container release, effectively ensure the success of the container release, and solve the problem in the prior art that the camera installed under the spreader may be blocked by the container because the container is located above the truck board, and the position relationship between the container and the truck board cannot be observed, and it is difficult to determine the appropriate container release position, resulting in the failure of the container release.

[0066] In addition, using a camera to obtain the position of a target box corner, a target component, a target container component, and a target truck component can effectively reduce costs compared to using a laser to obtain the position.

[0067] In some embodiments, when obtaining the position of the target box corner and the position of the target component by using a camera disposed on the sling, a first image to be processed is first obtained by using the camera disposed on the sling, and then the target box corner and the target component are extracted from the first image to be processed to obtain the position of the target box corner and the position of the target component.

[0068] Optionally, a first image to be processed is acquired for a target box corner and a target component by a camera arranged on the spreader, and the target box corner and the target component are extracted from the first image to be processed to determine the positions of the target box corner and the target component.

[0069] Specifically, a camera disposed on the lifting device is used to collect images of the target box corner and the area where the target component is located to obtain a first image to be processed.

[0070] Or, specifically, a camera is arranged on the sling to capture a short-time video, for example 1 second, of the target box corner and the target component area, and each image frame in the captured short-time video is extracted to obtain multiple images, and these multiple images are directly used as the first images to be processed, or these multiple images are sampled and the sampled images are used as the first images to be processed.

[0071] Specifically, after obtaining the first image to be processed, a deep learning image segmentation method is used to extract the target box corner and the target component in the first image to be processed, and the position of the target box corner and the position of the target component are obtained.

[0072] Wherein, if the number of the first image to be processed is 1, the target box corner and the target component are directly extracted from the first image to be processed to determine the required position of the target box corner and the position of the target component.

[0073] If the number of first images to be processed is greater than 1, target box corners and target components are extracted from multiple first images to be processed to obtain multiple groups of target box corner positions and target component positions, and the mean or median of the multiple groups of target box corner positions and target component positions is taken, and the mean or median of the multiple groups of target box corner positions is determined as the required target box corner position, and the mean or median of the multiple groups of target box corner positions is determined as the required target box corner position.

[0074] Alternatively, multiple groups of target box corner positions and target component positions are obtained, and the target box corner positions that are significantly different from the target box corner positions in other groups and the target box corner positions that are significantly different from the target box corner positions in other groups are eliminated. The mean or median of the remaining target box corner positions is determined as the desired target box corner position, and the mean or median of the remaining target component positions is determined as the desired target component position.

[0075] The positions of the target box corners and the target components can be represented by coordinates.

[0076] More specifically, when extracting the target box corner and the target component in the first image to be processed and obtaining the position of the target box corner and the position of the target component, after extracting the target box corner and the target component in the first image to be processed, the coordinates of the vertex (or corner point) of the target box corner can be determined as the position coordinates of the target box corner. In addition, according to the different categories of the target component, the coordinates of a point on the target component are determined as the position coordinates of the target component.

[0077] Among them, when the target component is a corner of a vehicle plate or a corner of an adjacent container, the coordinates of the vertices of the vehicle plate corner or the vertices of the corner are determined as the position coordinates of the target component; when the target component is a lock head, the coordinates of the center point of the lock head are determined as the position coordinates of the target component.

[0078] In this embodiment, by using a segmentation algorithm to the first image to be processed acquired by the camera, the target box corners and target components are extracted, and the positions of the target box corners and target components are determined, the cost can be effectively reduced. Compared with extracting the box corner lines through traditional morphological methods, the error in determining the positions of the target box corners and target components can be effectively reduced, and the accuracy and reliability of the positioning of the target box corners and target components can be improved.

[0079] Due to the different loading conditions of containers on the container truck, the types of target parts that can be selected may be limited. For example, when there are already containers on the container truck, the lock closer to the lowering position of the target container may be used and blocked by the loaded container. At this time, the hidden lock cannot be observed by the camera set on the spreader, and the hidden lock cannot be used as a target part.

[0080] Therefore, in some embodiments, when obtaining the position of the target box corner and the position of the target component through the camera disposed on the spreader, the target component can be first determined based on the loading information of the container truck.

[0081] The loading information of the container truck includes the truck board capacity and the information of the containers loaded on the truck board.

[0082] Specifically, the vehicle plate capacity is the number of containers that can be loaded on the vehicle plate, and the information on containers loaded on the vehicle plate includes the number of containers loaded on the vehicle plate.

[0083] For example, the vehicle plate capacity is 1 or 2, that is, the number of containers that can be loaded on the vehicle plate is 1 or 2.

[0084] The vehicle board capacity may also be determined based on actual working conditions. For example, in general, the size of the containers loaded on the same vehicle board is fixed. In this case, the vehicle board capacity may be determined based on the ratio of the vehicle board size to the size of the container to be loaded on the vehicle board.

[0085] Among them, the number of containers loaded on the vehicle is 0, which can also be said that the vehicle is not loaded with containers.

[0086] Specifically, the target parts are determined based on the pallet capacity and the number of containers loaded on the pallet.

[0087] Among them, if the vehicle plate is not loaded with containers, that is, the number of containers loaded on the vehicle plate is 0, and the vehicle plate capacity is one target container, the corners of the vehicle plate are determined as target parts, and the vehicle plate capacity is the number of containers that the vehicle plate can load; if the vehicle plate is not loaded with containers, and the vehicle plate capacity is two target containers, the corners of the vehicle plate near the front or rear side of the vehicle plate and the lock of the vehicle plate are determined as target parts; if the vehicle plate is loaded with the above-mentioned adjacent containers, and the vehicle plate capacity is two target containers, the corners of the vehicle plate near the front or rear side of the vehicle plate and the box corners adjacent to the target container in the adjacent containers are determined as target parts.

[0088] For example, Figure 2 As shown, the box corner of the target container M is AD, and the corner of the vehicle board is ad. If there is no container on the vehicle board of the container truck, and the vehicle board capacity is one target container, the corner ad of the vehicle board is determined as the target component; if there is no container on the vehicle board of the container truck, and the vehicle board capacity is two target containers, the corner ab of the vehicle board and the lock ef on the vehicle board are determined as the target component; if there is one container N on the vehicle board of the container truck, and the vehicle board capacity is two target containers, the corner ab of the vehicle board and the corner gh of the container N are determined as the target component.

[0089] In this embodiment, before obtaining the position of the target box corner and the position of the target component by the camera arranged on the spreader, one or two of the plate corners, the plate corners and the plate lock, and the box corners of the adjacent container are determined as the target components based on whether the plate has been loaded with containers and the plate capacity. In this way, based on the actual working conditions, reasonable target components can be determined. On this basis, the spreader can be better adjusted based on the position of the target box corner and the position of the target component, so that the target container is located above the idle area of ​​the plate, that is, the projection of the target container on the plane where the idle area is located is located in the idle area, thereby ensuring successful container placement.

[0090] In some embodiments, based on the position of the target box corner and the position of the corresponding target component, the sling is adjusted until the projection of the target container on the plane where the idle area is located is located in the idle area, that is, when the target container is located above the idle area, first, based on the position of the target box corner and the position of the corresponding target component, the deviation between the position of the target box corner and the target component is determined, and then based on the deviation between the target box corner and the target position, the projection of the target container on the plane where the idle area is located is controlled to be within the idle area.

[0091] Optionally, based on the position of the target box corner and the position of the corresponding target component, the deviation between the target box corner and the corresponding target component in the truck direction and the trolley direction is determined, and based on the deviation between the target box corner and the corresponding target component in the truck direction and the trolley direction, the spreader is adjusted until the projection of the target container on the plane where the idle area is located is within the idle area.

[0092] Among them, the direction of the large vehicle is the direction of the container truck, the direction of the small vehicle is the direction of the yard bridge, and the direction of the yard bridge is perpendicular to the direction of the container truck.

[0093] In addition, since the position of the target box corner and the corresponding target component can be represented by coordinates, the deviation of the target box corner and the corresponding target component in the direction of the large vehicle and the small vehicle can be determined based on the difference between the position coordinates of the target box corner and the corresponding target component.

[0094] For example, the direction of the vehicle is Figure 3 The x-axis direction is shown, and the direction of the car is Figure 3 When the target component is corner a1, the deviation between corner A and corner a1 is e1(△x1, △y1), where △x1 is the deviation between corner A and corner a1 in the direction of the trolley, △x1 is the absolute value of the difference between the x-coordinate of corner A and the x-coordinate of corner a1, △y1 is the deviation between corner A and corner a1 in the direction of the trolley, and △y1 is the absolute value of the difference between the y-coordinate of corner A and the y-coordinate of corner a1. When the target component is lock a2, the deviation between box corner A and lock a2 is e2(△x2, △y2), wherein △x2 is the deviation between box corner A and lock a2 in the direction of the trolley, △x2 is the absolute value of the difference between the x-coordinate of box corner A and the x-coordinate of lock a2, △y2 is the deviation between box corner A and lock a2 in the direction of the trolley, and △y2 is the absolute value of the difference between the y-coordinate of box corner A and the y-coordinate of lock a2. When the target component is the box corner a3 of the adjacent container, the deviation between the box corner A and the box corner a3 of the adjacent container is e3(△x3, △y3), wherein △x3 is the deviation between the box corner A and the box corner a3 of the adjacent container in the direction of the large vehicle, the value of △x3 is the absolute value of the difference between the x-coordinate of the box corner A and the x-coordinate of the box corner a3 of the adjacent container, △y3 is the deviation between the box corner A and the box corner a3 of the adjacent container in the direction of the small vehicle, and the value of △y3 is the absolute value of the difference between the y-coordinate of the box corner A and the y-coordinate of the box corner a3 of the adjacent container.

[0095] In other words, the deviation between the target box angle and the corresponding target component in the direction of the large vehicle is the distance between the target box angle and the corresponding target component in the direction of the large vehicle, and the deviation between the target box angle and the corresponding target component in the direction of the small vehicle is the distance between the target box angle and the corresponding target component in the direction of the small vehicle.

[0096] Specifically, based on the position of the target container corner and the position of the corresponding target component, the deviation between the target container corner and the corresponding target component in the direction of the large vehicle is determined as the first deviation of the target container corner, and based on the position of the target container corner and the position of the corresponding target component, the deviation between the target container corner and the corresponding target component in the direction of the small vehicle is determined as the second deviation of the target container corner. Afterwards, based on the first deviation of the target container corner and the second deviation of the target container corner, the spreader is adjusted until the projection of the target container on the plane where the free area is located is located within the free area.

[0097] It is possible that before adjusting the spreader, the target container is already above the idle area, that is, the projection of the target container on the plane where the idle area is located is located in the idle area. Therefore, specifically, the spreader is adjusted based on the first deviation of the target box angle and the second deviation of the target box angle until the projection of the target container on the plane where the idle area is located is located in the idle area. First, based on the first deviation of the target box angle and the second deviation of the target box angle, it is determined whether the projection of the target container on the plane where the idle area is located is located in the idle area. Then, based on whether the projection of the target container on the plane where the idle area is located is located in the idle area, the spreader is adjusted in combination with the first deviation and the second deviation of the target box angle.

[0098] More specifically, if no, that is, if it is determined that the projection of the target container on the plane where the free area is located is not located in the free area, the spreader is adjusted based on the first deviation of the target box angle and the second deviation of the target box angle until the projection of the target container on the plane where the free area is located is located in the free area.

[0099] Correspondingly, if yes, that is, it is determined that the projection of the target container on the plane where the free area is located is within the free area, an operation is performed by obtaining the position of the target container component and the position of the target container component through a camera arranged on one side of the container truck, that is, executing the above step S103.

[0100] In this embodiment, based on the deviation between the target container corner and the corresponding target component in the direction of the large vehicle and the direction of the small vehicle, it is determined that the projection of the target container on the plane where the free area is located is located in the free area, and when the projection of the target container on the plane where the free area is located is not located in the free area, the spreader is adjusted so that the position of the target container corner, i.e., the target container corner, grasped by the spreader changes, and the relative position of the target container and the free area changes until the projection of the target container on the plane where the free area is located is located in the free area. In this way, it can be ensured that the target container is located in the middle of the free area after being placed, effectively ensuring the successful placement of the container.

[0101] When the target container is located above the idle area, that is, when the projection of the target container on the plane where the idle area is located is located in the idle area, the deviation of the target container angle and the target component in the direction of the large vehicle is substantially consistent, and the deviation of the target container angle and the target component in the direction of the small vehicle is substantially consistent. Therefore, in some embodiments, when judging whether the projection of the target container on the plane where the idle area is located is located in the idle area based on the first deviation of the target angle and the second deviation of the target angle, the judgment can be made in combination with whether the first deviation of the target angle is consistent and whether the second deviation of the target angle is consistent.

[0102] Specifically, if the first deviation of the target box corner is consistent and the second deviation of the target box corner is consistent, it is determined that the projection of the target container on the plane where the idle area is located is located in the idle area; if the first deviation of the target box corner is inconsistent or the second deviation of the target box corner is inconsistent, it is determined that the projection of the target container on the plane where the idle area is located is not located in the idle area.

[0103] Exemplarily, if the first deviations a1-a4 of the target box angle are the same, or the difference between a1-a4 does not exceed the preset difference A, it is determined that the first deviations of the target box angle are consistent. If the first deviations a1-a4 of the target box angle are different from each other, or the difference between a1-a4 exceeds the preset difference A, it is determined that the first deviations of the target box angle are inconsistent.

[0104] Among them, whether the first deviations of the target box angles are consistent can be determined based on whether the difference between the first deviations of the target box angles is less than the first preset difference. Specifically, if the difference between the first deviations of any two target box angles is less than the first preset difference, it is determined that the first deviations of the two target box angles are consistent; on the contrary, if the difference between the first deviations of any two target box angles is greater than or equal to the first preset difference, it is determined that the first deviations of the two target box angles are inconsistent.

[0105] Accordingly, whether the second deviations of the target box corners are consistent can be determined based on whether the difference between the second deviations of the target box corners is less than the second preset difference. Specifically, if the difference between the second deviations of any two target box corners is less than the second preset difference, it is determined that the second deviations of the two target box corners are consistent; on the contrary, if the difference between the second deviations of any two target box corners is greater than or equal to the second preset difference, it is determined that the second deviations of the two target box corners are inconsistent.

[0106] Specifically, if the difference between the first deviations of all target box corners is less than the first preset difference, and the difference between the second deviations of all target box corners is less than the second preset difference, then the projection of the target container on the plane where the idle area is located is located within the idle area; if the difference between the first deviations of the target box corners is greater than or equal to the first preset difference, or the difference between the second deviations of the target box corners is greater than or equal to the second preset difference, then it is determined that the projection of the target container on the plane where the idle area is located is located within the idle area.

[0107] Alternatively, since both the container and the truck bed are rectangular, if the first deviation of the target container corner on the same side of the truck in the direction of the large truck is consistent, then the first deviation of the target container corner on the opposite side in the direction of the large truck is also consistent. Similarly, if the second deviation of the target container corner on the same side of the truck in the direction of the small truck is consistent, then the second deviation of the target container corner on the opposite side is also consistent. At this time, whether the projection of the target container on the plane where the free area is located is within the free area.

[0108] Therefore, specifically, if the difference of the first deviation of the target container corner located on the same side of the truck in the direction of the large vehicle is less than the first preset difference, and the second deviation of the target container corner located on the same side of the truck in the direction of the small vehicle is less than the second preset value, it is determined whether the projection of the target container on the plane where the free area is located is located in the free area. Correspondingly, if the difference of the first deviation of the target container corner located on the same side of the truck in the direction of the large vehicle is greater than or equal to the first preset difference, or the difference of the second deviation of the target container corner located on the same side of the truck in the direction of the small vehicle is greater than or equal to the second preset difference, it is determined that the projection of the target container on the plane where the free area is located is not located in the free area.

[0109] Exemplarily, taking a truck pallet that can carry a single 40-foot container as an example, the deviations of each corner of the container and the corresponding corner of the pallet are e1 (△x1, △y1), e2 (△x2, △y2), e3 (△x3, △y3) and e4 (△x4, △y4). Among them, e1 (△x1, △y1) and e3 (△x3, △y3) are the deviations of the target corner of the container on the same side of the truck in the direction of the large vehicle and the corresponding corner of the pallet, and e1 (△x1, △y1) and e2 (△x2, △y2) are the deviations of the target corner of the container on the same side of the truck in the direction of the small vehicle and the corresponding corner of the pallet. In addition, in the coordinates of e1-e4, the x coordinate represents the first deviation of the target corner, and the y coordinate represents the second deviation of the target corner.

[0110] At this time, if the value of |△x1-△x3| is less than the first preset difference, and the value of |△y1-△y2| is less than the second preset difference, it is determined that the projection of the single 40-foot container on the plane where the idle area is located is located in the idle area; if the value of |△x1-△x3| is greater than or equal to the first preset difference, and the value of |△y1-△y2| is greater than or equal to the second preset difference, it is determined that the projection of the single 40-foot container on the plane where the idle area is located is not located in the idle area.

[0111] The first preset difference and the second preset difference are both determined based on actual working conditions, and may be the same or different.

[0112] Exemplarily, the first preset difference and the second preset difference are both 15 mm.

[0113] In this embodiment, when the projection of the target container on the plane where the free area is located is within the free area, the deviations of the target container corner and the target component in the direction of the large vehicle are substantially consistent, and the deviations of the target container corner and the target component in the direction of the small vehicle are substantially consistent. Based on whether the difference between the first deviations of the target container corners is less than the first preset difference and whether the difference between the second deviations of the target container corners is less than the second preset difference, it is determined whether the projection of the target container on the plane where the free area is located is within the free area, which can more accurately determine whether the projection of the target container on the plane where the free area is located is within the free area, thereby ensuring a better container placement effect.

[0114] In some embodiments, when obtaining the position of the target container component and the position of the target truck component by using a camera disposed on one side of the truck, the second image to be processed is first obtained by using the camera disposed on one side of the truck, and then the target container component and the target truck component are extracted from the second image to be processed to obtain the positions of the target container component and the target truck component.

[0115] Optionally, a second image to be processed is acquired for the target container component and the target truck component through a camera arranged on one side of the truck, and the target container component and the target truck component are extracted from the second image to be processed to determine the positions of the target container component and the target truck component.

[0116] Specifically, a camera disposed on one side of the container truck is used to collect images of the area where the target container component and the target container truck component are located to obtain a second image to be processed.

[0117] Alternatively, specifically, a camera disposed on one side of the container truck is used to capture a short-time video, for example, 1 second, of the area where the target container component and the target container truck component are located, and each image frame in the captured short-time video is extracted to obtain a plurality of images, and the plurality of images are directly used as the second images to be processed, or the plurality of images are sampled and the sampled images are used as the second images to be processed.

[0118] Specifically, after obtaining the second image to be processed, the target container component and the target truck component in the second image to be processed are extracted using a deep learning image segmentation algorithm to obtain the position of the target container component and the position of the target truck component. The specific implementation can refer to the above-mentioned extraction of the target box corner and the target component in the first image to be processed to obtain the position of the target box corner and the target component, which will not be described in detail here.

[0119] The positions of the target container component and the target truck component can be represented by coordinates.

[0120] More specifically, after extracting the target container component and the target truck component in the second image to be processed, the coordinates of the corner points or the center points of the target container component are determined as the position coordinates of the target container component, and the coordinates of the corner points or the center points of the target truck component are determined as the position coordinates of the target truck component.

[0121] In this embodiment, by using a segmentation algorithm or the like on the second image to be processed acquired by the camera, the target container component and the target truck component are extracted, and the position of the target container component and the target truck component are determined. Compared with using laser, the cost is effectively reduced, and compared with extracting the box corner line by traditional morphology, the error in determining the position of the target box corner and the target component can be reduced, and the accuracy of determining the positioning of the target container component and the target truck component can be improved. In addition, by adding a camera on the side of the truck, the position of the target container component and the target truck component can be continuously observed after the projection of the container on the plane where the idle area is located is located in the idle area, so as to facilitate the selection of a suitable position for placing the container and improve the success rate of placing the container.

[0122] In some embodiments, when controlling the spreader to place the container to an idle area based on the position of the target container component and the position of the target truck component, the position deviation of the target container component and the corresponding target truck component is determined based on the position of the target container component and the position of the target truck component, and based on the position deviation, it is determined whether the projection of the target container component on the plane where the target truck component is located is located within the target truck component, and based on whether the projection of the target container component on the plane where the target truck component is located is located within the target truck component, the spreader is controlled to place the container to the idle area.

[0123] The position deviation between the target container component and the corresponding target truck component includes a deviation in the direction of the truck.

[0124] Specifically, based on the position of the target container component and the position of the target truck component, the deviation between the target container component and the corresponding target truck component in the direction of the large vehicle is determined as the target deviation. Afterwards, based on the target deviation, it is determined whether the projection of the target container component on the plane where the target truck component is located is located within the target truck component. If not, that is, the projection of the target container component on the plane where the target truck component is located is not located within the target truck component, then based on the target deviation, the spreader is adjusted until the projection of the target container component on the plane where the target truck component is located is located within the target truck component; if so, the spreader is controlled to place the container in an idle area.

[0125] Exemplarily, taking the truck direction as the x-axis, the difference between the x-coordinate of the position coordinate of the target container component and the x-coordinate of the position coordinate of the target truck component is the deviation between the target container component and the target truck component in the truck direction.

[0126] When the projection of the target container component on the plane where the target truck component is located is within the target truck component, the deviation between the target container component and the target truck component in the direction of the large vehicle is small, that is, the target deviation is small. The size of the target deviation can be measured by the size relationship between it and the preset target deviation. If the target deviation is greater than or equal to the preset target deviation, it is determined that the target deviation is large, and the projection of the target container component on the plane where the target truck component is located is not within the target truck component. If the target deviation is less than the preset target deviation, it is determined that the target deviation is small, and at this time, the projection of the target container component on the plane where the target truck component is located is within the target truck component.

[0127] Exemplarily, the preset target deviation is 15 mm.

[0128] In this embodiment, after the positions of the target container component and the target truck component are acquired by the camera on one side of the truck, the positional relationship between the target container and the truck deck can be accurately determined. Thereafter, based on the positional deviation between the target container component and the target truck component, it can be relatively accurately determined whether the projection of the target container component on the plane where the target truck component is located is located within the target truck component. The projection of the target container component on the plane where the target truck component is located is located within the target truck component, which can better ensure successful container placement. Therefore, based on the positional deviation between the target container component and the target truck component, the projection of the target container component on the plane where the target truck component is located can be relatively accurately and quickly adjusted to be within the target truck component by adjusting the spreader, and the spreader can be controlled to place the container, thereby realizing automatic container placement and ensuring the accuracy of container placement.

[0129] In some embodiments, after controlling the spreader to release the container, the position of the target container component and the position of the target truck component are continuously obtained through the camera on the side of the container truck, and then based on the position of the target container component and the position of the target truck component, it is determined whether the target container is in place, and based on whether the target container is in place, the spreader is controlled to continue releasing the container or to stop releasing the container and release the target container.

[0130] The introduction on obtaining the position of the target container component and the position of the target truck component can be found in the above content, which will not be elaborated here.

[0131] Since the projection of the target container component on the plane where the target truck component is located is located inside the target truck component, and the target container component and the target truck component are in contact or overlapped after the container is released, the height difference between the target container component and the target truck component is small after the container is released.

[0132] Specifically, when judging whether the target container is in place based on the position of the target container component and the position of the target truck component, the height difference between the target container component and the component truck component can be determined based on the position of the target container component and the position of the target truck component, and based on the height difference, it is determined whether the target container is in place.

[0133] Exemplarily, with the vertical direction as the y-axis, the difference between the y-coordinate of the position coordinate of the target container component and the y-coordinate of the position coordinate of the target truck component is the height difference between the target container component and the target truck component, that is, the height difference between the container and the truck deck.

[0134] The position coordinates of the target container component may be the coordinates of any point on the target container component, such as the coordinates of the point in the target container component that is closest to the target truck component. Similarly, the position coordinates of the target truck component may be the coordinates of any point on the target container component, such as the coordinates of the point in the target truck component that is farthest from the target container component. Generally, the position coordinates of the target truck component and the position coordinates of the target container component will be taken as points on the vertical center line of the two to ensure that the projection of the target container component on the plane where the target truck component is located is located within the target truck component.

[0135] More specifically, if the height difference between the target container component and the target truck component is less than or equal to the preset height difference, it is determined that the target container is in place; if the height difference between the target container component and the target truck component is greater than the preset height difference, it is determined that the target container is not in place.

[0136] Correspondingly, when controlling the spreader to continue lowering the container or to stop lowering the container and release the target container based on whether the target container is in place, if the target container is not in place, the spreader is controlled to continue lowering the container; if the target container is in place, the spreader is controlled to stop lowering the container and release the target container.

[0137] Exemplarily, taking the target truck component as the lock head, the target container component as the lock hole, and the preset height difference as H as an example, based on the positions of the lock head and the lock hole obtained by the camera on one side of the truck as Figure 4 shown in (a) of the figure, the lock hole is above the lock head, that is, the projection of the lock hole on the plane where the lock head is located is inside the lock head, and the height difference between the lock head and the lock hole is h1. If h1 > H, it is determined that the target container is not in place. At this time, the spreader is controlled to continue lowering the container. After that, based on the positions of the lock head and the lock hole obtained by the camera on one side of the truck as Figure 4 shown in (b) of the figure, the height difference between the lock head and the lock hole is reduced to h2, and h2 < H, then it is determined that the target container is in place, and the spreader is controlled to stop lowering the container and release the target container to complete the container lowering.

[0138] In addition, if the target container component and the target truck component coincide after the target container is in place, it is also possible to determine whether the target container is in place based on whether the target container component or the target truck component disappears.

[0139] Specifically, through the camera set on one side of the truck, for the target container component and the target truck component, a second image to be processed is continuously obtained. If the target truck component or the target container component cannot be extracted from the continuously obtained second image to be processed, it is determined that the target container is in place. On the contrary, if the target truck component and the target container component can be extracted, it is determined that the target container is not in place.

[0140] In this embodiment, after the spreader is controlled to release the container, the position of the target container component and the position of the target truck component are continuously obtained through the camera on one side of the truck, and based on the continuously obtained position of the target container component and the position of the target truck component, it is determined whether the target container is in place, and based on whether the target container is in place, the spreader is controlled to continue to release the container or to stop releasing the container and release the target container. Among them, since the projection of the target container component on the plane where the target truck component is located is located inside the target truck component, and the target container component and the target truck component are in contact or overlap after the container is released, as the container is gradually lowered, the distance between the target container component and the target truck component, including the height difference, will become smaller and smaller until it is close to or 0. Therefore, based on the position of the target container component and the position of the target truck component, it can be more accurately determined whether the target container is in place. In addition, based on whether the target container is in place, controlling the spreader to continue to release the container or stop releasing the container can better ensure the effect of releasing the container, ensure that the container is in place, and avoid unnecessary waste of resources.

[0141] It can be understood that in the process of controlling the spreader to release the container, the camera on one side of the container truck can be used to continuously monitor the deviation between the target container component and the target container truck component in the direction of the truck, and when the relative position between the target container component and the target container truck component is offset, that is, when the projection of the target container component on the plane where the target container truck component is located is not within the target container truck component, the target spreader is adjusted in time based on the target deviation to keep the projection of the target container component on the plane where the target container truck component is located continuously within the target container truck component, so as to achieve a better container release effect and ensure successful container release.

[0142] In addition, in the process of controlling the spreader to release the container, the speed of the spreader to release the container can also be controlled based on the height difference between the target truck component and the target container component. As the height difference decreases, the speed of the spreader to release the container is controlled to decrease to avoid failure in releasing the container and reduce the noise of releasing the container. Among them, the height difference between the target truck component and the target container component is positively correlated with the speed of the spreader to release the container.

[0143] For example, for the working condition that the truck bed can be loaded with a single 40-foot container, a single 20-foot container, and a double 20-foot container, different processing methods are used to control the spreader to release the container. The automatic container release process can be as follows: Figure 5 shown.

[0144] For the working condition that the vehicle board can load a single 40-foot container, that is, Figure 2In the first case (of the three cases from top to bottom), the target component can be determined to be the corner of the truck board. Based on the image collected by the camera set on the spreader, the corner coordinates of the truck board are calculated, and at the same time, the box corner coordinates (of a single 40-foot container) are calculated. After that, the deviations e1-e4 of the corner coordinates and the box corner coordinates are calculated, namely e1 (△x1, △y1), e2 (△x2, △y2), e3 (△x3, △y3) and e4 (△x4, △y4). Among them, e1 and e3 are the deviations of the target box corner on the same side of the truck in the direction of the large truck and the corresponding truck board corner, and e1 and e2 are the deviations of the target box corner on the same side of the truck in the direction of the small truck and the corresponding truck board corner. In the coordinates of e1-e4, the x coordinate represents the first deviation of the target box corner, and the y coordinate represents the second deviation of the target box corner. Afterwards, based on e1-e3, the difference in the direction of the large vehicle, i.e., the difference in the first deviation between e1 and e3 |△x1-△x3|, and the difference in the direction of the small vehicle, i.e., the difference in the second deviation between e1 and e2 |△y1-△y2| are calculated.

[0145] Among them, e1 and e3 are the deviations between the two land-side target box corners and the corresponding target components, namely the vehicle plate corners, and e2 and e4 are the deviations between the two sea-side target box corners and the corresponding target components, namely the vehicle plate corners.

[0146] For the condition that the truck can be loaded with double 20-foot containers but has not yet been loaded with containers, that is, the above Figure 2 In the second case (of the three cases from top to bottom) given, it can be determined that the target parts include the corners of the container plate and the plate lock. Based on the image captured by the camera set on the spreader, the box corner coordinates (of the target container) are calculated, and the corner coordinates of the plate and the lock coordinates of the plate are calculated at the same time. After that, based on the box corner coordinates and the corner coordinates, the deviation e1-e2 of the sea and land side corners and box corners is calculated, and based on the box corner coordinates and the lock coordinates, the deviation e3-e4 of the sea and land side box corners and locks is calculated. Based on e1-e3, the difference in the direction of the large vehicle, that is, the difference of the first deviation of e1 and e3 |△x1-△x3|, and the difference in the direction of the small vehicle, that is, the difference of the second deviation of e1 and e2 |△y1-△y2| are calculated.

[0147] For the truck bed that can carry two 20-foot containers but has already loaded one container, that is, Figure 2For the third case (out of the three cases from top to bottom) given, it can be determined that the target components include the corners of the container truck bed and the corners of the adjacent containers (i.e., the above-mentioned adjacent containers) that have been loaded. Based on the images captured by the cameras installed on the spreader, calculate the corner coordinates of the (target container), and at the same time calculate the corner coordinates of the truck bed and the corner coordinates of the adjacent containers. Then, based on the corner coordinates and the corner coordinates of the truck bed, calculate the deviation e1 - e2 between the sea-land side corner and the corner, and based on the corner coordinates and the corner coordinates of the adjacent containers, calculate the deviation e3 - e4 between the sea-land side corner of the container and the corner of the adjacent container. Based on e1 - e3, calculate the difference in the trolley direction, that is, the difference |△x1 - △x3| between the first deviations of e1 and e3, and the difference in the hoist direction, that is, the difference |△y1 - △y2| between the second deviations of e1 and e2.

[0148] Through the above process, after calculating the difference in the trolley direction and the difference in the hoist direction, determine whether the value of |△x1 - △x3| is less than the first preset difference E1, and whether the value of |△y1 - △y2| is less than the second preset difference E2. If the value of |△x1 - △x3| is less than the first preset difference E1, that is, |△x1 - △x3| < E1, and the value of |△y1 - △y2| is less than the second preset difference E2, that is, |△y1 - △y2| < E2, then it is determined that the container is above the container truck bed, that is, the projection of the container component on the plane where the container truck bed is located is on the container truck bed. Then, if it is determined through the camera installed on the gantry leg that the lock hole is above the lock head, that is, the projection of the lock hole on the plane where the lock head is located is within the lock head, control the spreader to lower the container; if |△x1 - △x3| >= E1, or |△y1 - △y2| >= E2, then it is determined that the projection of the container on the plane where the container truck bed is located is not within the container truck bed, adjust the spreader, and recalculate the target corner coordinates and the target component coordinates.

[0149] Exemplary Devices

[0150] Correspondingly, as Figure 6 shown, an embodiment of the present application further provides an automatic container lowering device, including an acquisition module 601, an identification module 602, and a determination module 603.

[0151] Among them,

[0152] The acquisition module 601 is used to obtain the position of the target corner and the position of the target component through the camera installed on the spreader. The target corner is the corner of the target container grabbed by the spreader, and the target component is a component on the idle area of the container truck bed of the container carrier or the corner of the adjacent container of the target container. The target corner and the target component have a corresponding relationship;

[0153] An adjustment module 602 is used to adjust the spreader based on the position of the target container corner and the corresponding position of the target component until the projection of the target container on the plane where the free area is located is located within the free area;

[0154] The acquisition module 601 is further used to acquire the position of the target container component and the position of the target truck component through a camera arranged on one side of the truck, the target container component and the target truck component being in contact or overlapped with each other after the spreader places the container in the idle area, the target container component being a component on the target container whose position remains unchanged relative to the target container, and the target truck component being a component on the truck whose position remains unchanged relative to the truck;

[0155] The control module 603 is used to control the spreader to place the container to the free area based on the position of the target container component and the position of the target truck component.

[0156] The automatic box placing device provided in this embodiment belongs to the same application concept as the automatic box placing method provided in the above embodiments of this application, and can execute the method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in this embodiment, please refer to the specific processing content of the automatic box placing method provided in the above embodiments of this application, and will not be repeated here.

[0157] The functions implemented by the above acquisition module 601, adjustment module 602 and control module 603 can be implemented in the form of software called by the same or different processors, and the embodiment of the present application is not limited thereto.

[0158] Exemplary Systems

[0159] The present application also provides an automatic container placement system, which includes a controller (not shown), a field bridge, a container truck, a (target) container, and a camera. The structural diagram of the system can be shown as follows: Figure 7 shown.

[0160] The yard crane is composed of gantry legs, spreaders, etc. The gantry legs are used to support the yard crane, and the spreader is used to grab the container on one side and place it on the container truck deck.

[0161] Container trucks are used to load and transport containers.

[0162] For example, the camera viewing angle may be Figure 7 As shown, the cameras include cameras ad arranged at the four corners of the spreader, and a camera ef arranged on the door leg AB close to the sea.

[0163] Camera ad is used to obtain the position of the target box corner and the position of the target component, and camera ef is used to obtain the position of the target container component and the position of the target truck component.

[0164] The controller is arranged in the container truck or yard bridge or other equipment that can communicate with the camera and control the spreader, and is used to control the spreader to release the container based on the position obtained by the camera.

[0165] For the introduction of the specific implementation of each part of the system, please refer to the above content and will not be repeated here.

[0166] Exemplary Electronic Devices

[0167] Another embodiment of the present application also provides an electronic device, see Figure 8 As shown, the electronic device includes: a memory 800 and a processor 810 .

[0168] The memory 800 is connected to the processor 810 and is used to store programs;

[0169] The processor 810 is used to implement the automatic box placing method disclosed in any of the above embodiments by running the program stored in the memory 800.

[0170] Specifically, the electronic device may further include: a bus, a communication interface 820 , an input device 830 and an output device 840 .

[0171] The processor 810, the memory 800, the communication interface 820, the input device 830 and the output device 840 are connected to each other via a bus.

[0172] A bus may include a pathway that transfers information between components of a computer system.

[0173] The processor 810 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0174] The processor 810 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0175] The memory 800 stores a program for executing the technical solution of the present application, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 800 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0176] The input device 830 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0177] Output device 840 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0178] The communication interface 820 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0179] The processor 810 executes the program stored in the memory 800 and calls other devices, which can be used to implement each step of any automatic box placement method provided in the above embodiments of the present application.

[0180] Those skilled in the art will understand that Figure 8 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0181] An embodiment of the present application also proposes a chip, which includes a processor and a data interface. The processor reads and runs the program stored in the memory through the data interface to execute the automatic box placement method introduced in any of the above embodiments. The specific processing process and its beneficial effects can be found in the introduction to the above-mentioned embodiment of the automatic box placement method.

[0182] An embodiment of the present application further provides a construction machinery, in which the above-mentioned automatic box placing device or the above-mentioned electronic device is provided.

[0183] Exemplarily, the engineering machinery may be a field crane.

[0184] In addition to the above-mentioned methods and devices, an embodiment of the present application proposes a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the automatic box placement method according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0185] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0186] In addition, an embodiment of the present application further proposes a storage medium on which a computer program is stored, and the computer program is executed by a processor to execute the steps of the automatic box placement method according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0187] It should be understood that the specific examples in this article are only intended to help those skilled in the art better understand the implementation methods of this specification, rather than to limit the scope of this specification.

[0188] It can be understood that in the various implementations of this specification, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the implementation methods of this specification.

[0189] It can be understood that the various embodiments described in this specification can be implemented individually or in combination, and the embodiments of this specification are not limited to this.

[0190] Unless otherwise specified, all technical and scientific terms used in the embodiments of this specification have the same meaning as those generally understood by those skilled in the art of the technical field of this specification. The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the scope of this specification. The term "and / or" used in this specification includes any and all combinations of one or more related listed items. The singular forms of "a", "above", and "the" used in the embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0191] It can be understood that the processor of the embodiment of this specification can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method implementation can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiment of this specification can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of this specification can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and performed. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0192] It is understood that the memory in the embodiments of this specification may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (ErasablePROM, EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0193] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this specification.

[0194] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method implementation methods and will not be repeated here.

[0195] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0196] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0197] In addition, each functional unit in each embodiment of the present specification may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0198] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of this specification. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0199] The above is only a specific implementation of this specification, but the protection scope of this specification is not limited to this. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this specification, which should be included in the protection scope of this specification. Therefore, the protection scope of this specification should be based on the protection scope of the claims.

Claims

1. An automatic box placing method, characterized in that: The method comprises: The position of the target container corner and the position of the target component are obtained by a camera arranged on the spreader, wherein the target container corner is the container corner of the target container grabbed by the spreader, and the target component is a component on the free area of ​​the truck plate or a container corner of an adjacent container to the target container, and the target container corner has a corresponding relationship with the target component; Based on the position of the target container corner and the position of the corresponding target component, adjusting the spreader until the projection of the target container on the plane where the free area is located is within the free area; The position of the target container component and the position of the target truck component are obtained by a camera arranged on one side of the truck, the target container component and the target truck component are in contact with or overlapped with each other after the spreader places the container, the target container component is a component on the target container whose position remains unchanged relative to the target container, and the target truck component is a component on the truck whose position remains unchanged relative to the truck; Based on the position of the target container component and the position of the target truck component, controlling the spreader to place the container to the idle area; Before obtaining the position of the target box corner and the position of the target component by using a camera disposed on the spreader, the method further includes: If the vehicle plate is not loaded with containers, and the vehicle plate capacity is one of the target containers, then the corners of the vehicle plate are determined as the target parts, and the vehicle plate capacity is the number of containers that can be loaded on the vehicle plate; If the vehicle plate is not loaded with containers and the vehicle plate capacity is two of the target containers, the corner of the vehicle plate near the front or rear of the vehicle plate and the lock head of the vehicle plate are determined as the target parts; If the vehicle plate has been loaded with one adjacent container and the vehicle plate capacity is two target containers, the corners of the vehicle plate near the front or rear side and the corners of the adjacent containers adjacent to the target container are determined as the target parts.

2. The automatic box placing method according to claim 1, characterized in that: The method of obtaining the position of the target box corner and the position of the target component by using a camera disposed on the lifting device includes: Acquire a first image to be processed for the target box corner and the target component through the camera disposed on the sling; The target box corner and the target component are extracted from the first image to be processed, and the position of the target box corner and the position of the target component are determined.

3. The automatic box placing method according to claim 1, characterized in that: The step of adjusting the spreader based on the position of the target container corner and the position of the corresponding target component until the projection of the target container on the plane where the idle area is located is located within the idle area comprises: Based on the position of the target box angle and the position of the corresponding target component, determining the deviation between the target box angle and the corresponding target component in the vehicle direction as a first deviation of the target box angle; Based on the position of the target box corner and the position of the corresponding target component, determining the deviation between the target box corner and the corresponding target component in the direction of the trolley as the second deviation of the target box corner; Based on the first deviation of the target container angle and the second deviation of the target container angle, determining whether the projection of the target container on the plane where the free area is located is within the free area; If not, the spreader is adjusted based on the first deviation of the target container angle and the second deviation of the target container angle until the projection of the target container on the plane where the free area is located is within the free area.

4. The automatic box placing method according to claim 3, characterized in that: The determining, based on the first deviation of the target container angle and the second deviation of the target container angle, whether the projection of the target container on the plane where the idle area is located is located in the idle area comprises: If the differences between the first deviations of the target container corners are all smaller than the first preset difference, and the differences between the second deviations of the target container corners are all smaller than the second preset difference, it is determined that the projection of the target container on the plane where the free area is located is located in the free area; If the difference between the first deviations of the target container corners is greater than or equal to the first preset difference, or the difference between the second deviations of the target container corners is greater than or equal to the second preset difference, it is determined that the target container is not located above the idle area.

5. The automatic box placing method according to claim 1, characterized in that: The method of obtaining the position of the target container component and the position of the target truck component by using a camera disposed on one side of the truck includes: By using a camera disposed on one side of the container truck, a second image to be processed is acquired for the target container component and the target container truck component; The target container component and the target truck component are extracted from the second image to be processed, and the positions of the target container component and the target truck component are determined.

6. The automatic box placing method according to claim 5, characterized in that: The controlling the spreader to place the container to the idle area based on the position of the target container component and the position of the target truck component comprises: Based on the position of the target container component and the position of the target truck component, determining the deviation between the target container component and the corresponding target truck component in the direction of the large vehicle as a target deviation; Based on the target deviation, determining whether the target container component is located above the target truck component; If not, adjusting the spreader based on the target deviation until the projection of the target container component on the plane where the target truck component is located is located within the target truck component; If so, the spreader is controlled to place the box in the idle area.

7. An electronic device, characterized in that: including memory and processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the automatic box placement method as described in any one of claims 1 to 6 by running the program in the memory.

8. An engineering machine, characterized in that: The construction machine is provided with the electronic device according to claim 7.

Images