A cargo box positioning method, device, equipment and storage medium

Abstract

The cargo box positioning method, device, equipment, and storage medium provided by the embodiments of the present application obtain the image information of the identification mark to be recognized on the shelf, calculate the first position information of the identification mark to be recognized based on the image information, then obtain the second position information of the identification mark to be recognized, calculate the general position coordinate conversion relationship based on these two position information, and finally determine the position information of all cargo boxes to be carried relative to the box-taking device based on this conversion relationship, so as to pick up the cargo box to be carried. Since the present application does not require a visual identification code to be installed at each grid opening on the shelf, the manufacturing cost and installation cost of the visual identification code are reduced; in addition, since only the image information of the identification mark to be recognized needs to be collected once, and the position information of all cargo boxes can be determined at one time through the calculated conversion relationship, the efficiency of positioning the cargo boxes is improved.

Description

[0001] This application claims the priority of a Chinese patent application with the application number 202211609084.X and the invention title "A Cargo Box Positioning Method, Device, Equipment and Storage Medium" filed on December 14, 2022, the entire content of which is incorporated herein by reference. Technical Field

[0002] This application relates to the field of robotics, and particularly to a cargo box positioning method, device, equipment and storage medium. Background Art

[0003] In a warehouse environment, when carrying the cargo boxes on the shelves in the warehouse, usually a transfer robot first transports the shelf with multiple cargo boxes to a preset fixed position at the workstation, and then a robotic arm in the box picking device at the workstation picks up the corresponding cargo box, so as to transport the cargo box to the target position. However, due to reasons such as the abnormal motion control system of the transfer robot itself and the installation error of the ground identification code that guides the movement of the transfer robot, there will be a certain deviation between the actual position where the transfer robot transports and places the shelf and the preset fixed position, which further causes the robotic arm in the box picking device to be unable to pick up the corresponding cargo box. Therefore, it is necessary to accurately locate the current position of the cargo box.

[0004] In the related art, in order to accurately locate the current position of the cargo box, usually a visual identification code is set at each grid opening on the shelf where the cargo box is placed. After roughly positioning the to-be-carried cargo box through the pre-set position information, the visual acquisition device in the box picking device is used to obtain the image information of the visual identification code at the grid opening where the to-be-carried cargo box is placed, and then the position information of the to-be-carried cargo box relative to the visual scanning device is calculated based on the image information. Since the visual scanning device is installed on the box picking device and the relative position relationship between the two is known, the accurate position information of the to-be-carried cargo box relative to the box picking device can be calculated, and then the robotic arm can be controlled to pick up the to-be-carried cargo box based on this position information.

[0005] However, since a visual identification code needs to be set at each grid opening, the cost of installing the visual identification code will increase; in addition, since the image information of the visual identification code at each grid opening needs to be collected before accurately positioning all the to-be-carried cargo boxes on the shelf, the efficiency of positioning the cargo box is reduced. Summary of the Invention

[0006] The present invention provides a cargo box positioning method, device, equipment and storage medium for improving the efficiency of positioning the cargo box. Specifically, the embodiments of the present application disclose the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a method for positioning a cargo box, which is applied to a terminal device. The terminal device is respectively connected to a control device and a vision scanning device on a box-taking device, and the control device is connected to a robotic arm on the box-taking device. The method includes:

[0008] After the transfer robot transports the shelf to the workstation, obtain the image information of the identification mark to be recognized on the shelf collected by the vision scanning device; wherein, the identification mark to be recognized includes at least one of a vision identification code set on the shelf, the top corner of the shelf, and the intersection position of the middle compartments of the shelf, and the number of vision identification codes set on the shelf is less than the number of middle compartments of the shelf;

[0009] Based on the image information of the identification mark to be recognized, calculate the first position information of the identification mark to be recognized in a first coordinate system; wherein, the first coordinate system is a coordinate system constructed based on the vision scanning device;

[0010] Obtain the second position information of the identification mark to be recognized in a second coordinate system; wherein, the second coordinate system is a coordinate system constructed based on the shelf;

[0011] Based on the first position information and the second position information, calculate the conversion relationship between the first coordinate system and the second coordinate system;

[0012] Obtain the third position information of at least one cargo box to be transported on the shelf in the second coordinate system. Based on the third position information and the conversion relationship, determine the fourth position information of the cargo box to be transported relative to the box-taking device, and send the fourth position information to the control device to trigger the control device to control the robotic arm to pick up the cargo box to be transported based on the fourth position information.

[0013] In combination with the first aspect, in a possible implementation manner of the first aspect, the number of the above-mentioned identification marks to be recognized is greater than or equal to four, and the positions of the identification marks to be recognized are not on a straight line.

[0014] In combination with the first aspect, in a possible implementation manner of the first aspect, the above-mentioned identification marks to be recognized include the four top corners of the front area of the shelf, or the four top corners of the top area of the shelf.

[0015] In combination with the first aspect, in a possible implementation manner of the first aspect, the above-mentioned identification marks to be recognized include the intersection positions of the compartments in the front area of the shelf, or the intersection positions of the compartments in the top area of the shelf.

[0016] In combination with the first aspect, in a possible implementation manner of the first aspect, the above-mentioned identification marks to be recognized include the top corner of the front area of the shelf and the intersection position of the compartments in the front area of the shelf; or the identification marks to be recognized include the top corner of the top area of the shelf and the intersection position of the compartments in the top area of the shelf.

[0017] In combination with the first aspect, in a possible implementation of the first aspect, the to-be-recognized identifier includes visual identification codes, the number of visual identification codes is greater than four, and each visual identification code is respectively arranged at four vertex angles of the front area of the shelf and at least one preset target position in the top area; or, each visual identification code is respectively arranged at four vertex angles of the top area of the shelf and at least one preset target position in the front area; wherein, the preset target positions include vertex angles and central positions.

[0018] In combination with the first aspect, in a possible implementation of the first aspect, the to-be-recognized identifier includes visual identification codes, the number of visual identification codes is four, and each visual identification code is respectively arranged at four vertex angles of the front area or the top area of the shelf.

[0019] In combination with the first aspect, in a possible implementation of the first aspect, the to-be-recognized identifier includes visual identification codes, the number of visual identification codes is three, and each visual identification code is respectively arranged at any three vertex angles of the front area or the top area of the shelf; or, the number of visual identification codes is two, and each visual identification code is respectively arranged at two vertex angles located diagonally in the front area or the top area of the shelf; or, the number of visual identification codes is one, and the visual identification code is arranged at the central position of the front area or the top area of the shelf.

[0020] In combination with the first aspect, in a possible implementation of the first aspect, calculating the conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information includes:

[0021] Calculating the first position information and the second position information by using a preset algorithm to generate a rotation parameter and a translation parameter;

[0022] Determining the conversion relationship between the first coordinate system and the second coordinate system based on the rotation parameter and the translation parameter.

[0023] In combination with the first aspect, in a possible implementation of the first aspect, determining the fourth position information of the to-be-carried cargo box relative to the box-taking device based on the third position information and the conversion relationship includes:

[0024] Determining the intermediate position information of the to-be-carried cargo box relative to the visual scanning device based on the third position information and the conversion relationship;

[0025] Determining the fourth position information of the to-be-carried cargo box relative to the box-taking device based on the intermediate position information and the preset position relationship between the visual scanning device and the box-taking device.

[0026] In combination with the first aspect, in a possible implementation of the first aspect, obtaining the second position information of the to-be-recognized identifier in the second coordinate system includes:

[0027] Construct a second coordinate system with the preset target point on the shelf as the origin;

[0028] Based on the preset dimension information of the grid openings on the shelf for placing the cargo boxes to be carried, calculate the second position information of the identification mark to be recognized in the second coordinate system.

[0029] In a second aspect, an embodiment of the present application further provides a cargo box positioning device, and the device includes:

[0030] A first acquisition module, configured to acquire the image information of the identification mark to be recognized collected by the vision scanning device after the transfer robot transports the shelf to the workstation; wherein, the identification mark to be recognized includes at least one of the vision identification code set on the shelf, the top corner of the shelf, and the intersection position of the grid openings in the shelf, and the number of vision identification codes set on the shelf is less than the number of grid openings in the shelf;

[0031] A first calculation module, configured to calculate the first position information of the identification mark to be recognized in the first coordinate system based on the image information of the identification mark to be recognized; wherein, the first coordinate system is a coordinate system constructed based on the vision scanning device;

[0032] A second acquisition module, configured to acquire the second position information of the identification mark to be recognized in the second coordinate system; wherein, the second coordinate system is a coordinate system constructed based on the shelf;

[0033] A second calculation module, configured to calculate the conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information;

[0034] A determination module, configured to acquire the third position information of at least one cargo box to be carried on the shelf in the second coordinate system, determine the fourth position information of the cargo box to be carried relative to the box-taking device based on the third position information and the conversion relationship, and send the fourth position information to the control device to trigger the control device to control the robotic arm to pick up the cargo box to be carried based on the fourth position information.

[0035] In a third aspect, an embodiment of the present application provides an electronic device (computer device), including: a processor and a memory, the memory is used to store computer-executable instructions; the processor is used to read the instructions from the memory and execute the instructions to implement the method in the foregoing first aspect and any implementation manner of the first aspect.

[0036] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, and computer instructions are stored in the computer-readable storage medium, and the computer instructions are used to cause the computer to execute the method in the foregoing first aspect and any implementation manner of the first aspect.

[0037] In addition, an embodiment of the present application also provides a computer program product, which includes a computing program stored on a computer-readable storage medium. The computer program includes program instructions that, when executed by a computer, cause the computer to execute the method in any implementation manner of the foregoing first aspect.

[0038] The cargo box positioning method, device, equipment, and storage medium provided by the embodiments of the present application obtain image information of an identification mark to be recognized on a shelf collected by a visual scanning device, calculate first position information of the identification mark to be recognized in a first coordinate system based on the image information of the identification mark to be recognized, then obtain second position information of the identification mark to be recognized in a second coordinate system, calculate a conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information, and finally obtain third position information of at least one cargo box to be carried on the shelf in the second coordinate system, determine fourth position information of the cargo box to be carried relative to the box-taking device based on the third position information and the conversion relationship, and send the fourth position information to a control device to trigger the control device to control a robotic arm to pick up the cargo box to be carried based on the fourth position information. Since the present application can determine the position information of the cargo box to be carried relative to the box-taking device by recognizing at least one of the identification marks including visual identification codes, the top corners of the shelf, and the intersection positions of the middle compartments of the shelf, it is not necessary to install visual identification codes on the shelf, or only a small number of visual identification codes need to be installed on the shelf, and it is not necessary to install a visual identification code at each compartment of the shelf, reducing the manufacturing cost and installation cost of the visual identification codes; in addition, since only the image information of the identification mark to be recognized needs to be collected once, and the position information of all cargo boxes can be determined at one time through the calculated conversion relationship, the efficiency of positioning the cargo boxes is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0040] Figure 1 It is a schematic diagram of a scenario of a cargo box positioning method provided by an embodiment of the present application;

[0041] Figure 2 It is a flowchart of a cargo box positioning method provided by an embodiment of the present application;

[0042] Figure 3A It is a schematic diagram of a front area of a shelf including an identification mark to be recognized provided by an embodiment of the present application;

[0043] Figure 3B Another schematic diagram of the front area of the shelf including the identification mark to be recognized provided by the embodiment of the present application;

[0044] Figure 3C A schematic diagram of setting a visual identification code in the front area of the shelf provided by the embodiment of the present application;

[0045] Figure 4 A schematic diagram of setting a visual identification code in the top area of the shelf provided by the embodiment of the present application;

[0046] Figure 5 A flowchart of calculating the second position information provided by the embodiment of the present application;

[0047] Figure 6 A flowchart of calculating the conversion relationship provided by the embodiment of the present application;

[0048] Figure 7 A flowchart of determining the fourth position information provided by the embodiment of the present application;

[0049] Figure 8 A schematic diagram of the structure of a cargo box positioning device provided by the embodiment of the present application;

[0050] Figure 9 A schematic diagram of the structure of an electronic device provided by the embodiment of the present application. Detailed implementation manners

[0051] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application and make the above-mentioned objects, features, and advantages of the embodiments of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0052] In the prior art, in order to accurately locate the current position of a cargo box, a visual identification code is usually set at each grid opening where the cargo box is placed on the shelf. After roughly positioning the cargo box to be transported by using the pre-set position information, the image information of the visual identification code at the grid opening where the cargo box to be transported is placed is obtained by using the visual acquisition device in the box-taking device, and then the position information of the cargo box to be transported relative to the visual scanning device is calculated based on the image information. Since the visual scanning device is installed on the box-taking device and the relative position relationship between the two is known, the accurate position information of the cargo box to be transported relative to the box-taking device can be calculated, and then the robotic arm can be controlled to pick up the cargo box to be transported based on this position information. However, since a visual identification code needs to be set at each grid opening, the cost of installing the visual identification code will increase; in addition, since the image information of the visual identification code at each grid opening needs to be collected before the accurate positioning of all the cargo boxes to be transported on the shelf can be performed, the efficiency of positioning the cargo box is reduced.

[0053] In view of this, embodiments of the present application propose a cargo box positioning method, device, equipment, and storage medium. By identifying at least one of the to-be-identified identifiers including visual identification codes, the top corners of the shelves, and the intersection positions of the middle compartments of the shelves, the position information of the to-be-carried cargo box relative to the box-taking device can be determined. Therefore, it is not necessary to install visual identification codes on the shelves, or only a small number of visual identification codes need to be installed on the shelves, and it is not necessary to install a visual identification code at each compartment of the shelves, reducing the manufacturing cost and installation cost of the visual identification codes. In addition, since only the image information of the to-be-identified identifier needs to be collected once, and the position information of all cargo boxes can be determined at one time through the calculated conversion relationship, the efficiency of positioning the cargo boxes is improved.

[0054] The cargo box positioning method provided by the embodiments of the present application is applied to a terminal device, such as Figure 1 shown Figure 1 FIG. is a schematic diagram of a scenario of a cargo box positioning method provided by an embodiment of the present application. The terminal device 102 is respectively connected to the control device 104 and the visual scanning device 1061 on the box-taking device 106. The control device 104 is connected to the robotic arm 1062 on the box-taking device 106. Among them, an upper-layer service system is deployed in the terminal device 102 to execute the cargo box positioning method. The visual scanning device 1061 can be a camera for scanning and identifying the visual identification codes on the shelves and collecting the image information of the visual identification codes. The control device 104 can include a Programmable Logic Controller (PLC for short), which is used to receive the position information of the to-be-carried cargo box relative to the box-taking device sent by the terminal device 102, and control the robotic arm 1062 to pick up the to-be-carried cargo box based on this position information. The robotic arm 1062 can be a suction cup type robotic arm or other types of robotic arms, and the embodiments of the present application do not make specific limitations on this.

[0055] The following details the technical solutions provided by the embodiments of the present application in conjunction with the accompanying drawings. Figure 2 FIG. is a flowchart of a cargo box positioning method provided by an embodiment of the present application. The method includes the following steps:

[0056] Step 202: After the transfer robot transports the shelf to the workstation, obtain the image information of the to-be-identified identifier on the shelf collected by the visual scanning device.

[0057] Among them, the to-be-identified identifier includes at least one of the visual identification codes set on the shelf, the top corners of the shelf, and the intersection positions of the middle compartments of the shelf, and the number of visual identification codes set on the shelf is less than the number of compartments of the shelf.

[0058] In some embodiments, there is no need to set visual identification codes on the shelves, and the top corners of the shelves and / or the intersection positions of the middle compartments of the shelves can be used as the identification marks to be recognized. In some embodiments, visual identification codes with a quantity less than the quantity of the middle compartments of the shelves can be set on the shelves, and the visual identification codes are used as the identification marks to be recognized on the shelves. In other embodiments, the visual identification codes and the top corners of the shelves can be used as the identification marks to be recognized on the shelves. Alternatively, the visual identification codes and the intersection positions of the middle compartments of the shelves can also be used as the identification marks to be recognized on the shelves. Or, the visual identification codes, the top corners of the shelves, and the intersection positions of the middle compartments of the shelves can also be jointly used as the identification marks to be recognized on the shelves.

[0059] Exemplarily, the quantity of the identification marks to be recognized can be greater than or equal to four, and the positions of the identification marks to be recognized are not on a straight line. The more the quantity of the identification marks to be recognized, the more accurate the position of the cargo box to be carried is determined. The embodiments of the present application do not limit the specific quantity of the identification marks to be recognized.

[0060] Exemplarily, the identification marks to be recognized include the top corners of the shelves. When the identification marks to be recognized only include the top corners of the shelves, the quantity of the top corners of the shelves is greater than or equal to four. For example, the identification marks to be recognized include the four top corners of the front area of the shelves, or the four top corners of the top area of the shelves. For another example, the identification marks to be recognized can also include the four top corners of the side area of the shelves. When the identification marks to be recognized include, in addition to the top corners of the shelves, the intersection positions of the middle compartments of the shelves and / or the visual identification codes, the quantity of the top corners of the shelves can be less than four. The embodiments of the present application do not limit the positions and quantities of the top corners of the shelves included in the identification marks to be recognized. The positions of the top corners of the shelves included in the identification marks to be recognized are related to the installation position of the visual scanning device.

[0061] For example, Figure 3A is a schematic diagram of the front area of a shelf provided by the embodiments of the present application including the identification marks to be recognized. As Figure 3A shown, Figure 3A the gray dots in represent the four top corners of the front area of the shelf, and the four top corners of the front area of the shelf are the identification marks to be recognized on the shelf.

[0062] Exemplarily, the identification mark to be recognized includes the intersection position of the compartments in the shelf. When the identification mark to be recognized only includes the intersection position of the compartments in the shelf, the number of the intersection positions of the shelf compartments is greater than or equal to four. For example, the identification mark to be recognized includes the intersection positions of at least four compartments located in the front area of the shelf, or the intersection positions of at least four compartments located in the top area of the shelf. For another example, the identification mark to be recognized may further include the intersection positions of at least four compartments located in the side area of the shelf. When the identification mark to be recognized includes, in addition to the intersection position of the compartment, the top corner of the shelf and / or the visual identification code, the number of the intersection positions of the shelf compartments may be less than four. The embodiments of the present application do not limit the position and number of the intersection positions of the compartments included in the identification mark to be recognized, and the intersection positions of the compartments included in the identification mark to be recognized are related to the installation position of the visual scanning device.

[0063] For example, Figure 3B FIG. is a schematic diagram of another front area of the shelf provided by the embodiment of the present application, including the identification mark to be recognized, as Figure 3B shown, the shelf includes compartments A to L, and the intersection point of each compartment is an identification mark to be recognized, Figure 3B and the gray dots in are all identification marks to be recognized.

[0064] Exemplarily, the identification mark to be recognized may include both the top corner of the shelf and the intersection position of the compartments in the shelf. For example, the identification mark to be recognized includes the top corner of the front area of the shelf and the intersection position of the compartments in the front area of the shelf. For another example, the identification mark to be recognized includes the top corner of the top area of the shelf and the intersection position of the compartments in the top area of the shelf.

[0065] Exemplarily, the identification mark to be recognized includes a visual identification code. The visual identification code is an identification code for positioning, and usually the Apriltag code can be used. The Apriltag code, as a visual positioning identifier, is similar to a two-dimensional code or a bar code. Of course, the ArUco code can also be used, and the embodiments of the present application do not make specific limitations thereto. The principle of positioning using the Apriltag code and the ArUco code is the prior art and will not be elaborated herein.

[0066] In addition, the number of the visual identification codes installed on the shelf in the embodiments of the present application is less than the number of the compartments in the shelf, and it is not necessary to install a visual identification code at each compartment of the shelf. As Figure 3C shown, Figure 3CSchematic diagram of setting a visual identification code in the front area of a shelf provided by an embodiment of the present application. Herein, the shelf includes compartments A to L, and the gray dots below each compartment represent the visual identification code corresponding to the compartment, that is, in the prior art, a visual identification code is provided at each compartment. In the embodiment of the present application, the number of visual identification codes set on the shelf is less than the number of compartments in the shelf, and the position of the visual identification codes set on the shelf in the embodiment of the present application is not limited.

[0067] In some embodiments, the number of visual identification codes may be greater than four and less than the number of compartments in the shelf, and each visual identification code is respectively set at four top corners of the front area of the shelf and at least one preset target position in the top area, or each visual identification code is respectively set at four top corners of the top area of the shelf and at least one preset target position in the front area.

[0068] Among them, the preset target positions include top corner positions and center positions. Taking the example where each visual identification code is respectively set at four top corners of the front area of the shelf and at four top corners of the top area for illustration.

[0069] Since the position information of the shelf can be determined by the visual identification codes at the four top corners of the front area, and the position information of the shelf can also be determined by the visual identification codes at the four top corners of the top area. Then, the two determined position information can be compared. For example, a difference comparison can be made. If the difference is less than a preset threshold, it indicates that the determined position information of the shelf is accurate; if the difference is greater than the preset threshold, it indicates that the determined position information of the shelf is inaccurate. At this time, the installation position of the visual identification code and the like can be rechecked for deviation, so that in addition to reducing the manufacturing cost and installation cost of the visual identification code, the accuracy of locating the cargo box can also be improved.

[0070] In some embodiments, the number of visual identification codes may be four, and each visual identification code may be respectively set at four top corners of the front area or the top area of the shelf. Please continue to refer to Figure 3C , it can be seen that the dots at the four top corners of the shelf respectively represent a visual identification code, and the four visual identification codes can be respectively labeled as TAG1, TAG2, TAG3, and TAG4.

[0071] In addition, Figure 4A schematic diagram of setting visual identification codes in the top area of a shelf provided by an embodiment of the present application. Among them, the four visual identification codes are represented by small rectangular frames located at the four top corners. By using four visual identification codes, the positioning of all cargo boxes can be achieved, greatly reducing the manufacturing cost and installation cost of the visual identification codes, and also reducing the later maintenance cost of the visual identification codes. In the embodiment of the present application, the position and quantity of the visual identification codes set on the shelf are not limited. The setting position of the visual identification codes is related to the installation position of the visual scanning device, and the quantity of the visual identification codes set is less than the quantity of the grid openings in the shelf.

[0072] In some other embodiments, the number of visual identification codes can be three, and each visual identification code is respectively set at any three top corners of the front area or the top area of the shelf. Since the entire surface where the shelf is located can still be determined by three visual identification codes, the position information of the shelf can be determined.

[0073] In some other embodiments, the number of visual identification codes can be two, and each visual identification code is respectively set at two top corners located diagonally in the front area or the top area of the shelf. The number of visual identification codes can also be one, and this visual identification code is set at the central position of the front area or the top area of the shelf. Combining two visual identification codes or one visual identification code, and the shelf size information, the position information of the shelf can be determined. In addition, the manufacturing cost and installation cost of the visual identification codes are further reduced.

[0074] It should be noted that when the identification mark to be recognized is a visual identification code, the specific implementation manner of obtaining the position information of the cargo box to be carried relative to the box-taking device is similar to that when the identification mark to be recognized is the intersection position of the top corner of the shelf and / or the grid opening in the shelf. In the following embodiments of the present application, the specific implementation manner of obtaining the position information of the cargo box to be carried relative to the box-taking device is exemplarily described by taking the identification mark to be recognized including a visual identification code as an example.

[0075] Step 204: Calculate the first position information of the identification mark to be recognized in the first coordinate system based on the image information of the identification mark to be recognized.

[0076] Among them, the first coordinate system is a coordinate system constructed based on the visual scanning device. Taking the identification mark to be recognized including a visual identification code as an example, please continue to refer to Figure 3C , the position information of the four visual identification codes in the first coordinate system can be respectively expressed as: PTAG1 Camera = (X1, Y1, Z1), PTAG2 Camera = (X2, Y2, Z2), PTAG3 Camera = (X3, Y3, Z3), PTAG4 Camera=(X4, Y4, Z4).

[0077] Step 206: Obtain the second position information of the identifier to be recognized in the second coordinate system.

[0078] Among them, the second coordinate system is a coordinate system constructed based on the shelf. Please continue to refer to Figure 3C , Figure 3C The coordinate system XYZ in is the second coordinate system. When calculating the second position information, as Figure 5 shown, Figure 5 is a flowchart of a method for calculating the second position information provided by an embodiment of the present application. The method includes the following steps:

[0079] Step 502: Construct a second coordinate system with a preset target point on the shelf as the origin.

[0080] Step 504: Calculate the second position information of the identifier to be recognized in the second coordinate system based on the preset dimension information of the grid opening on the shelf for placing the box to be carried.

[0081] Among them, the preset target point can be any pre-determined position in the shelf. As Figure 3C shown, the preset target point can be the top corner in the upper left corner among the four top corners of the shelf, so that this top corner can be used as the origin of the second coordinate system, and the X-axis is established in the horizontal direction, the Y-axis is established in the vertical direction, and the Z-axis is perpendicular to both the X-axis and the Y-axis respectively, thereby constructing the second coordinate system.

[0082] Among them, in Figure 3C , the preset dimension information of each grid opening can use N to represent the dimension of the grid opening in the horizontal direction, M to represent the dimension of the grid opening in the vertical direction, and the dimension in the Z-axis direction can be uniformly set to 0. Taking the visual identification code set at the four top corners of the front area of the shelf as an example of the identifier to be recognized, the second position information of the four visual identification codes in the second coordinate system can be respectively expressed as: PTAG1 Shelf =(0, 0, 0), PTAG2 Shelf =(3N, 0, 0), PTAG3 Shelf =(0, 4M, 0), PTAG4 Shelf =(3N, 4M, 0).

[0083] In some embodiments, taking the case where the identification mark to be recognized includes the intersection position of the grid openings in the shelf, and the grid opening intersection position is the grid opening intersection position in the front area of the shelf as an example, in order to determine the second position information of each identification mark to be recognized in the second coordinate system, the image information of the identification marks to be recognized collected by the above-mentioned visual scanning device may include all the grid openings in the front area of the shelf. For example, the visual scanning device may be a fish-eye camera. After the transfer robot transports the shelf to the workstation, the fish-eye camera collects the image information including all the grid openings in the front area of the shelf, and the second position information of each grid opening intersection position in the identification mark to be recognized in the second coordinate system can be determined through this image information. For another example, after the transfer robot transports the shelf to the workstation, the visual scanning device collects the image information of some grid openings on the shelf (for example, the upper half, lower half, left half, or right half of the grid openings in the front area of the shelf; for another example, the upper half, lower half, left half, or right half of the grid openings in the top area of the shelf), and the second position information of each grid opening intersection position in the identification mark to be recognized in the second coordinate system can be determined through this image information. For yet another example, the visual scanning device can also collect multiple images including the identification marks to be recognized during the process of the transfer robot transporting the shelf to the workstation, and the second position information of each grid opening intersection position in the identification mark to be recognized in the second coordinate system can be determined through these multiple images.

[0084] Step 208: Calculate the conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information.

[0085] Among them, as Figure 6 shown, Figure 6 FIG. is a flowchart of a method for calculating a conversion relationship provided by an embodiment of the present application. The method includes the following steps:

[0086] Step 602: Use a preset algorithm to calculate the first position information and the second position information to generate a rotation parameter and a translation parameter.

[0087] Step 604: Determine the conversion relationship between the first coordinate system and the second coordinate system based on the rotation parameter and the translation parameter.

[0088] Exemplarily, the preset algorithm may be the PnP (Perspective-n-Point) algorithm. This algorithm is a method for solving the correspondence between 3D and 2D points, which describes how to estimate the pose of a camera when the positions of n 3D space points are known. By substituting four sets of first position information and four sets of second position information into the following formula (1) and using the PnP algorithm, the rotation parameter R and the translation parameter t can be calculated. The specific solution process is prior art and will not be elaborated here. Thus, the conversion relationship between the first coordinate system and the second coordinate system can be determined based on the rotation parameter R and the translation parameter t.

[0089] PTAG Camera = PTAG Shelf R+t (1)

[0090] Among them, PTAG Camera represents the first position information of the identifier to be recognized; PTAG Shelf represents the second position information of the identifier to be recognized; R represents the rotation parameter; t represents the translation parameter.

[0091] Step 210, obtain the third position information of at least one box to be carried on the shelf in the second coordinate system, determine the fourth position information of the box to be carried relative to the box picking device based on the third position information and the conversion relationship, and send the fourth position information to the control device to trigger the control device to control the robotic arm to pick up the box to be carried based on the fourth position information.

[0092] Among them, for the method of determining the third position information of the box to be carried in the second coordinate system, please refer to the above process of determining the second position information. Taking Figure 3C the box to be carried placed at the middle grid E as an example, the third position information of this box to be carried can be expressed as Based on this, when determining the fourth position information, as Figure 7 shown, Figure 7 is a flowchart of a method for determining the fourth position information provided by an embodiment of the present application. This method includes the following steps:

[0093] Step 702, determine the intermediate position information of the box to be carried relative to the visual scanning device based on the third position information and the conversion relationship.

[0094] Step 704, determine the fourth position information of the box to be carried relative to the box picking device based on the intermediate position information and the preset position relationship between the visual scanning device and the box picking device.

[0095] Among them, continuing to take Figure 3C the box to be carried placed at the middle grid E as an example, based on the third position information and the conversion relationship, through the following formula (2), the position information of this box to be carried relative to the visual scanning device can be calculated, which is the intermediate position information.

[0096] PE Camera = PE Shelf R+t (2)

[0097] Among them, PE Camera represents the position information (such as the intermediate position information) of the box to be carried placed at grid E in the first coordinate system, and PE Shelf represents the position information (such as the third position information) of the box to be carried placed at grid E in the second coordinate system.

[0098] Since the vision scanning device is manually installed on the box picking device, the preset positional relationship between the vision scanning device and the box picking device is also known. Thus, based on the preset positional relationship and the intermediate position information calculated by formula (2), the intermediate position information can be converted into the fourth position information of the box to be carried relative to the box picking device.

[0099] After finally calculating the fourth position information of the box to be carried relative to the box picking device, the terminal device can send the fourth position information to the control device, triggering the control device to control the robotic arm to pick up the box to be carried based on the fourth position information.

[0100] In addition, it should be noted that when multiple boxes to be carried need to be picked up at one time, the fourth position information corresponding to each box to be carried can be calculated through the above formula (2). Thus, all the fourth position information is sent to the control device, so that multiple boxes to be carried can be picked up at one time. Similarly, when returning the boxes, this method can also be used to return multiple boxes at one time.

[0101] The following introduces the device embodiment corresponding to the foregoing method embodiment.

[0102] The embodiment of the present application further provides a box positioning device 800 for executing the box positioning method in the foregoing embodiment.

[0103] Specifically, as Figure 8 shown, the device includes: a first acquisition module 801, a first calculation module 802, a second acquisition module 803, a second calculation module 804, and a determination module 805. In addition, the device may further include other more or fewer units / modules, such as a storage unit, a sending unit, etc.

[0104] The first acquisition module 801 is configured to acquire the image information of the identification mark to be recognized on the shelf collected by the vision scanning device after the transfer robot transports the shelf to the workstation; the identification mark to be recognized includes at least one of the vision identification code set on the shelf, the top corner of the shelf, and the intersection position of the middle compartments of the shelf, and the number of vision identification codes set on the shelf is less than the number of middle compartments of the shelf.

[0105] The first calculation module 802 is configured to calculate the first position information of the identification mark to be recognized in the first coordinate system based on the image information of the identification mark to be recognized; wherein, the first coordinate system is a coordinate system constructed based on the vision scanning device.

[0106] The second acquisition module 803 is configured to acquire the second position information of the identification mark to be recognized in the second coordinate system; wherein, the second coordinate system is a coordinate system constructed based on the shelf.

[0107] The second calculation module 804 is configured to calculate the conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information.

[0108] The determination module 805 is configured to obtain the third position information of at least one container to be carried on the shelf in the second coordinate system, determine the fourth position information of the container to be carried relative to the container picking device based on the third position information and the conversion relationship, and send the fourth position information to the control device, triggering the control device to control the robotic arm to pick up the container to be carried based on the fourth position information.

[0109] In some embodiments, the identification mark to be recognized includes the four top corners of the front area of the shelf, or the four top corners of the top area of the shelf.

[0110] In some embodiments, the identification mark to be recognized includes the intersection positions of the compartments in the front area of the shelf, or the intersection positions of the compartments in the top area of the shelf.

[0111] In some embodiments, the identification mark to be recognized includes the top corners of the front area of the shelf and the intersection positions of the compartments in the front area of the shelf; or the identification mark to be recognized includes the top corners of the top area of the shelf and the intersection positions of the compartments in the top area of the shelf.

[0112] In some embodiments, the number of identification marks to be recognized is greater than or equal to four, and the positions of the respective identification marks to be recognized are not on a straight line.

[0113] In some embodiments, the number of visual identification codes is greater than four, and each visual identification code is respectively disposed at the four top corners of the front area of the shelf and at least one preset target position in the top area, or each visual identification code is respectively disposed at the four top corners of the top area of the shelf and at least one preset target position in the front area; wherein, the preset target positions include the top corner positions and the center positions.

[0114] In some embodiments, the number of visual identification codes is four, and each visual identification code is respectively disposed at the four top corners of the front area or the top area of the shelf.

[0115] In some embodiments, the number of visual identification codes is three, and each visual identification code is respectively disposed at any three top corners of the front area or the top area of the shelf; or the number of visual identification codes is two, and each visual identification code is respectively disposed at the two top corners located at the diagonal of the front area or the top area of the shelf; or the number of visual identification codes is one, and this visual identification code is disposed at the center position of the front area or the top area of the shelf.

[0116] Exemplarily, the second calculation module 804 is specifically configured to calculate the first position information and the second position information by using a preset algorithm to generate a rotation parameter and a translation parameter; and determine a conversion relationship between the first coordinate system and the second coordinate system based on the rotation parameter and the translation parameter.

[0117] Exemplarily, the determination module 805 is specifically configured to determine intermediate position information of the cargo box to be carried relative to the visual scanning device based on the third position information and the conversion relationship; and determine fourth position information of the cargo box to be carried relative to the box-taking device based on the intermediate position information and a preset position relationship between the visual scanning device and the box-taking device.

[0118] Exemplarily, the second acquisition module 803 is specifically configured to construct a second coordinate system with a preset target point on the shelf as the origin; and calculate second position information of the visual identification code in the second coordinate system based on preset dimension information of the grid opening on the shelf for placing the cargo box to be carried.

[0119] In a specific implementation, an embodiment of the present application further provides an electronic device, which may be the server in the foregoing embodiments and is used to implement all or part of the method steps for positioning the cargo box.

[0120] As Figure 9 shown, a schematic structural diagram of an electronic device provided by an embodiment of the present application is shown. It includes: at least one processor, a memory, and at least one interface. In addition, a communication bus may also be included for connecting these components.

[0121] Among them, the at least one processor may be a CPU or a processing chip, and is configured to read and execute computer program instructions stored in the memory, so that the at least one processor can execute the method processes in the foregoing various embodiments.

[0122] The memory may be a non-transitory memory, which may include a volatile memory, such as a high-speed random access memory (Random Access Memory, RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0123] The at least one interface includes an input / output interface and a communication interface. The communication interface may be a wired or wireless interface, so as to implement a communication connection between the electronic device and other devices. The input / output interface may be used to connect external devices, such as a display screen, a keyboard, etc.

[0124] In an embodiment of the present application, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0125] After the transfer robot transports the shelf to the workstation, obtain the image information of the identification marks to be recognized on the shelf collected by the vision scanning device; the identification marks to be recognized include at least one of the vision identification codes set on the shelf, the top corners of the shelf, and the intersection positions of the middle compartments of the shelf, and the number of vision identification codes set on the shelf is less than the number of middle compartments of the shelf.

[0126] Based on the image information of the identification marks to be recognized, calculate the first position information of the identification marks to be recognized in the first coordinate system; wherein, the first coordinate system is a coordinate system constructed based on the vision scanning device.

[0127] Obtain the second position information of the identification marks to be recognized in the second coordinate system; wherein, the second coordinate system is a coordinate system constructed based on the shelf.

[0128] Based on the first position information and the second position information, calculate the conversion relationship between the first coordinate system and the second coordinate system.

[0129] Obtain the third position information of at least one cargo box to be transported on the shelf in the second coordinate system, determine the fourth position information of the cargo box to be transported relative to the box-taking device based on the third position information and the conversion relationship, and send the fourth position information to the control device to trigger the control device to control the robotic arm to pick up the cargo box to be transported based on the fourth position information.

[0130] In some embodiments, the identification marks to be recognized include the four top corners of the front area of the shelf, or the four top corners of the top area of the shelf.

[0131] In some embodiments, the identification marks to be recognized include the intersection positions of the compartments in the front area of the shelf, or the intersection positions of the compartments in the top area of the shelf.

[0132] In some embodiments, the identification marks to be recognized include the top corners of the front area of the shelf and the intersection positions of the compartments in the front area of the shelf; or the identification marks to be recognized include the top corners of the top area of the shelf and the intersection positions of the compartments in the top area of the shelf.

[0133] In some embodiments, the number of identification marks to be recognized is greater than or equal to four, and the positions of the identification marks to be recognized are not on a straight line.

[0134] In the embodiments of the present application, the number of vision identification codes can be greater than four, and each vision identification code is respectively set at the four top corners of the front area of the shelf and at least one preset target position in the top area, or each vision identification code is respectively set at the four top corners of the top area of the shelf and at least one preset target position in the front area; wherein, the preset target positions include the top corner positions and the center positions.

[0135] In the embodiment of the present application, the number of visual identification codes can be four, and each visual identification code is respectively arranged at the four top corners of the front area or the top area of the shelf.

[0136] In the embodiment of the present application, the number of visual identification codes can be three, and each visual identification code is respectively arranged at any three top corners of the front area or the top area of the shelf; or, the number of visual identification codes can be two, and each visual identification code is respectively arranged at the two top corners located diagonally in the front area or the top area of the shelf; or, the number of visual identification codes is one, and this visual identification code is arranged at the central position of the front area or the top area of the shelf.

[0137] In the embodiment of the present application, when the processor executes the computer program, the following steps are further implemented: calculating the first position information and the second position information by using a preset algorithm to generate a rotation parameter and a translation parameter; and determining the conversion relationship between the first coordinate system and the second coordinate system based on the rotation parameter and the translation parameter.

[0138] In the embodiment of the present application, when the processor executes the computer program, the following steps are further implemented: determining the intermediate position information of the cargo box to be carried relative to the visual scanning device based on the third position information and the conversion relationship; and determining the fourth position information of the cargo box to be carried relative to the box-taking device based on the intermediate position information and the preset positional relationship between the visual scanning device and the box-taking device.

[0139] In the embodiment of the present application, when the processor executes the computer program, the following steps are further implemented: constructing a second coordinate system with a preset target point on the shelf as the origin; and calculating the second position information of the identification to be recognized in the second coordinate system based on the preset dimension information of the grid opening on the shelf for placing the cargo box to be carried.

[0140] The computer device provided in the embodiment of the present application has the same implementation principle and technical effects as those in the above method embodiment, and will not be elaborated here.

[0141] In some embodiments, the memory stores computer-readable program instructions, and when the processor reads and executes the program instructions in the memory, a cargo box positioning method in the foregoing embodiment can be implemented.

[0142] In addition, the embodiment of the present application further provides a computer program product for storing computer-readable program instructions, and when the instructions are executed by the processor, a cargo box positioning method in the foregoing embodiment can be implemented.

[0143] It should be noted that in this application, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprise," "include," or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0144] Each embodiment in this specification is described in a related manner. For the parts that are the same or similar among the embodiments, reference can be made to each other, and the differences between each embodiment and other embodiments are emphasized. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and reference can be made to the relevant parts of the method embodiments for the relevant content.

[0145] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in combination with these instruction execution systems, apparatus, or devices.

[0146] For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0147] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection part having one or more wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM).

[0148] In addition, a computer-readable medium can even be paper or other suitable media on which a program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation or other suitable processing as necessary, and then stored in a computer memory. It should be understood that various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof.

[0149] In the above embodiments, the plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having suitable combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0150] The above embodiments of the present invention do not constitute a limitation on the protection scope of the present invention.

Claims

1. A cargo box positioning method, characterized in that, Applied to a terminal device, the terminal device is respectively connected to a control device and a vision scanning device on a box-taking device, and the control device is connected to a robotic arm on the box-taking device, including: After a transfer robot transports a shelf to a workstation, obtaining image information of an identification mark to be recognized on the shelf collected by the vision scanning device; wherein, the identification mark to be recognized includes at least one of a vision identification code set on the shelf, the apex angle of the shelf, and the intersection position of the middle compartments of the shelf, and the number of vision identification codes set on the shelf is less than the number of middle compartments of the shelf; Based on the image information of the identification mark to be recognized, calculating first position information of the identification mark to be recognized in a first coordinate system; wherein, the first coordinate system is a coordinate system constructed based on the vision scanning device; Obtaining second position information of the identification mark to be recognized in a second coordinate system; wherein, the second coordinate system is a coordinate system constructed based on the shelf; Based on the first position information and the second position information, calculating a conversion relationship between the first coordinate system and the second coordinate system; Obtaining third position information of at least one box to be transported on the shelf in the second coordinate system, determining fourth position information of the box to be transported relative to the box-taking device based on the third position information and the conversion relationship, and sending the fourth position information to the control device to trigger the control device to control the robotic arm to pick up the box to be transported based on the fourth position information.

2. The cargo box positioning method according to claim 1, wherein The number of the identification marks to be recognized is greater than or equal to four, and the positions of the identification marks to be recognized are not on a straight line.

3. The cargo box positioning method according to claim 1, characterized in that, The identification mark to be recognized includes the four apex angles of the front area of the shelf, or the four apex angles of the top area of the shelf.

4. The cargo box positioning method according to claim 1, characterized in that The identification mark to be recognized includes the intersection positions of the compartments in the front area of the shelf, or the intersection positions of the compartments in the top area of the shelf.

5. The cargo box positioning method according to claim 1, characterized in that, The identification mark to be recognized includes the apex angle of the front area of the shelf and the intersection position of the compartments in the front area of the shelf; or the identification mark to be recognized includes the apex angle of the top area of the shelf and the intersection position of the compartments in the top area of the shelf.

6. The cargo box positioning method according to claim 1, wherein, The identification mark to be recognized includes a vision identification code set on the shelf, the number of the vision identification codes is greater than four, and each vision identification code is respectively set at the four apex angles of the front area of the shelf and at least one preset target position in the top area; or each vision identification code is respectively set at the four apex angles of the top area of the shelf and at least one preset target position in the front area; wherein, the preset target position includes an apex angle position and a center position.

7. The cargo box positioning method according to claim 1, characterized in that, The identification mark to be recognized includes a vision identification code set on the shelf, the number of the vision identification codes is four, and each vision identification code is respectively set at the four apex angles of the front area or the top area of the shelf.

8. The cargo box positioning method according to claim 1, characterized in that, The identification mark to be recognized includes a vision identification code set on the shelf, the number of the vision identification codes is three, and each vision identification code is respectively set at any three apex angles of the front area or the top area of the shelf; or, The number of the visual identification codes is two, and each of the visual identification codes is respectively arranged at two vertex angles located diagonally in the front area or the top area of the shelf; or, The number of the visual identification codes is one, and the visual identification code is arranged at the central position of the front area or the top area of the shelf.

9. The cargo box positioning method according to any one of claims 1-8, characterized in that, Calculating the conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information includes: Calculating the first position information and the second position information by using a preset algorithm to generate a rotation parameter and a translation parameter; Determining the conversion relationship between the first coordinate system and the second coordinate system based on the rotation parameter and the translation parameter.

10. The cargo box positioning method according to any one of claims 1-8, characterized in that, Determining the fourth position information of the to-be-carried cargo box relative to the box-taking device based on the third position information and the conversion relationship includes: Determining the intermediate position information of the to-be-carried cargo box relative to the visual scanning device based on the third position information and the conversion relationship; Determining the fourth position information of each to-be-carried cargo box relative to the box-taking device based on the intermediate position information and the preset positional relationship between the visual scanning device and the box-taking device.

11. The cargo box positioning method according to any one of claims 1-8, characterized in that Obtaining the second position information of the to-be-identified identifier in the second coordinate system includes: Constructing the second coordinate system with a preset target point on the shelf as the origin; Calculating the second position information of the to-be-identified identifier in the second coordinate system based on the preset dimension information of the grid opening on the shelf for placing the to-be-carried cargo box.

12. A cargo box positioning device, characterized in that, Including: A first obtaining module, configured to obtain the image information of the to-be-identified identifier on the shelf collected by a visual scanning device after a transfer robot transports the shelf to a workstation; wherein, the to-be-identified identifier includes at least one of a visual identification code set on the shelf, a vertex angle of the shelf, and an intersection position of grid openings in the shelf, and the number of visual identification codes set on the shelf is less than the number of grid openings in the shelf; A first calculation module, configured to calculate the first position information of the to-be-identified identifier in a first coordinate system based on the image information of the to-be-identified identifier; wherein, the first coordinate system is a coordinate system constructed based on the visual scanning device; A second obtaining module, configured to obtain the second position information of the to-be-identified identifier in a second coordinate system; wherein, the second coordinate system is a coordinate system constructed based on the shelf; A second calculation module, configured to calculate the conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information; A determination module, configured to obtain the third position information of at least one to-be-carried cargo box on the shelf in the second coordinate system, determine the fourth position information of the to-be-carried cargo box relative to a box-taking device based on the third position information and the conversion relationship, and send the fourth position information to a control device to trigger the control device to control a robotic arm to pick up the to-be-carried cargo box based on the fourth position information.

13. An electronic device, comprising: A processor and a memory, characterized in that The memory is used for storing computer-executable instructions; The processor is configured to read the instructions from the memory and execute the instructions to implement the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The storage medium stores computer program instructions. When a computer reads the instructions, the method according to any one of claims 1 to 11 is executed.

Images