Shelf positioning method, device, shelf handling equipment and storage medium

Abstract

The present invention provides a shelf positioning method, device, shelf handling equipment and storage medium. The shelf positioning method includes: obtaining a target image collected by an image sensor for a target shelf and calibration parameters of the image sensor, wherein a marking strip composed of visual markings is arranged on the target shelf, extracting a marking image and preset marking points of the visual markings in the marking image from the target image collected by the image sensor, calculating the position information of the marking strip relative to the image sensor based on the calibration parameters, the marking image and the preset marking points, and determining the positioning result of the target shelf according to the position information. Using visual markings to locate the target shelf avoids signal interference in actual scenes, thereby improving the accuracy of the shelf positioning result.

Description

Technical Field

[0001] The present invention relates to the field of warehousing technology, and in particular to a shelf positioning method, a shelf positioning device, a shelf handling equipment and a storage medium. Background Art

[0002] In recent years, with the rapid development of e-commerce, the number of user orders has increased exponentially. A warehouse needs to store a huge amount of goods. How to improve warehousing efficiency has become the focus of warehouse management.

[0003] When moving goods in a warehouse, in order to improve the efficiency of moving, the goods to be moved are usually placed on a shelf, and then the shelf handling equipment moves the shelf, so that multiple goods can be moved at the same time, improving the efficiency of goods moving. It can be seen that accurate positioning of the shelf is the key to the shelf handling equipment in the process of moving the shelf.

[0004] Current shelf positioning solutions mainly use radio frequency technology or wireless local area network (WLAN) technology for positioning. However, radio frequency technology and WLAN technology are greatly affected by the scene. Signal interference in actual scenes will lead to poor accuracy of shelf positioning results. Summary of the invention

[0005] In view of this, the embodiments of the present invention provide a shelf positioning method, device, shelf handling equipment and storage medium to solve the technical defects existing in the prior art.

[0006] According to a first aspect of an embodiment of the present invention, a shelf positioning method is provided, comprising:

[0007] Acquire a target image captured by an image sensor for a target shelf and calibration parameters of the image sensor, wherein an identification belt is arranged on the target shelf, and the identification belt is composed of visual identifications;

[0008] Extracting a logo image and preset logo points of a visual logo in the logo image from a target image;

[0009] Based on the calibration parameters, the identification image and the preset identification points, the position information of the identification tape relative to the image sensor is calculated;

[0010] According to the position information, the positioning result of the target shelf is determined.

[0011] Optionally, the step of extracting the identification image and the preset identification points of the visual identification in the identification image from the target image includes:

[0012] Binarization is performed on the target image to obtain a binary image;

[0013] Identify the covered area of ​​the visual logo in the binarized image;

[0014] A logo image composed of a coverage area is determined, and preset logo points of a visual logo are extracted from the coverage area.

[0015] Optionally, the step of identifying the coverage area of ​​the visual identifier in the binarized image comprises:

[0016] Perform boundary detection on the binary image to determine the boundary information of the binary image;

[0017] Based on the boundary information, the coverage area of ​​the visual logo in the binary image is identified.

[0018] Optionally, the identification band is composed of a plurality of visual identifications; the calibration parameters include an internal reference;

[0019] The step of calculating the position information of the marking tape relative to the image sensor based on the calibration parameters, the marking image and the preset marking points includes:

[0020] Using the internal reference, normalizing the preset identification points of each visual identification in the identification image to obtain the first coordinates of the preset identification points of each visual identification on the identification image;

[0021] Obtaining the second coordinates of the preset identification points of each visual identification in the world coordinate system;

[0022] Calculate the third coordinate of the center point of each visual mark on the mark image according to the first coordinate;

[0023] Calculate the fourth coordinate of the center point of each visual identifier in the world coordinate system according to the second coordinate;

[0024] According to the third coordinate and the fourth coordinate, the position information of the marking tape relative to the image sensor is calculated with the constraint that the center point of each visual marking satisfies the preset arrangement condition.

[0025] Optionally, the calibration parameters include internal references;

[0026] The step of calculating the position information of the marking tape relative to the image sensor based on the calibration parameters, the marking image and the preset marking points includes:

[0027] Using the internal reference, the preset marking point is normalized to obtain the first coordinate of the preset marking point on the marking image;

[0028] Obtain the second coordinate of the preset identification point in the world coordinate system;

[0029] Calculate the position information of the marking tape relative to the image sensor according to the first coordinate and the second coordinate;

[0030] Obtain the fifth coordinate of any point on the identification tape in the world coordinate system and the sixth coordinate of the any point on the identification image;

[0031] Calculate the seventh coordinate of any point according to the fifth coordinate and the position and posture information;

[0032] The sixth coordinate is subtracted from the seventh coordinate to obtain the reprojection error. The pose information is adjusted with the reprojection error being less than a preset threshold as a constraint.

[0033] Optionally, the calibration parameters further include distortion parameters; and the method further includes:

[0034] The logo image is dedistorted by using the distortion parameter to obtain a processed logo image.

[0035] Optionally, the calibration parameters also include external parameters; a plurality of identification tapes are arranged on the target shelf;

[0036] After the step of determining the positioning result of the target shelf according to the position information, the method further includes:

[0037] Calculate the rotation angle of each marking tape around the rotation axis in the world coordinate system according to the position information of each marking tape relative to the image sensor and the preset rotation matrix;

[0038] Calculate the pose information of each marking band in the world coordinate system according to the pose information of each marking band relative to the image sensor and the external parameters;

[0039] According to the rotation angle and posture information of any two marking belts in the world coordinate system, the curve shape between any two marking belts is fitted;

[0040] According to the curve shape between any two identification bands, the current shape of the target shelf is obtained.

[0041] Optionally, the identification tape is in a rectangular shape and is disposed on a crossbeam of the target shelf; the target shelf includes a plurality of crossbeams, and an identification tape is disposed on each crossbeam.

[0042] According to a second aspect of an embodiment of the present invention, there is provided a shelf positioning device, comprising:

[0043] An acquisition module is configured to acquire a target image captured by an image sensor for a target shelf and calibration parameters of the image sensor, wherein an identification band is arranged on the target shelf, and the identification band is composed of visual identification;

[0044] An extraction module is configured to extract a logo image and a preset logo point of a visual logo in the logo image from a target image;

[0045] A posture calculation module is configured to calculate the posture information of the marking tape relative to the image sensor based on the calibration parameters, the marking image and the preset marking points;

[0046] The positioning module is configured to determine the positioning result of the target shelf according to the posture information.

[0047] Optionally, the extraction module includes: a binarization processing unit, a recognition unit and an extraction unit;

[0048] A binarization processing unit is configured to perform binarization processing on the target image to obtain a binarized image;

[0049] a recognition unit configured to recognize a coverage area of ​​a visual identifier in the binarized image;

[0050] The extraction unit is configured to determine the identification image composed of the coverage area and extract the preset identification points of the visual identification from the coverage area.

[0051] Optionally, the recognition unit is further configured to perform boundary detection on the binary image to determine boundary information of the binary image; and based on the boundary information, recognize a coverage area of ​​the visual identifier in the binary image.

[0052] Optionally, the identification band is composed of a plurality of visual identifications; the calibration parameters include an internal reference;

[0053] The posture calculation module is further configured to use internal parameters to normalize the preset identification points of each visual identification in the identification image to obtain the first coordinates of the preset identification points of each visual identification on the identification image; obtain the second coordinates of the preset identification points of each visual identification in the world coordinate system; calculate the third coordinates of the center point of each visual identification on the identification image based on the first coordinates; calculate the fourth coordinates of the center point of each visual identification in the world coordinate system based on the second coordinates; and calculate the posture information of the identification belt relative to the image sensor based on the third coordinates and the fourth coordinates, with the center point of each visual identification satisfying the preset arrangement condition as a constraint condition.

[0054] Optionally, the calibration parameters include internal references;

[0055] The posture calculation module is further configured to use internal parameters to normalize the preset marking points to obtain the first coordinates of the preset marking points on the marking image; obtain the second coordinates of the preset marking points in the world coordinate system; calculate the posture information of the marking tape relative to the image sensor based on the first coordinate and the second coordinates; obtain the fifth coordinate of any point on the marking tape in the world coordinate system and the sixth coordinate of the any point on the marking image; calculate the seventh coordinate of the any point based on the fifth coordinate and the posture information; subtract the sixth coordinate from the seventh coordinate to obtain the reprojection error, and adjust the posture information with the reprojection error being less than a preset threshold as a constraint condition.

[0056] Optionally, the calibration parameters also include distortion parameters; the device also includes: a dedistortion module;

[0057] The dedistortion module is configured to perform dedistortion processing on the logo image using the distortion parameter to obtain a processed logo image.

[0058] Optionally, the calibration parameters also include external parameters; a plurality of identification bands are arranged on the target shelf; the shelf positioning device also includes: a shape estimation module;

[0059] The morphology estimation module is configured to calculate the rotation angle of each identification tape around the rotation axis in the world coordinate system based on the posture information of each identification tape relative to the image sensor and a preset rotation matrix; calculate the posture information of each identification tape in the world coordinate system based on the posture information of each identification tape relative to the image sensor and external parameters; fit the curve shape between any two identification tapes based on the rotation angle and posture information of the two identification tapes in the world coordinate system; and obtain the current shape of the target shelf based on the curve shape between any two identification tapes.

[0060] Optionally, the identification tape is in a rectangular shape and is disposed on a crossbeam of the target shelf; the target shelf includes a plurality of crossbeams, and an identification tape is disposed on each crossbeam.

[0061] According to a third aspect of an embodiment of the present invention, there is provided a shelf handling device, comprising: an image sensor, a memory, and a processor;

[0062] The image sensor is used to capture a target image of a target shelf and transmit the target image to a processor;

[0063] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the above-mentioned shelf positioning method is implemented.

[0064] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the above-mentioned shelf positioning method is implemented.

[0065] The present embodiment provides a shelf positioning method, device, shelf handling equipment and storage medium. The shelf positioning method includes: obtaining a target image captured by an image sensor for a target shelf and calibration parameters of the image sensor, wherein a marking tape composed of visual markings is arranged on the target shelf, extracting a marking image and a preset marking point of the visual marking in the marking image from the target image captured by the image sensor, calculating the position information of the marking tape relative to the image sensor based on the calibration parameters, the marking image and the preset marking point, and determining the positioning result of the target shelf according to the position information. By extracting the marking image in the captured target image and the preset marking point of the visual marking in the marking image, the position information of the marking tape relative to the image sensor is calculated based on the calibration parameters, the marking image and the preset marking point of the image acquisition device. Since the marking tape is pre-arranged on the target shelf and is composed of visual markings, the positioning result of the target shelf can be obtained. Using the marking tape to locate the target shelf avoids signal interference in the actual scene, thereby improving the accuracy of the shelf positioning result. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 is a flow chart of a shelf positioning method provided by an embodiment of the present invention;

[0067] Figure 2 is a flow chart of another shelf positioning method provided by one embodiment of the present invention;

[0068] Figure 3 is a flow chart of another shelf positioning method provided by an embodiment of the present invention;

[0069] Figure 4 is a structural schematic diagram of a shelf positioning device provided by an embodiment of the present invention;

[0070] Figure 5 It is a structural block diagram of a shelf handling device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited to the specific implementation disclosed below.

[0072] The terms used in one or more embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present invention. The singular forms of "a", "said" and "the" used in one or more embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present invention refers to and includes any or all possible combinations of one or more associated listed items.

[0073] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present invention, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present invention, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0074] First, the terms involved in one or more embodiments of the present invention are explained.

[0075] Shelf handling equipment is a kind of automated equipment that can transport shelves.

[0076] Binarization processing: The technology of setting the grayscale value of pixels on the image to 0 or 255, that is, making the entire image appear to be only black and white.

[0077] CCD camera: CCD is the abbreviation of Charge Coupled Device, which can convert light into electric charge and store and transfer the charge, and can also take out the stored charge to change the voltage. CCD cameras are widely used because of their small size, light weight, unaffected by magnetic fields, and anti-vibration and impact characteristics.

[0078] Image sensor calibration: In the image measurement process and machine vision applications, in order to determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image, a geometric model of image sensor imaging must be established. These geometric model parameters are the parameters of the image sensor. Under most conditions, these parameters must be obtained through experiments and calculations. The process of solving the parameters is called calibration. The main calibration methods are: traditional calibration method, active vision calibration method, self-calibration method, and zero distortion calibration method.

[0079] Visual logo: a pattern with special identification features, including QR codes, special geometric patterns, special logos or brand names, etc.

[0080] The present invention provides a shelf positioning method, and also provides a shelf positioning device, a shelf handling equipment, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0081] The execution subject of the shelf positioning method provided in the embodiment of the present invention may be a computing device, a server, etc. for performing shelf positioning, or a shelf transport device for automatically transporting shelves. The shelf positioning method provided in the embodiment of the present invention may be implemented by at least one of software, hardware circuits, and logic circuits set in the execution subject.

[0082] Figure 1 A flow chart of a shelf positioning method provided by an embodiment of the present invention is shown, which specifically includes the following steps.

[0083] Step 102 , obtaining a target image captured by an image sensor for a target shelf and calibration parameters of the image sensor, wherein an identification tape is arranged on the target shelf, and the identification tape is composed of visual identification.

[0084] In an embodiment of the present invention, the image sensor is a sensor that can support the conversion of optical images into digital signals and transmit digital signals of the image to the processor, such as a CCD camera. The image sensor captures the scene within the field of view, and transmits the captured target image to the execution subject of the embodiment of the present invention, and the execution subject obtains the target image captured by the image sensor. In an embodiment of the present invention, the scene within the field of view of the image sensor is the target shelf. In general, the image sensor can only capture a specific part of the target shelf, which is arranged with an identification tape (the identification tape is generally artificially pre-pasted on the target shelf). The identification tape can be pasted on the beam, upright, bottom, top and other positions of the target shelf that can be captured by the image sensor. The shape of the identification tape is not specifically limited, and can be a rectangle, a circle, a diamond, etc. In an implementation of the embodiment of the present invention, the image sensor is installed at the front end of the shelf handling equipment. In order to facilitate the image acquisition of the image sensor, the specific part can be the beam at the bottom of the target shelf. The identification tape is composed of a visual mark, and the visual mark can be a two-dimensional code, a special geometric pattern, a special mark or a brand, etc., that is, the identification tape includes at least one identification mark, and can be composed of a plurality of visual marks arranged to form the identification tape.

[0085] In an implementation of the embodiment of the present invention, the identification belt is in a rectangular shape and is arranged on a crossbeam of the target shelf.

[0086] Since the crossbeam of the shelf is located on the side of the shelf, it can be easily photographed by the image sensor, and the crossbeam of the shelf is generally long and narrow. Therefore, using a rectangular identification tape can improve the positioning accuracy. In order to ensure higher positioning accuracy, the size of the identification tape can be set to be the same as the size of the crossbeam.

[0087] In the embodiment of the present invention, the image sensor is pre-calibrated, and therefore, the calibration parameters of the image sensor can be obtained. The specific calibration method can be to calibrate the image sensor using a traditional calibration method, an active vision calibration method, a self-calibration method, or a zero-distortion calibration method. The specific calibration implementation process is not the main content discussed in the present invention and will not be described in detail here.

[0088] The calibration parameters mainly include intrinsic parameters, extrinsic parameters and distortion parameters. Among them, the intrinsic parameter is a 3x3 parameter matrix used to reflect the impact of the focal length properties of the image sensor on the image; the extrinsic parameter refers to the setting parameters of the image sensor relative to the world coordinate system; the distortion parameter includes radial distortion and tangential distortion, which is a set of arrays related to the degree of distortion of the image captured by the image sensor. The format of the distortion parameters of different image sensors is different.

[0089] When the image sensor collects the target image, it is necessary to ensure that the image sensor is in the current installation position or movable range, the identification belt should be included in the field of view, and the identification belt should be located in the center of the field of view as much as possible. In order to improve the positioning accuracy, the complete identification belt should be included as much as possible.

[0090] Step 104: extracting the identification image and the preset identification points of the visual identification in the identification image from the target image.

[0091] After acquiring the target image captured by the image sensor, the image content is identified and the identification image is extracted therefrom. The identification image refers to the image of the area where the identification band is located in the target image captured by the image sensor. The specific method of extracting the identification image can be to input the image into a pre-trained recognition model based on a neural network, and the recognition model can directly output the identification image in the image; the method of extracting the identification image can also be based on image processing, such as binarization of the image, pixel recognition, etc.

[0092] For the subsequent calculation of posture information, it is necessary to extract the preset identification points of the visual identification in the identification image. The preset identification points refer to points on the visual identification that are easy to extract, have a certain positional relationship, and relatively wrap the visual identification. For example, a square QR code will extract the four corner points, a circular dot matrix will extract the centers of each small circle, and a triangle polygon or other geometric features will also extract corner points.

[0093] In one implementation of the embodiment of the present invention, step 104 may be implemented in the following manner:

[0094] Binarization is performed on the target image to obtain a binary image;

[0095] Identify the covered area of ​​the visual logo in the binarized image;

[0096] A logo image composed of a coverage area is determined, and preset logo points of a visual logo are extracted from the coverage area.

[0097] In order to conveniently extract the identification band from the target image, the target image can be binarized. The binarization can adopt adaptive binarization, that is, it is used to determine whether a certain pixel point in the target image is a darker or brighter area in the nearby interval, that is, the pixel value of the pixel point is compared with the average or weighted average of the pixel values ​​of the surrounding area to obtain a binary image. The binary image presents an obvious visual effect of only black and white. The coverage area of ​​the visual identification will be obviously different from the surrounding area. Therefore, the coverage area of ​​the visual identification can be identified from the binary image. Further, the identification image composed of the coverage area can be determined, and the preset identification point of the visual identification can be extracted from the coverage area. The use of the binarization processing method improves the efficiency and accuracy of extracting the identification image.

[0098] In one implementation of the embodiment of the present invention, the step of performing binarization processing on the target image to obtain a binarized image can be specifically implemented in the following manner:

[0099] Extracting the region of interest in the target image;

[0100] The region of interest is binarized to obtain a binary image.

[0101] Since certain acquisition rules are generally set during image acquisition, such as the identification band is near the center of the field of view of the image sensor, etc., the region of interest (ROI) in the target image can be first extracted according to such acquisition rules. Of course, in addition to the above-mentioned extraction according to the acquisition rules, the ROI area in the target image can also be extracted using a neural network-based method, that is, the target image is input into a pre-trained neural network model, and the neural network model identifies the area in the target image that may have the identification band, which is the ROI area. After the ROI area is extracted, the ROI area is binarized to obtain a binary image. By extracting the ROI area in the target image and then binarizing the ROI area, the range of the ROI area is smaller than the range of the original target image, and it more accurately contains the identification band, thereby improving the accuracy of extracting the identification image.

[0102] In an implementation of the embodiment of the present invention, the step of identifying the coverage area of ​​the visual identifier in the binary image can be specifically implemented in the following manner:

[0103] Perform boundary detection on the binary image to determine the boundary information of the binary image;

[0104] Based on the boundary information, the coverage area of ​​the visual logo in the binary image is identified.

[0105] The boundary refers to the dividing line with high black-white contrast in the binary image, that is, there is a high black-white contrast on both sides of the boundary. Based on this, the boundary detection of the binary image can be performed according to the change of the black-white contrast to determine the boundary information of the binary image. The boundary information may include the position, shape, length, etc. of the boundary. Based on the boundary information of the binary image, all possible lines in the binary image can be obtained. By combining these lines, the coverage area that may be the visual logo can be identified. By determining the coverage area of ​​the visual logo through boundary detection, the coverage area of ​​each visual logo can be identified more conveniently and quickly, thereby improving the accuracy and efficiency of extracting the logo image.

[0106] In an implementation of the embodiment of the present invention, before the step of identifying the coverage area of ​​the visual identifier in the binary image based on the boundary information, the boundary information is filtered using a preset filtering method to obtain filtered boundary information.

[0107] Since some boundary lines do not meet the actual length requirements or some boundary angles do not meet the actual angle requirements during boundary detection, in order to exclude these boundaries that do not meet the requirements, after obtaining the boundary information of the binary image, the boundary information needs to be filtered. The filtering mainly filters the features of the above boundary lines or line combinations, such as excluding lines whose length or direction does not meet the requirements, excluding line combinations whose angles do not meet the requirements, etc. The specific filtering method can be determined according to the type of visual identification. The filtered boundary information is more in line with the actual requirements, and the extracted identification image is more in line with the actual situation and has higher accuracy.

[0108] Step 106, calculating the position information of the marker tape relative to the image sensor based on the calibration parameters, the marker image and the preset marker points.

[0109] After extracting the identification image and the preset identification points of the visual identification, the position information of the identification tape relative to the image sensor can be calculated based on the calibration parameters, the identification image and the preset identification points. Specifically, the calibration parameters stipulate some properties of the image sensor when collecting images, such as the distortion of the collected images, the focal length, etc., and the preset identification points of the visual identification on the identification image are some points that are easy to extract, have a certain position relationship, and relatively wrap the visual identification. Based on the calibration parameters, the identification image and the preset identification points, the position information of the identification tape relative to the image sensor can be calculated.

[0110] Specifically, the preset identification points of the visual identification in the identification image are normalized using internal parameters, and the first coordinates (u, v, 1) of the preset identification points of the visual identification on the identification image can be obtained. At the same time, because the visual identification is pre-designed, the second coordinates (x, y, z) of the preset identification points in the world coordinate system can be obtained. Because (u, v, 1) is actually the projection of the line from the image sensor to (x, y, z) on the image plane, and the preset identification points of the visual identification are on the same plane, therefore, the relative position information of the preset identification points relative to the image sensor and the relative posture information of the plane where the preset identification points are located relative to the image sensor can be calculated based on the first coordinate and the second coordinate using trigonometric relationships.

[0111] The specific calculation process can be as follows: let the rotation relationship between the image plane and the plane to which the preset identification point of the visual identification belongs be represented by the rotation matrix R, and the translation relationship between the image plane and the origin of the plane to which the preset identification point belongs be represented by the displacement vector T. According to the conversion relationship between the coordinate systems, the relationship between the first coordinate (u, v, 1) and the second coordinate (x, y, z) is as shown in formula (1):

[0112]

[0113] Under the premise that there are enough marking points, R and T can be obtained by solving formula (1), that is, the relative rotation and relative displacement between the image plane and the plane to which the preset marking points belong. When the number of marking points exceeds the number required to solve formula (1), the least square method can be used to solve it.

[0114] Specifically, in one implementation of the embodiment of the present invention, the identification band is composed of multiple visual identifications; the calibration parameters include internal parameters. Accordingly, step 106 can be implemented in the following manner:

[0115] Using the internal reference, normalizing the preset identification points of each visual identification in the identification image to obtain the first coordinates of the preset identification points of each visual identification on the identification image;

[0116] Obtaining the second coordinates of the preset identification points of each visual identification in the world coordinate system;

[0117] Calculate the third coordinate of the center point of each visual mark on the mark image according to the first coordinate;

[0118] Calculate the fourth coordinate of the center point of each visual identifier in the world coordinate system according to the second coordinate;

[0119] According to the third coordinate and the fourth coordinate, the position information of the marking tape relative to the image sensor is calculated with the constraint that the center point of each visual marking satisfies the preset arrangement condition.

[0120] In this embodiment, the identification band can be composed of multiple visual identifications. The preset identification points of each visual identification in the identification image are normalized using internal parameters to obtain the first coordinates of the preset identification points of each visual identification on the identification image. At the same time, because the visual identification is pre-designed, the second coordinates of the preset identification points of each visual identification in the world coordinate system can be obtained. Furthermore, the third coordinates of the center point of each visual identification on the identification image can be calculated based on the first coordinates. For example, if the visual identification is a rectangular QR code, the preset identification points are four corner points, and the intersection of the corner point connection line is the center point of the visual identification. The third coordinate of the center point can be calculated based on the first coordinates of the four corner points. Similarly, the fourth coordinate of the center point of each visual identification in the world coordinate system can also be calculated based on the second coordinates. Then, according to the third coordinate and the fourth coordinate, the position information of the marking tape relative to the image sensor is calculated with the center point of each visual mark satisfying the preset arrangement condition as a constraint condition, wherein the preset arrangement condition is the arrangement of the visual marks in the marking tape, which is generally affected by the shape of the marking tape. For example, if the marking tape is rectangular, the visual marks therein are arranged in rows, so the preset arrangement condition is that the center points are on the same straight line. For another example, if the marking tape is circular, the visual marks therein are arranged in a circle, so the preset arrangement condition is that the center points are on the circumference of the same circle. The specific formula for calculating the position information can be formula (1).

[0121] Since it is a marking tape, in principle, the center points of each visual mark on the same marking tape should meet the preset arrangement conditions, and the posture information is calculated based on this constraint condition. This can eliminate the situation of posture jump in advance and ensure the accuracy of posture information calculation.

[0122] In another implementation of the embodiment of the present invention, the calibration parameters include internal parameters. Accordingly, step 106 can also be implemented in the following manner:

[0123] Using the internal reference, the preset marking point is normalized to obtain the first coordinate of the preset marking point on the marking image;

[0124] Obtain the second coordinate of the preset identification point in the world coordinate system;

[0125] Calculate the position information of the marking tape relative to the image sensor according to the first coordinate and the second coordinate;

[0126] Obtain the fifth coordinate of any point on the identification tape in the world coordinate system and the sixth coordinate of the any point on the identification image;

[0127] Calculate the seventh coordinate of any point according to the fifth coordinate and the position and posture information;

[0128] The sixth coordinate is subtracted from the seventh coordinate to obtain the reprojection error. The pose information is adjusted with the reprojection error being less than a preset threshold as a constraint.

[0129] By normalizing the preset identification points of the visual identification in the identification image using internal parameters, the first coordinates of the preset identification points on the identification image can be obtained. At the same time, because the visual identification is pre-designed, the second coordinates of the preset identification points in the world coordinate system can be obtained. For situations where posture jumps may occur, it is necessary to perform iterative optimization under the premise that the preset angle jump may exist. During the optimization process, the part with more pixels on the image can achieve higher calculation accuracy. The rectangular identification belt can further improve the accuracy of the roll angle and yaw angle.

[0130] Specifically, the fifth coordinate of any point on the identification tape in the world coordinate system and the sixth coordinate of any point on the identification image are obtained. According to the fifth coordinate and the posture information, the seventh coordinate of any point is calculated (the seventh coordinate is the image point coordinate of any point in the current posture), and then the sixth coordinate is subtracted from the seventh coordinate to obtain the reprojection error. The posture information is adjusted with the constraint that the reprojection error is less than the preset threshold. After iterative optimization, the optimal posture information can be obtained. This process is also called reverse calculation, which can effectively deal with the situation of posture jumping and ensure the accuracy of posture information calculation.

[0131] In an implementation manner of the embodiment of the present invention, the calibration parameters also include distortion parameters; the method may also include the following steps:

[0132] The logo image is dedistorted by using the distortion parameter to obtain a processed logo image.

[0133] The extracted logo image is dedistorted using distortion parameters. The processed logo image is a planar image, which offsets the error caused by the distortion and can improve the accuracy of the final positioning.

[0134] Step 108: Determine the positioning result of the target shelf according to the position information.

[0135] After obtaining the position information of the identification tape relative to the image sensor, since the setting position of the identification tape on the target shelf is certain, the positioning result of the target shelf can be determined based on the position information. The positioning result of the target shelf is the position information of the target shelf relative to the image sensor calculated based on the position information and the preset setting position of the identification tape.

[0136] In an implementation of the embodiment of the present invention, the calibration parameters also include external parameters; a plurality of identification tapes are arranged on the target shelf;

[0137] In step 108, the following steps may also be included:

[0138] Calculate the rotation angle of each marking tape around the rotation axis in the world coordinate system according to the position information of each marking tape relative to the image sensor and the preset rotation matrix;

[0139] Calculate the pose information of each marking band in the world coordinate system according to the pose information of each marking band relative to the image sensor and the external parameters;

[0140] According to the rotation angle and posture information of any two marking belts in the world coordinate system, the curve shape between any two marking belts is fitted;

[0141] According to the curve shape between any two identification bands, the current shape of the target shelf is obtained.

[0142] In practical applications, a target shelf may be arranged with multiple identification tapes. After obtaining the position information of each identification tape relative to the image sensor, the position information of each identification tape relative to the image sensor may be combined to describe the shape of the target shelf.

[0143] Specifically, let the point on a certain surface of the target shelf be One of the points is the point where the label is attached. jk The pose is R jk ,T jk , according to the preset rotation matrix, the rotation angle of the point around the three rotation axes in the world coordinate system can be obtained. Since the pose information of these m×n points relative to the image sensor is obtained, the pose information of each point in the world coordinate system can be converted under the premise of having the external parameters of the image sensor relative to the world coordinate system. Since the shelf is a frame structure, the shape of the curve between any two points can be fitted according to the rotation angle along the frame direction at the two points and the pose information of the two points relative to the world coordinate system. Similarly, as long as the curve shape between the points facing the image sensor is fitted, it is the current shape of the target shelf in front of it.

[0144] This embodiment adds a target shelf shape assessment function, which can provide more suitable data support for shelf transportation, etc.

[0145] In one or more embodiments of the present invention, the identification tape may be in a rectangular shape, and the identification tape may be arranged on a beam of the target shelf; the target shelf may include a plurality of beams, and an identification tape is arranged on each beam.

[0146] The embodiment of the present invention is applied, by obtaining the target image captured by the image sensor for the target shelf and the calibration parameters of the image sensor, wherein the target shelf is arranged with a marking tape composed of visual markings, extracting the marking image and the preset marking points of the visual markings in the marking image from the target image captured by the image sensor, calculating the position information of the marking tape relative to the image sensor based on the calibration parameters, the marking image and the preset marking points, and determining the positioning result of the target shelf according to the position information. By extracting the marking image in the captured target image and the preset marking points of the visual markings in the marking image, the position information of the marking tape relative to the image sensor is calculated based on the calibration parameters, the marking image and the preset marking points of the image acquisition device, and since the marking tape is pre-arranged on the target shelf and is composed of visual markings, the positioning result of the target shelf can be obtained. Using the marking tape to locate the target shelf avoids signal interference in the actual scene, thereby improving the accuracy of the shelf positioning result.

[0147] based on Figure 1 The embodiment shown, Figure 2 A flow chart of another shelf positioning method provided by an embodiment of the present invention is shown, which specifically includes the following steps.

[0148] Step 102 , obtaining a target image captured by an image sensor for a target shelf and calibration parameters of the image sensor, wherein an identification tape is arranged on the target shelf, and the identification tape is composed of visual identification.

[0149] Step 104: extracting the identification image and the preset identification points of the visual identification in the identification image from the target image.

[0150] Step 110: Use a preset verification method to verify whether the visual identification in the identification image meets the preset visual identification characteristics. If the verification result is passed, execute step 106, otherwise return to execute step 104.

[0151] Step 106, calculating the position information of the marker tape relative to the image sensor based on the calibration parameters, the marker image and the preset marker points.

[0152] Step 108: Determine the positioning result of the target shelf according to the position information.

[0153] Among them, steps 102, 104, 106, and 108 are Figure 1 The steps in the illustrated embodiment are the same and will not be described again here.

[0154] Since the preset identification points of the visual identification in the extracted identification image may not be the identification points of the actual visual identification due to various reasons, but may be caused by insufficient filtering, it is necessary to verify whether the visual identification in the identification image meets the preset visual identification features and confirm whether the visual identification is the pre-set visual identification. For example, the QR code will be decoded, the geometric features will be feature checked, and the special identification will be image matched, etc. That is, it is necessary to use the preset verification method to verify whether the visual identification in the identification image meets the preset visual identification features. Only when the verification is passed, the step of calculating the posture information will be executed. The specific verification method can be to input the identification image into the corresponding verification function or model to confirm whether the output result is verified.

[0155] The embodiment of the present invention is applied, by obtaining the target image captured by the image sensor for the target shelf and the calibration parameters of the image sensor, wherein the target shelf is arranged with a marking tape composed of visual markings, extracting the marking image and the preset marking points of the visual markings in the marking image from the target image captured by the image sensor, calculating the position information of the marking tape relative to the image sensor based on the calibration parameters, the marking image and the preset marking points, and determining the positioning result of the target shelf according to the position information. By extracting the marking image in the captured target image and the preset marking points of the visual markings in the marking image, the position information of the marking tape relative to the image sensor is calculated based on the calibration parameters, the marking image and the preset marking points of the image acquisition device, and since the marking tape is pre-arranged on the target shelf and is composed of visual markings, the positioning result of the target shelf can be obtained. Using the marking tape to locate the target shelf avoids signal interference in the actual scene, thereby improving the accuracy of the shelf positioning result. Before calculating the pose information, a preset verification method is used to verify whether the visual identification in the identification image meets the preset visual identification features. Only when the verification is passed will the pose information calculation step be executed, avoiding the problem of the preset identification points not conforming to the actual situation due to insufficient filtering, and the positioning result is more accurate.

[0156] For ease of understanding, the shelf positioning method provided by the embodiment of the present invention is introduced below in combination with a specific application scenario. In the embodiment of the present invention, a CCD camera is installed at the front end of the shelf handling device, and the shelf handling device also includes a processor, which executes the following Figure 3 The shelf positioning method shown, Figure 3 A flow chart of another shelf positioning method provided by an embodiment of the present invention is shown, which specifically includes the following steps.

[0157] The first step is to calibrate the CCD camera, obtain the internal parameters and distortion parameters of the CCD camera, and arrange the visual signs in specific parts of the shelf.

[0158] In the second step, a CCD camera is used to capture a picture containing the complete identification tape in the field of view.

[0159] The third step is to perform binarization on the camera ROI area containing the identification band to obtain a binary image.

[0160] The binarization process here is adaptive binarization, which is used to determine whether a pixel in the image is a darker or brighter area in the surrounding area, that is, the pixel value of the pixel is compared with the average or weighted average of the pixel values ​​in the surrounding area.

[0161] The fourth step is to perform boundary detection in the binary image, find all boundaries in the binary image, and perform boundary filtering to extract the identification image and the preset identification points of each visual identification in the identification image that participate in the pose calculation.

[0162] According to the obvious pattern of black and white pairs on both sides of the boundary, the binary image is detected to determine all the boundaries in the binary image. Extracting boundaries can be roughly understood as extracting all possible lines in the binary image, and combining and filtering these lines to find out the areas that may be visual logos. Boundary filtering mainly filters the features of these lines or line combinations, such as excluding lines whose length or direction does not meet the requirements, or excluding line combinations that do not meet the requirements at certain angles. The specific filtering method is determined according to the type of visual logo. According to the results of boundary detection, the logo image can be extracted from the binary image, and the preset logo points of each visual logo participating in the pose calculation can be extracted from the logo image, such as extracting the four corner points of a square QR code.

[0163] The fifth step is to verify each visual marker in the extracted marker image. After verification and confirmation, the position information of the marker belt relative to the CCD camera is calculated based on the internal parameters and distortion parameters of the CCD camera and the preset marker points.

[0164] Since the preset identification points extracted in the previous step may not be the identification points of the actual visual identification due to various reasons, but may be caused by insufficient filtering, it is necessary to verify the unique special visual features of each visual identification in the identification image to confirm whether it meets the preset visual identification features. After verification, the position information of the identification belt relative to the CCD camera is calculated based on the internal parameters and distortion parameters of the CCD camera and the preset identification points. The position information is output in the form of coordinates, and the output result is the position T and posture R of the visual identification relative to the CCD camera. The position T is the position (x, y, z) of the visual identification relative to the CCD camera, and the posture R is the rotation quaternion (x, y, z, w) of the visual identification relative to the CCD camera.

[0165] After obtaining the position information of the visual identification to the CCD camera, the positioning result of the target shelf can be converted and directly output so that the shelf handling equipment can accurately transport the shelf according to the positioning result.

[0166] The present invention uses visual identification to locate the target shelf. The positioning result of the target shelf is obtained by calculating the position information of the visual identification, which has higher accuracy than the positioning solution of radio frequency technology or WLAN technology, and has higher adaptability than the deep learning solution.

[0167] Corresponding to the above method embodiment, the present invention also provides a shelf positioning device embodiment, Figure 4 FIG. 2 shows a schematic diagram of a shelf positioning device provided by an embodiment of the present invention. Figure 4 As shown, the device comprises:

[0168] The acquisition module 420 is configured to acquire a target image captured by the image sensor for the target shelf and calibration parameters of the image sensor, wherein a marking tape is arranged on the target shelf, and the marking tape is composed of visual markings;

[0169] An extraction module 440 is configured to extract a marker image and a preset marker point of a visual marker in the marker image from a target image;

[0170] The posture calculation module 460 is configured to calculate the posture information of the marking tape relative to the image sensor based on the calibration parameters, the marking image and the preset marking points;

[0171] The positioning module 480 is configured to determine the positioning result of the target shelf according to the position information.

[0172] The embodiment of the present invention is applied, by obtaining the target image captured by the image sensor for the target shelf and the calibration parameters of the image sensor, wherein the target shelf is arranged with a marking tape composed of visual markings, extracting the marking image and the preset marking points of the visual markings in the marking image from the target image captured by the image sensor, calculating the position information of the marking tape relative to the image sensor based on the calibration parameters, the marking image and the preset marking points, and determining the positioning result of the target shelf according to the position information. By extracting the marking image in the captured target image and the preset marking points of the visual markings in the marking image, the position information of the marking tape relative to the image sensor is calculated based on the calibration parameters, the marking image and the preset marking points of the image acquisition device, and since the marking tape is pre-arranged on the target shelf and is composed of visual markings, the positioning result of the target shelf can be obtained. Using the marking tape to locate the target shelf avoids signal interference in the actual scene, thereby improving the accuracy of the shelf positioning result.

[0173] Optionally, the extraction module 440 includes: a binarization processing unit, a recognition unit and an extraction unit;

[0174] A binarization processing unit is configured to perform binarization processing on the target image to obtain a binarized image;

[0175] a recognition unit configured to recognize a coverage area of ​​a visual identifier in the binarized image;

[0176] The extraction unit is configured to determine the identification image composed of the coverage area and extract the preset identification points of the visual identification from the coverage area.

[0177] Optionally, the recognition unit is further configured to perform boundary detection on the binary image to determine boundary information of the binary image; and based on the boundary information, recognize a coverage area of ​​the visual identifier in the binary image.

[0178] Optionally, the identification band is composed of a plurality of visual identifications; the calibration parameters include an internal reference;

[0179] The posture calculation module 460 is further configured to use internal parameters to normalize the preset identification points of each visual marker in the identification image to obtain the first coordinates of the preset identification points of each visual marker on the identification image; obtain the second coordinates of the preset identification points of each visual marker in the world coordinate system; calculate the third coordinates of the center point of each visual marker on the identification image based on the first coordinates; calculate the fourth coordinates of the center point of each visual marker in the world coordinate system based on the second coordinates; and calculate the posture information of the marking belt relative to the image sensor based on the third coordinates and the fourth coordinates, with the center point of each visual marker satisfying the preset arrangement condition as a constraint condition.

[0180] Optionally, the calibration parameters include internal references;

[0181] The posture calculation module 460 is further configured to use internal parameters to normalize the preset marking points to obtain the first coordinates of the preset marking points on the marking image; obtain the second coordinates of the preset marking points in the world coordinate system; calculate the posture information of the marking tape relative to the image sensor based on the first coordinate and the second coordinates; obtain the fifth coordinate of any point on the marking tape in the world coordinate system and the sixth coordinate of the any point on the marking image; calculate the seventh coordinate of any point based on the fifth coordinate and the posture information; subtract the sixth coordinate from the seventh coordinate to obtain the reprojection error, and adjust the posture information with the reprojection error being less than a preset threshold as a constraint condition.

[0182] Optionally, the calibration parameters also include distortion parameters; the device also includes: a dedistortion module;

[0183] The dedistortion module is configured to perform dedistortion processing on the logo image using the distortion parameter to obtain a processed logo image.

[0184] Optionally, the calibration parameters also include external parameters; a plurality of identification bands are arranged on the target shelf; the shelf positioning device also includes: a shape estimation module;

[0185] The morphology estimation module is configured to calculate the rotation angle of each identification tape around the rotation axis in the world coordinate system based on the posture information of each identification tape relative to the image sensor and a preset rotation matrix; calculate the posture information of each identification tape in the world coordinate system based on the posture information of each identification tape relative to the image sensor and external parameters; fit the curve shape between any two identification tapes based on the rotation angle and posture information of the two identification tapes in the world coordinate system; and obtain the current shape of the target shelf based on the curve shape between any two identification tapes.

[0186] Optionally, the identification tape is in a rectangular shape and is disposed on a crossbeam of the target shelf; the target shelf includes a plurality of crossbeams, and an identification tape is disposed on each crossbeam.

[0187] The above is a schematic scheme of a shelf positioning device of this embodiment. It should be noted that the technical scheme of the shelf positioning device and the technical scheme of the shelf positioning method described above are of the same concept, and the details of the technical scheme of the shelf positioning device that are not described in detail can all be referred to the description of the technical scheme of the shelf positioning method described above.

[0188] Figure 5 The structural block diagram of a shelf handling device provided by an embodiment of the present invention is shown. The components of the shelf handling device 500 include but are not limited to an image sensor 510, a memory 520 and a processor 530. The processor 530 is connected to the image sensor 510 and the memory 520 via a bus 540, and a database 560 is used to store data.

[0189] The rack handling device 500 also includes an access device 550, which enables the rack handling device 500 to communicate via one or more networks 570. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 550 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world microwave interconnection access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0190] In one embodiment of the present invention, the above components of the shelf handling device 500 and Figure 5 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 5 The structure diagram of the rack handling device shown is for illustrative purposes only and is not intended to limit the scope of the present invention. Those skilled in the art may add or replace other components as needed.

[0191] The image sensor 510 is used to capture a target image of a target shelf and transmit the target image to the processor 530; the processor 530 is used to execute the following computer executable instructions, which are implemented when executed by the processor:

[0192] Acquire a target image captured by the image sensor 510 and calibration parameters of the image sensor, wherein a marking tape is arranged on the target shelf, and the marking tape is composed of visual markings;

[0193] Extracting a logo image and preset logo points of a visual logo in the logo image from a target image;

[0194] Based on the calibration parameters, the identification image and the preset identification points, the position information of the identification tape relative to the image sensor is calculated;

[0195] According to the position information, the positioning result of the target shelf is determined.

[0196] After obtaining the positioning result, the shelf handling equipment can be controlled to transport the target shelf according to the positioning result.

[0197] The above is a schematic scheme of a shelf handling device of this embodiment. It should be noted that the technical scheme of the shelf handling device and the technical scheme of the shelf positioning method described above are of the same concept, and the details of the technical scheme of the shelf handling device that are not described in detail can all be referred to the description of the technical scheme of the shelf positioning method described above.

[0198] An embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions, which can implement the steps of the above-mentioned shelf positioning method when executed by a processor.

[0199] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the shelf positioning method described above are of the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the shelf positioning method described above.

[0200] The above describes specific embodiments of the present invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0201] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0202] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the described action sequence, because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of the present invention.

[0203] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0204] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of the present invention. The present invention selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A shelf positioning method, characterized in that: include: Acquire a target image captured by an image sensor for a target shelf and calibration parameters of the image sensor, wherein a plurality of identification bands are arranged on the target shelf, the identification bands are composed of visual identifications, and the calibration parameters include external parameters; Extracting a marker image and a preset marker point of a visual marker in the marker image from the target image; Calculating the position information of the marking tape relative to the image sensor based on the calibration parameters, the marking image and the preset marking points; Determine the positioning result of the target shelf according to the position information; Calculate the rotation angle of each marking tape around the rotation axis in the world coordinate system according to the position information of each marking tape relative to the image sensor and the preset rotation matrix; Calculate the pose information of each marking band in the world coordinate system according to the pose information of each marking band relative to the image sensor and the external parameter; According to the rotation angle and posture information of any two marking bands in the world coordinate system, the shape of the curve between any two marking bands is fitted; The current shape of the target shelf is obtained according to the shape of the curve between any two identification bands.

2. The method according to claim 1, characterized in that The step of extracting the identification image and the preset identification points of the visual identification in the identification image from the target image comprises: Performing binarization processing on the target image to obtain a binarized image; identifying a coverage area of ​​the visual marker in the binarized image; A logo image composed of the coverage area is determined, and preset logo points of the visual logo are extracted from the coverage area.

3. The method according to claim 2, characterized in that The step of identifying the coverage area of ​​the visual identifier in the binary image comprises: Performing boundary detection on the binary image to determine boundary information of the binary image; Based on the boundary information, a coverage area of ​​the visual marker in the binarized image is identified.

4. The method according to claim 1, characterized in that: The identification band is composed of a plurality of visual identifications; the calibration parameters include internal parameters; The step of calculating the position information of the marking tape relative to the image sensor based on the calibration parameters, the marking image and the preset marking points comprises: Using the internal reference, normalizing the preset identification points of each visual identification in the identification image to obtain the first coordinates of the preset identification points of each visual identification on the identification image; Obtaining second coordinates of the preset identification points of each visual identification in the world coordinate system; Calculating the third coordinates of the center points of the visual markers on the marker image according to the first coordinates; Calculating, based on the second coordinate, a fourth coordinate of the center point of each visual identifier in a world coordinate system; According to the third coordinate and the fourth coordinate, the position information of the marking tape relative to the image sensor is calculated with the constraint that the center points of the visual markings satisfy the preset arrangement condition.

5. The method according to claim 1, characterized in that The calibration parameters include internal references; The step of calculating the position information of the marking tape relative to the image sensor based on the calibration parameters, the marking image and the preset marking points comprises: Using the internal reference, normalizing the preset identification point to obtain a first coordinate of the preset identification point on the identification image; Obtaining a second coordinate of the preset identification point in the world coordinate system; Calculating the position information of the marking tape relative to the image sensor according to the first coordinate and the second coordinate; Obtaining a fifth coordinate of any point on the identification tape in the world coordinate system and a sixth coordinate of the any point on the identification image; Calculate the seventh coordinate of any point according to the fifth coordinate and the posture information; The sixth coordinate is subtracted from the seventh coordinate to obtain a reprojection error, and the posture information is adjusted with the reprojection error being less than a preset threshold as a constraint condition.

6. The method according to claim 4 or 5, characterized in that: The calibration parameters also include distortion parameters; the method also includes: The logo image is subjected to dedistortion processing using the distortion parameters to obtain the logo image after processing.

7. The method according to any one of claims 1 to 5, characterized in that The identification belt is in a rectangular shape and is arranged on a crossbeam of the target shelf; the target shelf includes a plurality of crossbeams, and the identification belt is arranged on each crossbeam.

8. A shelf positioning device, characterized in that: include: An acquisition module is configured to acquire a target image captured by an image sensor for a target shelf and calibration parameters of the image sensor, wherein a plurality of identification bands are arranged on the target shelf, the identification bands are composed of visual identifications, and the calibration parameters include external parameters; An extraction module is configured to extract a marker image and a preset marker point of a visual marker in the marker image from the target image; A posture calculation module is configured to calculate the posture information of the marking tape relative to the image sensor based on the calibration parameters, the marking image and the preset marking points; A positioning module, configured to determine a positioning result of the target shelf according to the position information; A morphology estimation module is configured to calculate a rotation angle of each marking band around a rotation axis in a world coordinate system according to the position information of each marking band relative to the image sensor and a preset rotation matrix; Calculate the pose information of each marking band in the world coordinate system according to the pose information of each marking band relative to the image sensor and the external parameter; According to the rotation angle and posture information of any two identification tapes in the world coordinate system, the curve shape between any two identification tapes is fitted; according to the curve shape between any two identification tapes, the current shape of the target shelf is obtained.

9. A shelf handling device, comprising: Image sensors, memory and processors; The image sensor is used to capture a target image for a target shelf and transmit the target image to the processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the shelf positioning method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the shelf positioning method according to any one of claims 1 to 7.

Images