Benchmark label detection method and device, electronic equipment and storage medium

Abstract

The application discloses a reference label detection method and device, electronic equipment and storage medium, and relates to the technical field of visual calibration. The method comprises the following steps: acquiring a label image and a label code length of a target reference label; determining graphical identification information of an unobstructed square grid according to an unobstructed area of the label image; generating a candidate grid according to the label code length and the graphical identification information of the unobstructed square grid; selecting a plurality of starting grids as search starting points in the candidate grid, moving a preset step length along a target direction, selecting graphical identification information of a label code length of grids from each starting grid, and forming a candidate sequence corresponding to each starting grid; and comparing the candidate sequences corresponding to each starting grid, and determining a cyclic code with the highest occurrence frequency as label code information of the target reference label. The technical scheme of the embodiment of the application realizes accurate identification of the reference label in the case that the obstruction level is high.

Description

Technical Field

[0001] This invention relates to the field of visual calibration technology, and in particular to a reference label detection method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the widespread application of robotics in various fields, precise robot collaboration has become a key technology in machine intelligence. Visual calibration technology, as an important component of precise robot collaboration, plays a crucial role in vision-based target detection and calibration, especially in surveillance and security. Visual calibration technologies can be divided into two categories: one is based on feature extraction methods, and the other utilizes specially designed fiducial markers. The former typically requires high resolution and extensive computation, and may suffer from low recognition speed due to hardware limitations, making it difficult to meet the needs of real-time applications. The latter, due to its controllable data volume and information, can more effectively handle varied scenarios such as target distortion, rotation, occlusion, and complex backgrounds. Therefore, fiducial markers have been widely used in augmented reality applications, medical, spatial, and robot-assisted tasks.

[0003] The ability to handle occlusion is a key factor in evaluating the performance of a benchmark tag-based localization system, directly impacting its speed and accuracy. Existing benchmark tag systems have limited occlusion handling capabilities. In occlusion experiments with different tags, most benchmark tags become undetectable at occlusion levels of 10%, with only AprilTag, ArUco (tag-based augmented reality), TopoTag (geographic tag), DeepTag (deep learning tag), and RuneTag (Chinese text tag) being detectable at occlusion levels of up to 10%. Specifically, AprilTag, ArUco, TopoTag, and DeepTag can be detected at occlusion levels up to 20%, while RuneTag can be detected at occlusion levels up to 30%. This demonstrates that most benchmark tags perform poorly in resisting occlusion, highlighting the urgent need to design a benchmark tag capable of accurate identification even under high occlusion levels. Summary of the Invention

[0004] This invention provides a reference tag detection method, apparatus, electronic device, and storage medium, which enables accurate identification of reference tags even under conditions of high occlusion.

[0005] According to one aspect of the present invention, a reference label detection method is provided, the method comprising:

[0006] Obtain the label image and label encoding length of the target reference label; note that there are label occlusion areas in the label image;

[0007] Based on the unoccluded area of ​​the label in the label image, determine the graphic identification information of the unoccluded square grid in the label image;

[0008] Based on the label code length and the graphic identification information of the unoccluded square grids in the label image, candidate grids are generated; the graphic identification information of the square grids in the candidate grids is the same as that of the square grids corresponding to each occluded square grid in the label image.

[0009] In the label image, for each unoccluded square grid, multiple starting grids are selected from the corresponding square grids in the candidate grids as search starting points. Along the target direction, the target length corresponding to the target direction is moved sequentially. Starting from each starting grid, the graphic identification information of grids with a label encoding length is selected to form a candidate sequence corresponding to each starting grid. The target direction includes mutually perpendicular arrangement directions or movement directions. The target length corresponding to the arrangement direction is the length of a single square grid; the target length corresponding to the movement direction is a preset step length.

[0010] The candidate sequences corresponding to each starting grid are compared, and the cyclic code that appears most frequently is determined as the tag encoding information of the target reference label.

[0011] According to another aspect of the present invention, a reference label detection device is provided, the device comprising:

[0012] The label image acquisition module is used to acquire the label image of the target reference label and the label encoding length of the target reference label; wherein, there are label occlusion areas in the label image;

[0013] The graphic label information determination module is used to determine the graphic identification information of the unoccluded square grid in the label image based on the unoccluded area of ​​the label in the label image;

[0014] The alternative grid generation module is used to generate alternative grids based on the label code length and the graphic identification information of the unoccluded square grids in the label image; the graphic identification information of the square grids in the alternative grids is the same as that of the square grids corresponding to each occluded square grid in the label image.

[0015] The candidate sequence generation module is used to select multiple starting grids as search starting points from the square grids corresponding to each unoccluded square grid in the label image in the candidate grids. It then moves sequentially along the target direction by the target length corresponding to that direction. Starting from each starting grid, it selects the graphic identification information of grids with a label encoding length to form a candidate sequence corresponding to each starting grid. The target direction includes mutually perpendicular arrangement directions or movement directions. The target length corresponding to the arrangement direction is the length of a single square grid. The target length corresponding to the movement direction is a preset step length.

[0016] The label encoding information determination module is used to compare the candidate sequences corresponding to each starting grid and determine the cyclic code that appears most frequently as the label encoding information of the target reference label.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] At least one processor; and

[0019] A memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the reference tag detection method according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the reference tag detection method according to any embodiment of the present invention.

[0022] The technical solution of this invention obtains the label image and label encoding length of the target reference label. Based on the unoccluded area of ​​the label in the label image, it determines the graphic identifier information of the unoccluded square grids in the label image. Based on the label encoding length and the graphic identifier information of the unoccluded square grids in the label image, it generates candidate grids. This achieves the conversion of image information of the label image into graphic identifier information. Because the graphic identifier information of the square grids in the candidate grids is the same as that of the square grids corresponding to each occluded square grid in the label image, interference from the occluded square grids in the candidate grids on the decoding process of the target reference label based on the label image is avoided. Multiple starting grids are selected from the square grids corresponding to each unoccluded square grid in the label image in the candidate grids as search starting points. The target length corresponding to the target direction is moved sequentially along the target direction. Starting from each starting grid, a label encoding length is selected... The graphic identification information of the grid forms a candidate sequence corresponding to each starting grid. The target direction includes mutually perpendicular arrangement directions or movement directions. The target length corresponding to the arrangement direction is the length of a single square grid, and the target length corresponding to the movement direction is a preset step length. Based on the graphic identification information of the square grids corresponding to each unoccluded square grid in the label image in the candidate grid, and the correlation between the graphic identification information of square grids in the target reference label, the decoding and detection of the label encoding information of the target reference label based on the label image with occlusion areas is realized, which improves the detection and recognition rate of the target reference label. By comparing the candidate sequences corresponding to each starting grid, the cyclic code that appears most frequently is determined as the label encoding information of the target reference label, which further improves the accuracy of the detection of the label encoding information of the target reference label and realizes accurate identification of the reference label under the condition of high occlusion level.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of a reference label detection method provided in Embodiment 1 of the present invention;

[0026] Figure 2This is a schematic diagram of 18 types of 7×7 target reference tags provided in Embodiment 1 of the present invention;

[0027] Figure 3 This is a schematic diagram of an ARToolKit reference tag provided according to Embodiment 1 of the present invention;

[0028] Figure 4 This is a schematic diagram of an ARTag reference tag provided according to Embodiment 1 of the present invention;

[0029] Figure 5 This is a schematic diagram of an AprilTag reference tag provided according to Embodiment 1 of the present invention;

[0030] Figure 6 This is a schematic diagram of an ArUco reference tag provided according to Embodiment 1 of the present invention;

[0031] Figure 7 This is a schematic diagram of a RuneTag reference tag provided according to Embodiment 1 of the present invention;

[0032] Figure 8 This is a schematic diagram of a CCTag reference tag provided according to Embodiment 1 of the present invention;

[0033] Figure 9 This is a schematic diagram of a TopoTag reference label provided according to Embodiment 1 of the present invention;

[0034] Figure 10 This is a schematic diagram of a DeepTag reference tag provided according to Embodiment 1 of the present invention;

[0035] Figure 11 This is a flowchart of a reference label detection method provided in Embodiment 2 of the present invention;

[0036] Figure 12 This is a schematic diagram of an 11×11 target reference tag provided according to Embodiment 2 of the present invention;

[0037] Figure 13 This is a schematic diagram of the target reference label for the width of the first square grid provided in Embodiment 2 of the present invention;

[0038] Figure 14 This is a schematic diagram of the target reference label for the width of the second square grid provided in Embodiment 2 of the present invention;

[0039] Figure 15 This is a schematic diagram of a target reference label containing a square graphic according to Embodiment 2 of the present invention;

[0040] Figure 16This is a schematic diagram of a target reference label containing a circular graphic according to Embodiment 2 of the present invention;

[0041] Figure 17 This is a schematic diagram of the target reference label representing the proportion of the area of ​​the first graphic provided in Embodiment 2 of the present invention;

[0042] Figure 18 This is a schematic diagram of the target reference label representing the proportion of the area of ​​the second graphic provided in Embodiment 2 of the present invention;

[0043] Figure 19 This is a flowchart of a reference label detection method provided in Embodiment 2 of the present invention;

[0044] Figure 20 This is a label image of a target reference label when 57% is obscured from the lower left, according to Embodiment 2 of the present invention;

[0045] Figure 21 This is a label image of a target reference label when it is 78% obscured from directly above and below, according to Embodiment 2 of the present invention;

[0046] Figure 22 This is a schematic diagram of a target reference label containing a circular graphic according to Embodiment 2 of the present invention;

[0047] Figure 23 This is a label image of a target reference label when 78% is obscured from the upper left and lower right, as provided in Embodiment 2 of the present invention;

[0048] Figure 24 A schematic diagram of the structure of a reference label detection device according to Embodiment 3 of the present invention;

[0049] Figure 25 This is a schematic diagram of the structure of an electronic device that implements the reference tag detection method of this invention. Detailed Implementation

[0050] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0051] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0052] Example 1

[0053] Figure 1 This is a flowchart illustrating a reference label detection method provided in Embodiment 1 of the present invention. This embodiment of the invention is applicable to detecting reference labels under conditions of high occlusion. The method can be executed by a reference label detection device, which can be implemented in hardware and / or software. This reference label detection device can be configured in an electronic device that carries the reference label detection function, such as in a robot.

[0054] See Figure 1 The benchmark label detection method shown includes:

[0055] S101. Obtain the label image of the target reference label and the label encoding length of the target reference label.

[0056] When robots require high-precision positioning, target reference tags can be used to assist in positioning. Each target reference tag has a unique tag code. The robot can identify the tag code information to determine the coordinates of the corresponding positioning point, thus assisting in positioning. The tag code length can be the length of the tag code information. Optionally, increasing the tag code length can provide a greater number of selectable tag codes. The tag code length can be set and adjusted by technicians based on the robot's application scenario and the computing efficiency of the computing device. It can be understood that the number of reference tags required varies in different application scenarios, and to ensure the uniqueness of the tag code information, the minimum tag code length also varies in different application scenarios. Increasing the tag code length does not significantly affect the accuracy of reference tag recognition; however, in actual experiments, it has been found that increasing the tag code length leads to a longer detection time for the computing device. Therefore, when using computing devices with different computing efficiencies to detect reference tags, the maximum tag code length will also vary. Optionally, in the same application scenario, such as fruit picking, the label encoding length of each baseline label is the same.

[0057] The label image can be an image of the target reference label captured by the robot. The label image contains occluded areas. This can be understood as follows: during the robot's camera image acquisition of the target reference label, occlusion areas may appear in the captured image due to obstructions in the application environment (such as leaves in a fruit-picking scenario) or lighting conditions. These occluded areas are regions in the label image where the original graphic of the target reference label cannot be recognized.

[0058] Specifically, the robot's camera can capture image of the target reference tag. The tag code length of the target reference tag, pre-set by the technician, can also be obtained.

[0059] S102. Based on the unoccluded area of ​​the label in the label image, determine the graphic identification information of the unoccluded square grid in the label image.

[0060] The unoccluded area of ​​the label can be the region in the label image where the original graphic of the target reference label can be identified. The unoccluded square grid can be a square grid in the label image where the graphic identification information can be identified. The target reference label can include the square grid and the graphic contained within the square grid. The graphic identification information can be used to identify whether the square grid contains a graphic. Optionally, the graphic identification information can be represented in the form of numbers, letters, strings, or text. For example, the graphic identification information can be 0 or 1; where 0 can be used to indicate that the corresponding grid does not contain a graphic; 1 can be used to indicate that the corresponding grid contains a graphic.

[0061] Specifically, based on the outline of the square grid and the outline of the graphic in the unoccluded area of ​​the label graphic, it can be used to detect whether the unoccluded square grid in the label image contains a graphic. Based on the detection results of whether the unoccluded square grid in the label image contains a graphic, the graphic identification information of the unoccluded square grid in the label image can be determined.

[0062] In an optional embodiment of the present invention, determining the graphic identification information of the unobstructed square grid in the label image based on the unobstructed area of ​​the label image includes: detecting the remaining area of ​​the unobstructed square grid in the label image based on the unobstructed area of ​​the label image; and determining the graphic identification information of the unobstructed square grid in the label image when the remaining area of ​​the grid is greater than or equal to a preset remaining area.

[0063] The remaining area of ​​the unoccluded square grid can be considered as the remaining area of ​​the unoccluded square grid in the label image. This remaining area can be used to characterize the accuracy of detecting the graphic identifier information of the unoccluded square grid. It can be understood that when there is a label occlusion area in the label image of the target reference label, the unoccluded square grid in the label image can include completely unoccluded square grids and / or partially occluded square grids. During the detection of graphic identifier information of the unoccluded square grid, the accuracy of the graphic identifier information determined based on completely unoccluded square grids is certain; however, the accuracy of the graphic identifier information determined based on partially occluded square grids is uncertain. Therefore, by introducing the remaining area of ​​the unoccluded square grid, the unoccluded square grids are filtered to improve the accuracy of the graphic identifier information of the unoccluded square grids in the determined label image, thereby improving the recognition accuracy of the label encoding information of the target reference label. The preset remaining area can be a pre-set minimum value of the remaining grid area. Optionally, the preset remaining area can be the minimum remaining area of ​​the square grid that allows for the identification of graphics contained within the square grid. The preset remaining area can be set and adjusted by technicians based on the area of ​​the square grid and the area of ​​the graphics in the target reference label. A remaining grid area greater than or equal to the preset remaining area means that, based on the unobstructed square grid, at least a partial outline of the graphics contained within the square grid can be identified to determine whether the unobstructed square grid contains graphics. A remaining grid area less than the preset remaining area means that, based on the unobstructed square grid, it is difficult to identify the outline of the graphics contained within the square grid, making it difficult to determine whether the unobstructed square grid contains graphics.

[0064] Specifically, based on the outlines of the square grid and the graphic in the unoccluded area of ​​the label image, the remaining area of ​​the unoccluded square grid in the label image can be detected. This remaining area is compared to a preset remaining area. If the remaining area is greater than or equal to the preset remaining area, it is then determined whether the unoccluded square grid in the label image contains a graphic, based on the outlines of the square grid and the graphic in the unoccluded area of ​​the label graphic. Based on the detection results of whether the unoccluded square grid in the label image contains a graphic, the graphic identification information of the unoccluded square grid in the label image is determined.

[0065] This solution further filters unobstructed square grids in label images by comparing the remaining area of ​​the unobstructed square grids with a preset remaining area. This improves the accuracy of the graphic identification information of unobstructed square grids in label images, thereby enhancing the accuracy of label encoding information detection for target reference labels.

[0066] In an optional embodiment of the present invention, determining the graphic identification information of the unobstructed square grid in the label image based on the unobstructed area of ​​the label image includes: detecting the remaining area of ​​the unobstructed square grid in the label image based on the outline of the square grid in the unobstructed area of ​​the label image; detecting the remaining area of ​​the graphic contained in the unobstructed square grid based on the graphic outline in the unobstructed area of ​​the label image; and determining the graphic identification information of the unobstructed square grid in the label image based on the proportion of the remaining graphic area in the remaining grid area.

[0067] The outline of the square grid in the unoccluded area of ​​the label image can be the outline of a single square grid in that area. The outline of the graphic in the unoccluded area of ​​the label image can be the outline of the graphic contained within a single square grid in that area. Optionally, the outlines of the square grid and the graphic in the unoccluded area of ​​the label image can be represented by coordinates. The remaining area of ​​the graphic contained in the unoccluded square grid can be the remaining area of ​​the graphic contained in the unoccluded square grid within the label image. The percentage of the remaining graphic area in the remaining grid area can be used to characterize whether the unoccluded square grid contains a graphic.

[0068] Specifically, the remaining area of ​​the unoccluded square grid in the label image can be detected based on the coordinates corresponding to the outline of the square grid in the unoccluded area of ​​the label. Similarly, the remaining area of ​​the graphic in the unoccluded square grid in the label image can be detected based on the coordinates corresponding to the outline of the graphic in the unoccluded area of ​​the label. The ratio between the remaining area of ​​the graphic and the remaining area of ​​the label can be calculated to obtain the proportion of the remaining area of ​​the graphic in the remaining area of ​​the label. This proportion can be compared with a preset proportion. If the remaining area of ​​the graphic is greater than or equal to the preset proportion, it is determined that the unoccluded square grid contains a graphic; if the remaining area of ​​the graphic is less than the preset proportion, it is determined that the unoccluded square grid does not contain a graphic. This determines the graphic identification information of the unoccluded square in the label image. The preset proportion can be a pre-determined minimum proportion of the remaining area of ​​the graphic in the remaining area of ​​the label. The preset proportion can be set and adjusted by technicians based on the area of ​​the square grid and the area of ​​the graphic.

[0069] This scheme detects the remaining area of ​​unoccluded square grids in a label image based on the outline of the square grid in the unoccluded area of ​​the label. It also detects the remaining area of ​​the graphic contained in the unoccluded square grid based on the graphic outline in the unoccluded area of ​​the label. By calculating the proportion of the remaining graphic area in the remaining grid area, the scheme determines the graphic identification information of the unoccluded square grids in the label image. This improves the detection efficiency of the graphic identification information of the unoccluded square grids in the label image, thereby improving the detection efficiency of the label encoding information of the target reference label.

[0070] S103. Generate candidate grids based on the label code length and the graphic identification information of the unobstructed square grids in the label image.

[0071] The candidate grid can be a reference grid determined based on the label image. The candidate grid can be used as the basis for decoding the target reference label. The candidate grid can include at least one square grid. The candidate grid can include square grids corresponding to unoccluded square grids in the label image and square grids corresponding to occluded square grids in the label image. The graphic identification information of the unoccluded square grids in the label image is determined. The graphic identification information of the square grids corresponding to occluded square grids in the label image is uncertain. That is, the graphic identification information of each occluded square grid in the label image can include or not include graphics. To avoid interference from each occluded square grid in the label image on the decoding process of the target reference label's label encoding information, the graphic identification information of the square grids corresponding to each occluded square grid in the label image can be unified by unifying the graphic identification information of the candidate grids, thereby improving the accuracy of detecting the target reference label's label encoding information.

[0072] Specifically, based on the label code length, the number of grids in the arrangement direction and the number of grids in the movement direction can be determined to obtain the initial candidate grids. The number of grids in the arrangement direction and the number of grids in the movement direction are the same, both being the label code length. The same graphic identifier information is selected as the graphic identifier information for each occluded square grid in the label image. Based on the graphic identifier information of the unoccluded square grids in the label image and the graphic identifier information of the occluded square grids in the label image, graphic identifier information is filled into the corresponding positions of the initial candidate grids to generate the candidate grids.

[0073] S104. In the label image, for each unoccluded square grid, select multiple starting grids as search starting points in the corresponding square grids in the candidate grids. Move the target length corresponding to the target direction along the target direction in sequence. Starting from each starting grid, select the graphic identification information of the grids with a label code length to form the candidate sequence corresponding to each starting grid.

[0074] The starting grid can be the square grid in the candidate grid that serves as the search starting point for each unoccluded square grid in the label image. The target direction can be the direction of a single movement during a single search. The target direction can include the arrangement direction or the movement direction. The arrangement direction can be the arrangement direction of the square grids in the target reference label. The movement direction can be the direction of change in the order of elements in the square grids of the target reference label. The arrangement direction is perpendicular to the movement direction. This can be understood as multiple groups of square grids in the target reference label being arranged according to the arrangement direction; the order of elements in each group of square grids differs by a preset step length in the movement direction. The preset step length can be a pre-set step length for the change in the order of elements in the square grids in the movement direction of the target reference label. When decoding the target reference label, the preset step length can be a pre-set step length for a single movement in the movement direction during a single search. The preset step length can be set and adjusted based on the experience of a technician. For example, the preset step length can be the grid length of one or more square grids in the candidate grid. The target length corresponding to the arrangement direction is the length of a single square grid. The target length corresponding to the movement direction is the preset step length. This can be understood as follows: for the target reference label, the graphic identification information of the square grid reached by moving a single square grid length along the arrangement direction is the same as the graphic identification information of the directional grid reached by moving a square grid a preset step length along the movement direction. Based on this, starting from any initial grid, the graphic identification information of the square grid selected by moving a single square grid length along the arrangement direction is the same as the graphic identification information of the square grid selected by moving a preset step length along the movement direction. Therefore, during a single search movement, either the arrangement direction or the movement direction can be arbitrarily selected within the target direction.

[0075] Candidate sequences can serve as reference information for determining the tag encoding information of the target baseline label. A candidate sequence can contain graphic identifier information of the label encoding length. The graphic identifier information in the candidate sequence is arranged in search order. The number of candidate sequences can include the total number of search results for candidate sequences in the corresponding square grids of the unoccluded square grids in the label image, or a partial number of search results for candidate sequences in the corresponding square grids of the unoccluded square grids in the label image. Counting the total number of search results for candidate sequences in the corresponding square grids of the unoccluded square grids in the label image as the number of candidate sequences ensures the comprehensiveness of the candidate sequences, thereby guaranteeing the decoding accuracy of the target baseline label; however, it also increases computation time. Through experiments, technicians have found that the number of candidate sequences can be 1 / 3 of the total number of search results for candidate sequences in the corresponding square grids of the unoccluded square grids in the label image. This ensures both the decoding accuracy of the target baseline label's tag encoding information and significantly reduces computation time, thus balancing computational efficiency.

[0076] Specifically, in the label image, for each unoccluded square grid, multiple starting grids are selected as search starting points from the corresponding square grids in the candidate grids. Then, along the target direction (i.e., the arrangement direction or movement direction), each unoccluded square grid in the label image is moved sequentially by the target length corresponding to the target direction. Starting from each starting grid, the graphic identification information of grids with a label encoding length is selected to form the candidate sequence corresponding to each starting grid.

[0077] S105. Compare the candidate sequences corresponding to each starting grid and determine the cyclic code that appears most frequently as the tag encoding information of the target reference tag.

[0078] The tag encoding information of the target reference label can serve as a unique identifier for the target reference label. The tag encoding information can be represented in the form of numbers, letters, strings, or text. The tag encoding information can be pre-associated with the coordinates of the positioning point. The tag encoding information can be a cyclic sequence (or cyclic code). For example, for each target reference label containing an n×n square grid, it can be decoded into a cyclic sequence (or cyclic code) with a tag encoding length of n. The tag encoding information has periodicity, rotation invariance, and no starting point. Periodicity can be understood as the elements of the sequence automatically repeating after a certain number of elements. Rotation invariance can be understood as the new cyclic sequence being equivalent to the original cyclic sequence after a cyclic shift of elements. No starting point can be understood as the cyclic sequence having no beginning or end; any element can serve as the starting point. Therefore, multiple cyclic codes generated in the order of the cyclic sequence, using any element as the starting point, are equivalent. For example, the tag encoding information can be [1,1,0,1,0,0,1]. Equivalent cyclic sequences could include [1,0,1,0,0,1,1], [0,1,0,0,1,1,1], [1,0,0,1,1,1,0], [0,0,1,1,1,0,1], [0,1,1,1,0,1,0], and [1,1,1,0,1,0,0]. For example, Figure 2 This is a schematic diagram of 18 types of 7×7 target reference labels. (For example...) Figure 2 As shown, aoctag_1 to aoctag_18 are 18 target reference tags with different tag encoding information. The tag encoding information for aoctag_1 can be [0,0,1,0,0,0,0], [0,0,0,1,0,0,0], [0,0,0,0,1,0,0], [0,0,0,0,0,1,0], [0,0,0,0,0,0,1], or [1,0,0,0,0,0,0]. The tag encoding information for aoctag_2 to aoctag_18 is determined in the same way as that for aoctag_1, and will not be elaborated here. Therefore, when counting cyclic codes, the candidate sequence and the cyclic sequence equivalent to the candidate sequence can be counted as the same cyclic code. Either the candidate sequence or the cyclic sequence equivalent to the candidate sequence can be chosen as the content of the cyclic code. Therefore, based on the correlation between the graphic identification information of the square grid of the target reference label, the target reference label can be decoded according to the graphic identification information of the unoccluded area of ​​the label in the target reference label, while ensuring the recognition accuracy of the label encoding information of the target reference label.

[0079] Specifically, the candidate sequences corresponding to each starting grid are compared, and the cyclic codes corresponding to each candidate sequence are counted. The cyclic code that appears most frequently is determined as the tag encoding information of the target reference label.

[0080] In an optional embodiment of the present invention, before comparing the candidate sequences corresponding to each starting grid and determining the cyclic code that appears most frequently as the tag encoding information of the target reference label, the method further includes: filtering and eliminating candidate sequences in the candidate sequences corresponding to each starting grid where all elements are the same.

[0081] In order to improve the recognition accuracy of the tag encoding information of the target reference tag when generating the tag encoding information, there must be at least one element in the tag encoding information that is different from the other elements.

[0082] Specifically, before comparing the candidate sequences corresponding to each starting grid and determining the cyclic code that appears most frequently as the tag encoding information of the target reference label, candidate sequences with identical elements can be filtered and eliminated from the candidate sequences corresponding to each starting grid.

[0083] This scheme avoids interference from unoccluded square grids at the edges of unoccluded areas of the label in the label image during a single search by filtering and eliminating candidate sequences with identical elements from the candidate sequences corresponding to each starting grid. This further improves the accuracy of label encoding information detection for the target reference label.

[0084] Existing reference labels can be categorized into square reference labels and round reference labels based on their design shape. For example, Figures 3 to 6 As shown, the square reference tags include ARToolKit (Augmented Reality Toolkit tag), ARTag (Augmented Reality tag), AprilTag (April tag), and ArUco (tag-based Augmented Reality technology tag), etc.; Figures 7 to 8 As shown, circular benchmark labels include RuneTag (Chinese text label) and CCTag (China Computer Federation label), etc. Existing benchmark tables, in terms of design principles, can include benchmark labels using topological information and benchmark labels based on deep learning. For example, Figure 9 As shown, the baseline label for using topological information can be a TopoTag (geographic label); such as Figure 10As shown, the benchmark label based on deep learning can be DeepTag (deep learning label). Existing benchmark labeling systems have limited capabilities in handling occlusion. In occlusion experiments with different labels, most benchmark labels cannot be detected when the occlusion level reaches 10%. Only AprilTag, ArUco (tag-based augmented reality label), TopoTag (geographic label), DeepTag (deep learning label), and RuneTag (Chinese text label) can be detected at occlusion levels of 10%. Specifically, AprilTag, ArUco, TopoTag, and DeepTag can be detected at occlusion levels below 20%, while RuneTag (Chinese text label) can be detected at occlusion levels below 30%. Therefore, most benchmark labels perform poorly in resisting occlusion, and there is an urgent need to design a benchmark label that can be accurately identified even under high occlusion levels.

[0085] The technical solution of this invention obtains the label image and label encoding length of the target reference label. Based on the unoccluded area of ​​the label in the label image, it determines the graphic identifier information of the unoccluded square grids in the label image. Based on the label encoding length and the graphic identifier information of the unoccluded square grids in the label image, it generates candidate grids. This achieves the conversion of image information of the label image into graphic identifier information. Because the graphic identifier information of the square grids in the candidate grids is the same as that of the square grids corresponding to each occluded square grid in the label image, interference from the occluded square grids in the candidate grids on the decoding process of the target reference label based on the label image is avoided. Multiple starting grids are selected from the square grids corresponding to each unoccluded square grid in the label image in the candidate grids as search starting points. The target length corresponding to the target direction is moved sequentially along the target direction. Starting from each starting grid, a label encoding length is selected... The graphic identification information of the grid forms a candidate sequence corresponding to each starting grid. The target direction includes mutually perpendicular arrangement directions or movement directions. The target length corresponding to the arrangement direction is the length of a single square grid, and the target length corresponding to the movement direction is a preset step length. Based on the graphic identification information of the square grids corresponding to each unoccluded square grid in the label image in the candidate grid, and the correlation between the graphic identification information of square grids in the target reference label, the decoding and detection of the label encoding information of the target reference label based on the label image with occlusion areas is realized, which improves the detection and recognition rate of the target reference label. By comparing the candidate sequences corresponding to each starting grid, the cyclic code that appears most frequently is determined as the label encoding information of the target reference label, which further improves the accuracy of the detection of the label encoding information of the target reference label and realizes accurate identification of the reference label under the condition of high occlusion level.

[0086] Example 2

[0087] Figure 11 This is a flowchart of a reference label detection method provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment adds the following steps before acquiring the label image and label encoding length of the target reference label: "acquiring the label encoding length and label encoding information; generating an initial grid based on the label encoding length; determining graphic identifier information for each label encoding length based on the label encoding information and generating a first sequence; arranging the first sequences for each label encoding length in the arrangement direction, arranging the elements in each first sequence in the movement direction, and adjusting the order of elements in at least one first sequence according to a preset step length to generate graphic identifier information corresponding to each first sequence; drawing graphics in the square grids at corresponding positions in the initial grid based on the graphic identifier information corresponding to each first sequence to generate the target reference label." This ensures that the graphic identifier information between the square grids in the arrangement and movement directions of the generated target reference label is correlated, thereby achieving accurate identification of the target reference label even under high occlusion levels. It should be noted that parts not detailed in this embodiment can be found in the descriptions of other embodiments.

[0088] See Figure 11 The benchmark label detection method shown includes:

[0089] S1101. Obtain the label encoding length and label encoding information.

[0090] The tag encoding length can be the length of the tag encoding information. Optionally, increasing the tag encoding length can provide a greater number of selectable tag encoding information. For example, Figure 12 This is a schematic diagram of an 11×11 target reference label. (For example...) Figure 12As shown, the tag encoding information of the target reference tag can be a cyclic sequence of [1,1,0,0,1,1,0,1,0,0,0]. The tag encoding length of the target reference tag is 11 bits. Compared with a tag encoding length of 7 bits, increasing the tag encoding length provides a greater number of selectable tag encoding information. The tag encoding length can be set and adjusted by technicians according to the application scenario of the robot and the computing efficiency of the computing device. It can be understood that the number of reference tags required in different application scenarios is different. In order to ensure the uniqueness of the tag encoding information of the reference tag, the minimum value of the tag encoding length of the reference tag is also different in different application scenarios. Increasing the tag encoding length of the reference tag does not significantly affect the recognition accuracy of the reference tag. However, in actual experiments, it was found that increasing the tag encoding length leads to an increase in the detection time of the computing device for the reference tag. Therefore, when using computing devices with different computing efficiencies to detect the reference tag, the maximum value of the tag encoding length of the reference tag is also different. Optionally, in the same application scenario, such as a fruit picking scenario, the tag encoding length of each reference tag is the same. Tag encoding information can be used to uniquely identify target reference tags. Tag encoding information can be represented in the form of numbers, letters, strings, or text. Tag encoding information can be pre-associated with the coordinates of positioning points to achieve robot-assisted positioning based on the tag encoding information. Tag encoding information can be a cyclic sequence (or cyclic code). Tag encoding information has the characteristics of periodicity, rotation invariance, and no starting point. Periodicity means that after a certain number of elements, the elements of the sequence will automatically repeat. Rotation invariance means that if elements in the cyclic sequence are cyclically shifted, the resulting new cyclic sequence is equivalent to the original cyclic sequence. No starting point means that the cyclic sequence has no beginning or end; any element can serve as the starting point of the cyclic sequence.

[0091] Specifically, it is possible to obtain the label code length and label code information pre-set by the technicians.

[0092] S1102. Generate an initial grid based on the label encoding length.

[0093] The initial grid can be the grid before the graphic identification information of the undetermined square grid is established. The initial grid is square in shape as a whole, and contains at least one square grid within it. The size and number of grids of the initial grid are the same as those of the target reference label, but the square grids in the initial grid do not contain graphics; that is, the graphics of the square grids in the initial grid are all empty.

[0094] Specifically, the number of grid cells in the arrangement direction and the number of grid cells in the movement direction can be determined based on the tag code length to obtain the initial grid. The number of grid cells in the arrangement direction and the number of grid cells in the movement direction are the same, both being the tag code length.

[0095] S1103. Based on the label encoding information, determine the graphic identification information of the label encoding length and generate the first sequence.

[0096] Graphical identifier information is used to identify whether a corresponding grid contains graphics. Optionally, the graphic identifier information can be represented in the form of numbers, letters, strings, or text. For example, the graphic identifier information can be 0 or 1; where 0 can be used to indicate that the corresponding grid does not contain graphics; and 1 can be used to indicate that the corresponding grid contains graphics.

[0097] The first sequence can be used to equivalently identify the tag encoding information. The order of each element in the first sequence corresponds to the order of the graphic identification information corresponding to the tag encoding information. It can be understood that the order of each element in the first sequence can be the same as the order of the graphic identification information corresponding to the tag encoding information; the order of each element in the first sequence can also be cyclically equivalent to the order of the graphic identification information corresponding to the tag encoding information. For example, the tag encoding information can be [1,1,0,1,0,0,1]. The first sequence can be [1,1,0,1,0,0,1], [1,0,1,0,0,1,1], [0,1,0,0,1,1,1], [1,0,0,1,1,1,0], [0,0,1,1,1,0,1], [0,1,1,1,0,1,0] or [1,1,1,0,1,0,0].

[0098] Optionally, the label code length and the number of graphic identifiers can be determined according to the label encoding information. Alternatively, the corresponding graphic identifiers can be selected sequentially from the graphic identifiers corresponding to the label encoding information to generate the first sequence.

[0099] Optionally, any bit can be selected as the starting bit from the label encoding information, and equivalent encoding information can be generated according to the order of elements in the label encoding information. The label encoding length and the number of graphic identifiers can be determined according to the equivalent encoding information. The first sequence can be generated by sequentially selecting the corresponding graphic identifiers from the graphic identifiers corresponding to the equivalent encoding information.

[0100] S1104. Arrange the first sequence of label encoding length in the arrangement direction, arrange the elements in each first sequence in the movement direction, and adjust the order of the elements in at least one first sequence according to a preset step length to generate graphic identification information corresponding to each first sequence.

[0101] The arrangement direction can be the arrangement direction between each first sequence. The movement direction can be the arrangement direction of the elements in the first sequence. The arrangement direction and the movement direction are perpendicular. The preset step length can be the step length for changing the order of elements in the square grid in the movement direction of the pre-set target reference label. During the generation of the target reference label, the preset step length can be the step length corresponding to the number of order adjustment positions of the elements in the pre-set first sequence. Specifically, the order of elements between two adjacent first sequences in the arrangement direction differs by the number of order adjustment positions corresponding to the preset step length in the movement direction. Optionally, when adjusting the order of elements in at least one first sequence, the preset step length can be equivalent to the number of square grids. For example, in the target reference label, the preset step length can be the length of two square grids; when adjusting the order of elements in the first sequence, the order of elements in the first sequence can be adjusted by two positions in the movement direction.

[0102] Specifically, the label encoding length of the first sequence can be arranged in the arrangement direction. Elements in each first sequence can be arranged in the movement direction. Furthermore, the number of order adjustment bits corresponding to the preset step length can be determined. The order of elements in each first sequence can be adjusted sequentially according to the arrangement direction, corresponding to the preset step length, to generate graphic identification information for each first sequence.

[0103] In an optional embodiment of the present invention, the order of elements in at least one first sequence is adjusted according to a preset step length to generate graphic identification information corresponding to each first sequence, including: cyclically shifting and adjusting the order of elements in at least one first sequence according to a preset step length to generate graphic identification information corresponding to each first sequence.

[0104] The target reference labels generated using the cyclic shift adjustment method satisfy the following rule: For a square grid at any position in the target reference label, the square grid reached by moving it a single grid length in the arrangement direction is the same as the square grid reached by moving it a preset step length in the movement direction. For cases exceeding the edge of the first sequence (equivalent to exceeding the initial grid), a cyclic query can be performed in either the arrangement or movement direction. For example, if the first grid arrangement order is from bottom to top, for cases exceeding the edge of the first sequence, the query can be performed at the bottom of the first sequence; similarly, if the second grid arrangement order is from left to right, for cases exceeding the edge of the first sequence, the query can be performed at the leftmost position of the first sequence. This rule also applies to other cases where the arrangement or movement direction is different.

[0105] During cyclic shift adjustment, for a single first sequence, the order of elements in adjacent first sequences along the arrangement direction is adjusted by moving the elements of the first sequence by a preset step length. For portions exceeding the edge of the first sequence, they are moved to the side where the first sequence is empty, according to the order of the elements in the first sequence. For example, if the first sequence is a single row of elements; the preset step length is the length of one square grid; and the movement direction is from left to right; for a single row of first sequences, the order of elements in the first sequence of the row below can be adjusted by moving the entire first sequence of the row above one position to the right, and moving the excess portion to the leftmost side of that row according to the order of the elements in the first sequence. Similarly, if the first sequence is a single column of elements; the preset step length is the length of one square grid; and the movement direction is from top to bottom; for a single column of first sequences, the order of elements in the first sequence of the column to its right can be adjusted by moving the entire first sequence of the column to its left one position down, and moving the excess portion to the top of that column according to the order of the elements in the first sequence.

[0106] Specifically, the number of positions to be adjusted can be determined according to a preset step length. The order of elements in each first sequence on the first grid arrangement can be shifted and adjusted sequentially according to the preset step length. For a single first sequence, for elements that have shifted beyond the first sequence, the empty positions in each first sequence are filled according to the original order of the elements, generating the corresponding graphic identifier information for each first sequence.

[0107] For example, the label encoding length can be 7; the first sequence can be [1,1,0,1,0,0,1]; the preset step length can be the length of one square grid. Seven first sequences can be arranged in the arrangement direction. Elements in each first sequence can be arranged in the movement direction. The preset step length can be used to determine the number of order adjustment bits corresponding to that preset step length, i.e., 1 bit. The order of elements in each first sequence in the arrangement direction can be shifted and adjusted sequentially according to the preset step length, i.e., 1 bit. For a single first sequence, for elements that have shifted beyond the first sequence, the empty positions in each first sequence are filled according to the original order of the elements in the first sequence, generating the graphic identification information corresponding to each first sequence.

[0108] For example, the graphical identifier information corresponding to each first sequence can be represented in the following way:

[0109]

[0110] The arrangement direction is from top to bottom; the movement direction is from left to right; the preset step length corresponds to 1 digit of the order adjustment; the original first sequence's graphic identifier information is [1,1,0,1,0,0,1]; the graphic identifier information of the first sequence adjacent to the original first sequence in the arrangement direction is [1,1,1,0,1,0,0]. That is, according to the preset step length corresponding to 1 digit of the adjustment, the elements in the first sequence are moved sequentially in the movement direction, and the empty positions in the first sequence are filled in according to the order of the elements in the original first sequence, resulting in the graphic identifier information of the first sequence adjacent to the original first sequence in the arrangement direction, which is [1,1,1,0,1,0,0]. The process of determining the graphic identifier information of other first sequences in the arrangement direction is similar to that of the first sequences adjacent to the original first sequence in the arrangement direction, and will not be elaborated here. Thus, the graphic identifier information of each first sequence is generated.

[0111] This solution introduces a method that cyclically shifts and adjusts the order of elements in at least one first sequence according to a preset step length to generate graphic identification information corresponding to each first sequence. Compared with the order adjustment, the cyclic shift adjustment method is more convenient and further improves the recognition efficiency of the target reference label.

[0112] S1105. In the initial grid, according to the graphic identification information corresponding to each first sequence, draw the graphic in the square grid at the corresponding position to generate the target reference label.

[0113] Each first sequence is arranged in the order of its arrangement direction; each element in the first sequence is arranged in the order of its movement direction. The graphic identification information corresponding to each first sequence corresponds to the position of each square grid in the initial grid. Therefore, based on the graphic identification information corresponding to each first sequence, the graphic to be drawn in the corresponding square grid in the initial grid can be determined, thus accurately generating the target reference label. The target reference label may include a square grid and the graphic drawn in the square grid. Whether the square grid of the reference label contains a graphic can be determined according to the graphic identification information corresponding to the first sequence. Optionally, the graphic contained in the square grid may include hollow or solid graphics; the shape of the graphic contained in the square grid may include square, circle, or star, etc.; the color of the graphic contained in the square grid may be black, which is not limited in this solution. Preferably, the graphic contained in the square grid may be a black solid graphic. Optionally, the area occupied by the graphic in the square grid may be greater than or equal to 50% of the area of ​​the square grid.

[0114] Specifically, based on the graphic identification information corresponding to each first sequence, the graphic corresponding to the graphic identification information can be drawn in the square grid at the corresponding position in the initial grid to generate the target reference label.

[0115] In an optional embodiment of the present invention, after generating the target reference label, the method further includes: obtaining adjustment information of the target reference label; and adjusting the target reference label according to the adjustment information of the target reference label.

[0116] The adjustment information for the target reference label can be used to adjust the target reference label. This adjustment information includes the grid line width of the square grid, the shape of the graphic, and / or the area of ​​the graphic.

[0117] Specifically, after generating the target baseline label, the adjustment information of the target baseline label can be obtained. Based on this adjustment information, the target baseline label can be adjusted accordingly.

[0118] For example, the label encoding length of the target reference label can be 7; the target reference label can be a 7×7 square grid consisting of 8 horizontal lines and 8 vertical lines, and a black solid graphic existing inside the square grid arranged in a certain encoding order. Figure 13 This is a schematic diagram of the target reference label for the width of the first square grid; Figure 14 This is a schematic diagram of the target reference label for the width of the second square grid. The width of the first square grid is smaller than the width of the second square grid. Figure 15 It is a schematic diagram of a target reference label containing a square graphic; Figure 16 It is a schematic diagram of a target reference label containing a circular graphic. Figure 17 This is a schematic diagram of the target baseline label representing the area of ​​the first graphic; Figure 18 This is a schematic diagram of the target baseline label representing the area of ​​the first graphic. The area of ​​the first graphic is smaller than the area of ​​the second graphic.

[0119] This solution improves the accuracy of target reference label encoding information detection by obtaining the adjustment information of the target reference label after it is generated, and then adjusting the target reference label according to the adjustment information.

[0120] S1106. Obtain the label image of the target reference label and the label encoding length of the target reference label.

[0121] The label image contains areas where the label is obscured.

[0122] S1107. Based on the unoccluded area of ​​the label in the label image, determine the graphic identification information of the unoccluded square grid in the label image.

[0123] S1108. Generate candidate grids based on the label code length and the graphic identification information of the unobstructed square grids in the label image.

[0124] The graphic identification information of the square grids in the candidate grids is the same as that of the square grids corresponding to each occluded square grid in the label image.

[0125] S1109. In the label image, for each unoccluded square grid, select multiple starting grids as search starting points in the corresponding square grids in the candidate grids. Move the target length corresponding to the target direction along the target direction in sequence. Starting from each starting grid, select the graphic identification information of the grids with a label code length to form the candidate sequence corresponding to each starting grid.

[0126] The target direction includes either a mutually perpendicular arrangement direction or a movement direction. The target length corresponding to the arrangement direction is the length of a single square grid. The target length corresponding to the movement direction is the preset step length.

[0127] S1110. Compare the candidate sequences corresponding to each starting grid and determine the cyclic code that appears most frequently as the tag encoding information of the target reference label.

[0128] The technical solution of this invention, before acquiring the label image and label encoding length of the target reference label, obtains the label encoding length and label encoding information. Based on the label encoding length, an initial grid is generated. Based on the label encoding information, graphic identification information of the label encoding length is determined and a first sequence is generated. The first sequence of the label encoding length is arranged in the arrangement direction, and the elements in each first sequence are arranged in the movement direction. The order of the elements in at least one first sequence is adjusted according to a preset step length to generate graphic identification information corresponding to each first sequence. In the initial grid, based on the graphic identification information corresponding to each first sequence, a graphic is drawn in the square grid at the corresponding position to generate the target reference label. This makes the graphic identification information between the square grids of the generated target reference label correlated, thereby achieving accurate identification of the target reference label under high occlusion levels.

[0129] Figure 19 This is a flowchart of a reference label detection method provided for Embodiment 2 of the present invention. Figure 19 The benchmark label detection method shown is a preferred embodiment of the present invention.

[0130] See Figure 19 The method shown includes:

[0131] S1901. Obtain the label image of the target reference label.

[0132] Among them, there are label occlusion areas in the label image of the target reference label.

[0133] S1902. Detect the square grid outline and graphic outline in the unoccluded area of ​​the label in the label image.

[0134] S1903. Calculate the remaining area of ​​the unobstructed square grids in the label image, and remove unobstructed square grids whose remaining area is less than the preset remaining area.

[0135] S1904. Based on the outline of the square grid and the outline of the graphic in the unoccluded area of ​​the label in the label image, calculate the proportion of the remaining area of ​​the graphic in the remaining area of ​​the grid, determine whether the unoccluded square grid in the label image contains a graphic, and determine the graphic identification information of the unoccluded square grid in the label image.

[0136] The graphic can be a black graphic.

[0137] S1905. Save the graphic identification information and center point coordinates of the unobstructed square grid in the label image.

[0138] The graphic identification information is used to identify whether a graphic is contained within a square grid. The coordinates of the graphic's center point are determined by the remaining square grid. For example, the remaining square grid can be a quadrilateral; the graphic's center point can be obtained by connecting the midpoints of the four sides of the quadrilateral diagonally. It can be understood that the graphic's center point coordinates are not the true center or centroid of the square grid, but rather an approximate point that roughly represents the position of the square grid.

[0139] S1906. Generate candidate grids based on the label code length and the graphic identification information of the unobstructed square grids in the label image.

[0140] Essentially, the candidate grid is formed by converting the image information of the label image into graphic identifier information for each square grid in the candidate grid. The graphic identifier information of the square grids in the candidate grid is the same as that of the square grids corresponding to each occluded square grid in the label image.

[0141] Specifically, the label image can be completed based on the unoccluded square grid in the label image, using the label encoding length. The graphic identifier information of the square grid corresponding to the unoccluded square grid in the label image can be determined based on the graphic identifier information and center point coordinates of the graphic identifier in the label image. Alternatively, the graphic identifier information can be completed based on the graphic identifier information of each occluded square grid in the label image to generate candidate grids.

[0142] For example, the target reference label can be a 7×7 reference label. A 7×7 two-dimensional array (i.e., a candidate grid without filled graphic identification information) can be created based on the length of the target reference label. For each square grid in the two-dimensional array, if the graphic identification information of the corresponding unoccluded square grid in the label image is 1, then the graphic identification information of that square grid is 1; otherwise, the graphic identification information of that square grid is 0. The corresponding positions in the two-dimensional array are filled with the graphic identification information of each unoccluded square grid in the label image; the graphic identification information of the occluded square grids in the label image is set to 0, and the two-dimensional array is padded. This generates the candidate grid.

[0143] 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 1 0 0 0 1 1 1 1 1 0 0 0 1 1 1 1 1 0 0 0 1

[0144] This method yields a two-dimensional array (i.e., candidate grid) containing 0 and 1 information. Each bit of the graphic identifier in the two-dimensional array can be used to indicate whether a solid black graphic exists in the corresponding square grid in the label image.

[0145] S1907. In the label image, for each unoccluded square grid, select multiple starting grids as search starting points in the corresponding square grids of the candidate grids. Move the target length corresponding to the target direction along the target direction in sequence. Starting from each starting grid, select the graphic identification information of the grids with a label encoding length to form a candidate sequence corresponding to each starting grid. Compare the candidate sequences corresponding to each starting grid and determine the cyclic code that appears most frequently as the label encoding information of the target reference label.

[0146] For example, the tag encoding information of an n×n target reference label can be represented by a cyclic sequence of tag encoding length n. Therefore, for the target reference label, this two-dimensional array (i.e., the candidate grid) is not the final decoding result of the target reference label. In fact, it is necessary to further decode this two-dimensional array to determine the tag encoding information of the target reference label, thereby obtaining the decoding result of the target reference label. In order to calculate the decoding result of this two-dimensional array (i.e., the candidate grid), it is necessary to count the most frequently occurring subarray (i.e., the candidate sequence) of length n in the two-dimensional array (i.e., the candidate grid).

[0147] The following calculation can be performed on all detected bits (excluding padded bits) in the n×n two-dimensional array (i.e., the square grids in the candidate grids corresponding to the unoccluded square grids in the label image):

[0148] Step 1: Create an empty cyclic sequence B.

[0149] Step 2: In the label image, select an initial position (i.e., the search starting point) from the corresponding square grid in each unoccluded square grid in the candidate grid, denoted as p.ij .

[0150] Where i is the row number of the two-dimensional array; the bottom row is row 0; the top row is row (n-1); j is the column number of the two-dimensional array, with the leftmost column being column 0 and the rightmost column being column (n-1). The code for this position (i.e., the selected graphic identifier information) is b0.

[0151] Step 3: Add element b0 to the cyclic sequence B, so that B = [b0].

[0152] Step 4: Assign the value i = (i-1) mod n, or assign the value j = (j+1) mod n to obtain the code of the new position (i.e. the selected graphic identifier information) b1.

[0153] The arrangement direction is from top to bottom; the movement direction is from left to right; and the preset step length is the length of a single square grid. In this step, the probability of selecting the arrangement direction as the target direction is equal to the probability of selecting the movement direction; the mod operation is a modulo operation.

[0154] Step 5: Check if the new bit is the detected bit. If it is, add element b1 to the circular sequence B. At this point, B = [b0, b1].

[0155] Step 6: Repeat steps 4 and 5 until the length of the cyclic sequence B reaches the label encoding length n. At this point, B = [b0,b1,b2,b3,b4,b5,b6]. Record the cyclic sequence B as a valid result (i.e., a candidate sequence).

[0156] Step 7: Repeat steps 1 to 6 until enough valid results are collected. Statistically analyze these results and take the most frequent cyclic sequence (i.e., cyclic code) as the decoding result (i.e., tag encoding information) of the target reference tag.

[0157] Counting all valid results from a target baseline label ensures decoding accuracy but increases computation time. Experimental results show that even counting only about one-third of the randomly selected valid results guarantees decoding accuracy and significantly reduces computation time. The specific number of valid results to count for the target baseline label should be determined based on the actual application scenario.

[0158] After going through steps 1 to 7 above, a cyclic sequence of length n can be obtained. This cyclic sequence is determined as the tag encoding information of the target reference label of the occluded image.

[0159] For example, Figure 20 This is a label image of a target baseline label when 57% is obscured from the lower left. The information after image decoding is:

[0160] 0 1 1 1 0 1 0 0 0 1 1 1 0 1 0 0 0 1 1 1 0 0 0 0 0 1 1 1 0 0 0 0 0 1 1 0 0 0 0 0 0 1

[0161] The final target baseline label encoding information is [1,1,1,0,1,0,0,], which is correct.

[0162] For example, Figure 21 This is a label image of a target reference label when it is 78% obscured from directly above and below. The information after image decoding is:

[0163] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 1 1 1 1 0 0 0 1

[0164] The final target baseline label encoding information is [1,1,0,0,0,1,1,], which is correct.

[0165] For example, Figure 22 This is a schematic diagram of a target reference label containing a circular graphic. Figure 23 This is the label image of the target baseline label when 78% is obscured from the upper left and lower right. The information after image decoding is:

[0166] 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 0 0 0 1 1 0 0 1 1 0 0 1

[0167] The final target baseline label encoding information is [1,1,0,0,1,1,0,], which is correct.

[0168] The above technical solutions improve the recognition rate of target reference tags, thereby improving the positioning accuracy of assisted positioning based on target reference tags, which can provide a more reliable and efficient collaborative foundation for multi-robot systems.

[0169] Example 3

[0170] Figure 24 This is a schematic diagram of a reference label detection device provided in Embodiment 3 of the present invention. This embodiment of the present invention is applicable to reference label detection situations. The device can execute a reference label detection method and can be implemented in hardware and / or software. The device can be configured in an electronic device that carries the reference label detection function, such as a robot.

[0171] See Figure 24The reference label detection device shown includes: a label image acquisition module 2401, a graphic label information determination module 2402, a candidate grid generation module 2403, a candidate sequence generation module 2404, and a label encoding information determination module 2405. The label image acquisition module 2401 is used to acquire the label image of the target reference label and the label encoding length of the target reference label; wherein, the label image contains label occlusion areas; the graphic label information determination module 2402 is used to determine the graphic identification information of the unoccluded square grids in the label image based on the unoccluded areas of the label; the candidate grid generation module 2403 is used to generate candidate grids based on the label encoding length and the graphic identification information of the unoccluded square grids in the label image; the graphic identification information of the square grids in the candidate grids is the same as that of the square grids corresponding to each occluded square grid in the label image; the candidate sequence generation module 2404 is used to determine the graphic identification information of each unoccluded square grid in the label image. In the candidate square grid, multiple starting grids are selected as search starting points. Moving along the target direction, the target length corresponding to the target direction is moved sequentially. Starting from each starting grid, the graphic identification information of grids with a label encoding length is selected to form a candidate sequence corresponding to each starting grid. The target direction includes mutually perpendicular arrangement directions or movement directions. The target length corresponding to the arrangement direction is the length of a single square grid. The target length corresponding to the movement direction is a preset step length. The label encoding information determination module 2405 compares the candidate sequences corresponding to each starting grid and determines the most frequently occurring cyclic code as the label encoding information of the target reference label.

[0172] The technical solution of this invention obtains the label image and label encoding length of the target reference label. Based on the unoccluded area of ​​the label in the label image, it determines the graphic identifier information of the unoccluded square grids in the label image. Based on the label encoding length and the graphic identifier information of the unoccluded square grids in the label image, it generates candidate grids. This achieves the conversion of image information of the label image into graphic identifier information. Because the graphic identifier information of the square grids in the candidate grids is the same as that of the square grids corresponding to each occluded square grid in the label image, interference from the occluded square grids in the candidate grids on the decoding process of the target reference label based on the label image is avoided. Multiple starting grids are selected from the square grids corresponding to each unoccluded square grid in the label image in the candidate grids as search starting points. The target length corresponding to the target direction is moved sequentially along the target direction. Starting from each starting grid, a label encoding length is selected... The graphic identification information of the grid forms a candidate sequence corresponding to each starting grid. The target direction includes mutually perpendicular arrangement directions or movement directions. The target length corresponding to the arrangement direction is the length of a single square grid, and the target length corresponding to the movement direction is a preset step length. Based on the graphic identification information of the square grids corresponding to each unoccluded square grid in the label image in the candidate grid, and the correlation between the graphic identification information of square grids in the target reference label, the decoding and detection of the label encoding information of the target reference label based on the label image with occlusion areas is realized, which improves the detection and recognition rate of the target reference label. By comparing the candidate sequences corresponding to each starting grid, the cyclic code that appears most frequently is determined as the label encoding information of the target reference label, which further improves the accuracy of the detection of the label encoding information of the target reference label and realizes accurate identification of the reference label under the condition of high occlusion level.

[0173] In an optional embodiment of the present invention, the device further includes: a label encoding length acquisition module, configured to acquire a label encoding length and label encoding information before acquiring the label image of the target reference label and the label encoding length of the target reference label; an initial network generation module, configured to generate an initial grid according to the label encoding length; a first sequence generation module, configured to determine graphic identification information of the label encoding length according to the label encoding information and generate a first sequence; the graphic identification information is used to identify whether the corresponding grid contains a graphic; a graphic identification information generation module, configured to arrange the first sequences of the label encoding length in the arrangement direction, arrange the elements in each first sequence in the movement direction, and adjust the order of at least one element in the first sequence according to a preset step length to generate graphic identification information corresponding to each first sequence; the arrangement direction is perpendicular to the movement direction; the order of elements between two adjacent first sequences in the arrangement direction differs by a preset step length corresponding to the order adjustment number in the movement direction; and a target reference label generation module, configured to draw a graphic in a square grid at a corresponding position in the initial grid according to the graphic identification information corresponding to each first sequence to generate a target reference label.

[0174] In an optional embodiment of the present invention, the graphic identification information generation module includes: a cyclic shift adjustment unit, configured to perform cyclic shift adjustment on the order of elements in at least one first sequence according to a preset step length, to generate graphic identification information corresponding to each first sequence; wherein the order of elements between two adjacent first sequences in the arrangement direction differs by a preset step length corresponding to the number of order adjustment bits in the movement direction.

[0175] In an optional embodiment of the present invention, the device further includes: an adjustment information acquisition module, configured to acquire adjustment information of the target reference label after the target reference label is generated; wherein the adjustment information includes the grid line width of the square grid, the shape of the graphic and / or the area of ​​the graphic; and a target reference label adjustment module, configured to adjust the target reference label according to the adjustment information of the target reference label.

[0176] In an optional embodiment of the present invention, the graphic identification information determination module 2402 includes: a first grid remaining area detection unit, configured to detect the remaining area of ​​the unobstructed square grid in the label image based on the unobstructed area of ​​the label in the label image; and a first graphic identification information determination unit, configured to determine the graphic identification information of the unobstructed square grid in the label image when the remaining area of ​​the grid is greater than or equal to a preset remaining area.

[0177] In an optional embodiment of the present invention, the graphic identification information determination module 2402 includes: a second grid remaining area detection unit, configured to detect the remaining area of ​​the unobstructed square grid in the label image based on the outline of the square grid in the unobstructed area of ​​the label image; a graphic remaining area determination unit, configured to detect the remaining area of ​​the graphic contained in the unobstructed square grid based on the graphic outline in the unobstructed area of ​​the label image; and a second graphic identification information determination unit, configured to determine the graphic identification information of the unobstructed square grid in the label image based on the proportion of the remaining graphic area in the remaining grid area.

[0178] In an optional embodiment of the present invention, the device further includes: a candidate sequence filtering module, configured to filter and remove candidate sequences in the candidate sequences corresponding to each of the starting grids before comparing the candidate sequences corresponding to each of the starting grids and determining the cyclic code that appears most frequently as the tag encoding information of the target reference label.

[0179] The reference label detection device provided in this embodiment of the invention can execute the reference label detection method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0180] In the technical solutions of this invention, the acquisition, storage, and application of the target reference label's label image, label encoding length, label encoding information, and adjustment information all comply with relevant laws and regulations and do not violate public order and good morals.

[0181] Example 4

[0182] Figure 25 A schematic diagram of an electronic device 2500 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0183] like Figure 25As shown, the electronic device 2500 includes at least one processor 2501 and a memory, such as a read-only memory (ROM) 2502 and a random access memory (RAM) 2503, communicatively connected to the at least one processor 2501. The memory stores computer programs executable by the at least one processor. The processor 2501 can perform various appropriate actions and processes based on the computer program stored in the ROM 2502 or loaded into the RAM 2503 from storage unit 2508. The RAM 2503 can also store various programs and data required for the operation of the electronic device 2500. The processor 2501, ROM 2502, and RAM 2503 are interconnected via a bus 2504. An input / output (I / O) interface 2505 is also connected to the bus 2504.

[0184] Multiple components in electronic device 2500 are connected to I / O interface 2505, including: input unit 2506, such as keyboard, mouse, etc.; output unit 2507, such as various types of monitors, speakers, etc.; storage unit 2508, such as disk, optical disk, etc.; and communication unit 2509, such as network card, modem, wireless transceiver, etc. Communication unit 2509 allows electronic device 2500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0185] Processor 2501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 2501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 2501 performs the various methods and processes described above, such as benchmark label detection methods.

[0186] In some embodiments, the reference tag detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 2508. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 2500 via ROM 2502 and / or communication unit 2509. When the computer program is loaded into RAM 2503 and executed by processor 2501, one or more steps of the reference tag detection method described above may be performed. Alternatively, in other embodiments, processor 2501 may be configured to perform the reference tag detection method by any other suitable means (e.g., by means of firmware).

[0187] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0188] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0189] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0190] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0191] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0192] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability.

[0193] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0194] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting a reference label, characterized in that, The method includes: Obtain the label image of the target reference label and the label encoding length of the target reference label; wherein, there is a label occlusion area in the label image; Based on the unoccluded area of ​​the label in the label image, determine the graphic identification information of the unoccluded square grid in the label image; Based on the label encoding length and the graphic identification information of the unoccluded square grids in the label image, candidate grids are generated; the graphic identification information of the square grids in the candidate grids is the same as that of the square grids corresponding to each occluded square grid in the label image. In the label image, for each unoccluded square grid in the corresponding square grid in the candidate grid, multiple starting grids are selected as search starting points. Along the target direction, the grids are moved sequentially by the target length corresponding to the target direction. Starting from each of the starting grids, the graphic identification information of the label-encoded length of grids is selected to form a candidate sequence corresponding to each of the starting grids. The target direction includes mutually perpendicular arrangement directions or movement directions. The target length corresponding to the arrangement direction is the length of a single square grid. The target length corresponding to the movement direction is a preset step length. The candidate sequences corresponding to each starting grid are compared, and the cyclic code that appears most frequently is determined as the tag encoding information of the target reference label.

2. The method according to claim 1, characterized in that, Before obtaining the tag image of the target reference tag and the tag encoding length of the target reference tag, the method further includes: Get the tag encoding length and tag encoding information; An initial grid is generated based on the label encoding length; Based on the label encoding information, determine the graphic identification information of the label encoding length and generate a first sequence; the graphic identification information is used to identify whether the corresponding grid contains a graphic. The label encoding length of the first sequence is arranged in the arrangement direction, and the elements in each first sequence are arranged in the movement direction. The order of at least one element in the first sequence is adjusted according to a preset step length to generate graphic identification information corresponding to each first sequence. The arrangement direction is perpendicular to the movement direction. The order of elements between two adjacent first sequences in the arrangement direction is adjusted by a preset step length corresponding to the number of order adjustment bits in the movement direction. In the initial grid, based on the graphic identification information corresponding to each of the first sequences, a graphic is drawn in a square grid at the corresponding position to generate a target reference label.

3. The method according to claim 2, characterized in that, The step of adjusting the order of at least one element in the first sequence according to a preset step length to generate graphic identification information corresponding to each first sequence includes: According to a preset step length, the order of at least one element in the first sequence is cyclically shifted and adjusted to generate graphic identification information corresponding to each first sequence.

4. The method according to claim 2, characterized in that, Following the generation of the target baseline label, the following is also included: Obtain the adjustment information of the target reference label; wherein, the adjustment information includes the grid line width of the square grid, the shape of the graphic and / or the area of ​​the graphic; The target reference label is adjusted according to the adjustment information of the target reference label.

5. The method according to claim 1, characterized in that, The step of determining the graphic identification information of the unobstructed square grid in the label image based on the unobstructed area of ​​the label image includes: Based on the unoccluded area of ​​the label in the label image, detect the remaining area of ​​the unoccluded square grid in the label image; When the remaining area of ​​the grid is greater than or equal to the preset remaining area, the graphic identification information of the unobstructed square grid in the label image is determined.

6. The method according to claim 1, characterized in that, The step of determining the graphic identification information of the unobstructed square grid in the label image based on the unobstructed area of ​​the label image includes: Based on the outline of the square grid in the unoccluded area of ​​the label image, detect the remaining area of ​​the unoccluded square grid in the label image; Based on the graphic outline in the unoccluded area of ​​the label image, detect the remaining area of ​​the graphic contained in the unoccluded square grid; Based on the proportion of the remaining area of ​​the graphic in the remaining area of ​​the grid, the graphic identification information of the unobstructed square grid in the label image is determined.

7. The method according to claim 1, characterized in that, Before comparing the candidate sequences corresponding to each of the starting grids and determining the cyclic code that appears most frequently as the tag encoding information of the target reference label, the method further includes: In the candidate sequences corresponding to each starting grid, candidate sequences with identical elements are filtered out.

8. A reference label detection device, characterized in that, The device includes: The label image acquisition module is used to acquire the label image of the target reference label and the label encoding length of the target reference label; wherein, the label image contains a label occlusion area; The graphic label information determination module is used to determine the graphic identification information of the unobstructed square grid in the label image based on the unobstructed area of ​​the label in the label image; The alternative grid generation module is used to generate alternative grids based on the label code length and the graphic identification information of the unoccluded square grids in the label image; the graphic identification information of the square grids in the alternative grids is the same as that of the square grids corresponding to each occluded square grid in the label image. The candidate sequence generation module is used to select multiple starting grids as search starting points from the corresponding square grids in the candidate grids for each unoccluded square grid in the label image, and sequentially move along the target direction by the target length corresponding to the target direction. Starting from each of the starting grids, it selects the graphic identification information of the label-encoded length of grids to form a candidate sequence corresponding to each of the starting grids. The target direction includes mutually perpendicular arrangement directions or movement directions. The target length corresponding to the arrangement direction is the length of a single square grid. The target length corresponding to the movement direction is a preset step length. The tag encoding information determination module is used to compare the candidate sequences corresponding to each of the starting grids and determine the cyclic code that appears most frequently as the tag encoding information of the target reference tag.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the reference tag detection method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the reference tag detection method according to any one of claims 1-7.

Images