A deep learning detection method for annular coded marks

Abstract

The application discloses a kind of annular coding mark deep learning detection method.Method includes: obtaining image to be detected;Input training completed coding mark detection network, output candidate target confidence, boundary box regression parameter and N-dimensional binary coding probability vector;Probability vector is binarized, and N-bit binary code is obtained;Encoding is converted into minimum cyclic shift value and is mapped as class number;Candidate target center coordinates are spatially clustered;The class number appearance frequency in each cluster is counted, and the highest frequency class is determined as the final identity identification;The boundary box parameters of candidate target belonging to the identification are averaged to obtain the final boundary box.The application uses binary coding structure prior setting efficient and simple binary vector coding, simultaneously introduces cyclic perception dynamic label matching strategy during network training, combines clustering and voting fusion used during network inference, improves the recognition accuracy and positioning stability under large-scale code library.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a deep learning method for detecting ring-coded markers. Background Technology

[0002] Circular coded markers are widely used in photogrammetry, camera calibration, large-scale 3D reconstruction, and industrial vision measurement due to their high positioning accuracy, flexible deployment, and unique identification. In practical engineering applications, markers often face complex imaging conditions such as varying lighting, motion blur, and cluttered backgrounds, thus requiring reliable and robust detection and recognition methods. Traditional methods rely on manually designed features and heuristic rules, resulting in cumbersome algorithm processes, limited adaptability to complex imaging conditions, low recognition recall, and processing efficiency that fails to meet the demands of large-scale applications.

[0003] In recent years, deep learning technology has been introduced into the field of coded marker detection in order to improve the robustness of detection. However, existing deep learning methods are mostly limited to small-scale marker sets and are difficult to extend to large-scale scenarios containing hundreds or thousands of unique identifiers. Specifically, existing methods generally suffer from the following shortcomings: First, the visual features of different markers in large-scale code libraries are highly similar, and fine-grained differences are difficult to distinguish. Traditional deep learning methods often use general one-hot encoding or embedding learning to represent marker categories, without utilizing the prior binary structure of ring encoding, resulting in a significant drop in recognition accuracy in large code libraries of hundreds to thousands of categories. Second, ring-coded markers have cyclic equivalence properties, meaning that the same physical marker can correspond to multiple different binary sequences depending on the starting position of the reading. Traditional fixed label or minimum encoding strategies introduce label ambiguity and sample distribution shifts during supervised training, affecting the stability and generalization ability of model training. Third, the acquisition of large-scale real-labeled data is costly, and there is a domain shift between purely synthetic data and the real imaging distribution, resulting in insufficient generalization of the model in actual deployment.

[0004] In summary, existing deep learning-based ring coding marker detection methods face core technical bottlenecks such as low recognition accuracy, unstable training, and difficulty in data acquisition when dealing with large-scale code library scenarios. There is an urgent need for a robust detection scheme that can explicitly utilize the prior of the ring coding structure, effectively handle the cyclic equivalence property, and take into account both data scale and authenticity. Summary of the Invention

[0005] The technical problem this invention aims to solve is that existing deep learning-based circular coding marker detection methods, when using one-hot coding or embedded coding for network training with large-scale code libraries, suffer from sparsity in the feature space and insufficient inter-class discriminability due to the rapid increase in codeword dimensionality. This makes it difficult for the network to learn effective discriminative features, and the recognition accuracy degrades significantly with the increase in code library size. This invention introduces a binary coding method based on the prior structure of the marker and proposes a cyclic-aware dynamic label matching strategy during network training. This solves the problem of identity ambiguity caused by the cyclic equivalence property leading to multiple binary representations of the same marker, thereby improving network training performance and marker recognition capability.

[0006] This invention provides a deep learning method for detecting ring-coded markers, comprising the following steps: Acquire the image to be detected; The image to be detected is input into the trained coded marker detection network. After forward computation by the network, the confidence of the candidate target, the bounding box regression parameters, and the N-dimensional binary coded probability vector are output, where N is the number of bits in the binary code of the ring coded marker and N is a positive integer. The N-dimensional binary code probability vector of each candidate target is binarized to obtain the N-bit binary code of the candidate target; The N-bit binary code of each candidate target is converted into a minimum cyclic shift value, and the minimum cyclic shift value is mapped to a category number; The center coordinates of each candidate target are determined based on the bounding box regression parameters, and the center coordinates are spatially clustered to obtain at least one cluster. The frequency of occurrence of the category number of all candidate targets within each cluster is counted, and the category number with the highest frequency is determined as the final identity identifier of the corresponding ring code for that cluster. The bounding box regression parameters of the candidate targets belonging to the final identity identifier are averaged, and the averaged bounding box is used as the final bounding box of the ring-coded mark.

[0007] The beneficial effects of this invention are as follows: Addressing the difficulty of fine-grained differentiation in large-scale code libraries, it employs N-dimensional binary encoding vectors to directly represent the identity marker, utilizing prior coding structure to avoid linear growth in classifier complexity with the number of categories. To address label ambiguity caused by the cyclic equivalence property of binary encoding vectors, a cyclic-aware dynamic label matching mechanism is introduced during the training phase. By selecting the version with the smallest loss among all cyclic shift versions as the supervision signal, label ambiguity is eliminated and the distribution of samples at each encoding bit tends to be balanced. Simultaneously, during the inference phase, the binary code is converted into the minimum cyclic shift value and mapped to the category number, ensuring that the same physical marker always corresponds to a unique identity identifier. To address the difficulty of obtaining real-world labeled data and the problem of synthetic data domain offset, a hybrid training dataset combining physically synthesized rendering data and real-world data with multi-view geometric automatic annotation is constructed, balancing data scale and realism, effectively improving the model's generalization robustness. Furthermore, spatial clustering and majority voting fusion are used to eliminate duplicate detection conflicts, and the bounding box parameters are averaged to suppress positioning bias, thereby achieving high-precision and robust detection of ring-coded marker identity and location under complex imaging conditions. Attached Figure Description

[0008] The specific structure of the present invention will now be described in detail with reference to the accompanying drawings.

[0009] Figure 1 This is a block diagram of the circular coding mark detection method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the physical composition rendering data generation pipeline in an embodiment of the present invention; Figure 3 This is a schematic diagram of the automatic multi-view geometry annotation framework according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the overall process and core mechanism of the circular coding mark detection method according to an embodiment of the present invention; Figure 5 This is a diagram showing the recognition results of a previously unseen mark category according to an embodiment of the present invention; Figure 6 This is a diagram showing the recognition results in a new test scenario according to an embodiment of the present invention; Figure 7 This is a block diagram of a circular coding mark detection device according to an embodiment of the present invention; Figure 8 This is a schematic block diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0010] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0011] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0012] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0013] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0014] The terminology defined in this invention is as follows: Ring-coded marker: A cooperative marker with binary code arranged on a ring, serving as a unique identifier, and can be encoded and decoded through the light and dark code segments on the ring.

[0015] Cyclic equivalence property: The code band of a circular coded flag is closed along the circumference. The same physical flag can produce N different binary sequences due to different decoding start positions. These sequences are cyclically shifted from each other, which is called the cyclic equivalence property.

[0016] Cyclic sensing: refers to explicitly sensing and utilizing the cyclic shift equivalence property of the circular encoding label during training, and selecting the cyclic shift version corresponding to the minimum classification loss as the supervision signal through a dynamic label matching mechanism.

[0017] Minimum cyclic shift value: For an N-bit binary encoded vector, the binary string corresponding to the shift version with the smallest value among all N cyclic shift versions serves as the unique identifier of the encoding.

[0018] Physically synthesized rendering data: training image samples generated by procedurally simulating the camera imaging process, including perspective transformation, background embedding, and imaging degradation simulation.

[0019] Automatic annotation of real data using multi-view geometry: By utilizing multi-view geometric constraints, annotation of real scene image data is automatically generated through bundle adjustment and reprojection.

[0020] YOLOv5: An end-to-end object detection network.

[0021] DBSCAN: A density-based algorithm for noisy spatial clustering applications.

[0022] CIoU: Complete Intersection over Union loss, which takes into account the overlap area between the predicted box and the ground truth box, the distance between their center points, and the difference in aspect ratio.

[0023] BCE: Binary Cross-Entropy Loss.

[0024] like Figure 1 As shown, the first embodiment of the present invention is: a deep learning detection method for ring-coded markers, comprising the following steps: S1. Obtain the image to be detected.

[0025] In this step, the image to be detected is any scene image containing at least one ring-shaped coded marker, which can be acquired by imaging devices such as cameras or video cameras. In practical engineering applications, this image often faces complex imaging conditions such as changes in lighting, motion blur, and cluttered backgrounds, all of which may affect the accuracy of subsequent detection and recognition of the ring-shaped coded marker.

[0026] S2. Input the image to be detected into the trained coded marker detection network. After forward computation by the network, output the confidence of the candidate target, the bounding box regression parameters, and the N-dimensional binary coding probability vector, where N is the number of bits in the binary code of the ring coded marker and N is a positive integer.

[0027] In this step, the encoded label detection network adopts an end-to-end convolutional detection network structure. In this embodiment, the encoded label detection network uses the YOLOv5 network structure as its basic framework. After forward computation, it outputs the confidence scores of several candidate targets, bounding box regression parameters, and an N-dimensional binary encoded probability vector. The detection method in this scheme does not improve the main network structure, but rather adopts an N-bit binary encoding method for the category representation of the output vector, where N corresponds to the number of bits in the ring-coded label. This encoding method differs from traditional one-hot encoding, where the output dimension is fixed at N and does not increase with the number of categories to be distinguished. Compared to current advanced embedding encoding, this encoding method can directly utilize prior information of the encoding structure without needing to learn an additional embedding vector, making it more efficient. By using a low-dimensional binary vector to directly represent the category identity of the ring-coded label, the problem of classifier complexity increasing linearly with the number of categories is avoided. At the same time, the prior information of the binary structure of the ring-coded label is preserved, enabling the network to learn more discriminative fine-grained features.

[0028] S3. Binarize the N-dimensional binary code probability vector of each candidate target to obtain the N-bit binary code of the candidate target.

[0029] In a specific embodiment, the binarization processing of the N-dimensional binary encoded probability vector for each candidate target includes: S31. Compare each probability value in the N-dimensional binary encoded probability vector with a preset threshold. S32. If the probability value of this bit is greater than or equal to the preset threshold, then the bit is binarized to 1; S33. If the probability value of this bit is less than the preset threshold, then the bit is binarized to 0; S34. The N-bit binary code of the candidate target is formed by the binarization results of all bits, wherein the preset threshold is 0.5.

[0030] In this embodiment, each probability value represents the probability of that bit being 1. By comparing each value with a preset threshold of 0.5, the continuous probability output is converted into discrete binary codewords. This operation is simple and efficient, requiring no additional learning parameters. Furthermore, the threshold of 0.5 is a natural dividing point in terms of probability, which can preserve the classification information of the network output to the greatest extent and provide accurate codeword input for subsequent category mapping.

[0031] S4. Convert the N-bit binary code of each candidate target into a minimum cyclic shift value, and map the minimum cyclic shift value to a category number.

[0032] In this step, for an N-bit binary encoded vector All N cyclic shift versions can be represented as The binary string corresponding to the smallest shifted version is taken as the minimum cyclic shift value for this encoding. This operation is a standard deambiguation step in ring-coded flag decoding, ensuring that the same physical flag always corresponds to a unique identifier. Because ring-coded flags have cyclic equivalence properties—meaning the same physical flag can correspond to N different binary sequences depending on the starting position—this ambiguity can be eliminated through the minimum cyclic shift value conversion. This ensures that the same flag, regardless of its appearance in the image, can be mapped to the same category number, thereby ensuring the accuracy and consistency of subsequent identification.

[0033] S5. Determine the center coordinates of each candidate target based on the bounding box regression parameters, and perform spatial clustering on the center coordinates to obtain at least one cluster.

[0034] In this step, the DBSCAN density clustering algorithm is used to spatially cluster the center coordinates of all candidate results. The neighborhood radius is empirically set to 5 pixels, and the minimum number of elements in each cluster is 1. This groups candidate predictions that are geographically close together into the same cluster, with predictions within the same cluster corresponding to the same physical label. Since the detection network may generate multiple overlapping or adjacent candidate boxes for the same physical label during inference, spatial clustering can effectively organize these duplicate candidate predictions together, providing a foundation for subsequent category fusion and bounding box optimization. The DBSCAN algorithm does not require pre-specifying the number of clusters and can automatically identify noise points, making it suitable for practical applications where the number of candidate boxes is uncertain and their distribution is uneven.

[0035] S6. Count the frequency of occurrence of the category number of all candidate targets in each cluster, and determine the category number with the highest frequency as the final identity identifier of the ring code corresponding to the cluster.

[0036] In this step, given that the correctly predicted results of the fully trained network constitute a significant majority within the same candidate bounding box, the class number with the highest frequency among all candidate results in each cluster is used as the final identity of the label corresponding to that cluster. This majority voting mechanism effectively overcomes the problem of unreliable classification confidence of a single candidate bounding box under complex imaging conditions, fully utilizes the redundant information between multiple candidate predictions of the same physical label, and significantly improves the accuracy and robustness of label identification.

[0037] S7. The bounding box regression parameters of the candidate targets belonging to the final identity identifier are averaged, and the bounding box obtained by the average processing is used as the final bounding box of the ring-coded mark.

[0038] In this step, the parameters of all candidate bounding boxes belonging to the category number within the cluster are averaged, and this average value is used as the final bounding box of the marker. This processing method effectively suppresses the random positioning deviation of each detection box by using mean estimation. Compared with selecting any single box, it can output a more stable positioning result that is closer to the true boundary of the target, thus achieving high-precision and robust detection of the identity and location of ring-coded markers under complex imaging conditions.

[0039] In a specific embodiment, the encoded flag detection network is trained through the following steps: A1. Obtain training samples, wherein the training samples contain N-bit binary encoded reference labels of ring-coded flags.

[0040] The training samples include physically synthesized rendering data and real-world data with automatically labeled multi-view geometry. Labeling real-world scene data is extremely costly, and image degradation in complex scenes makes labeling itself difficult and inconsistent. While purely synthetic data can be generated in batches, it suffers from domain offset compared to the real-world imaging distribution, resulting in insufficient generalization when used alone. This invention combines the two: physically synthesized rendering data provides a large-scale sample covering diverse imaging conditions at low cost, ensuring the model learns a sufficient number of encoded feature characteristics; the automatically labeled multi-view geometry data uses a small number of reliable observations as seeds, automatically completing accurate labeling of a large number of difficult samples through multi-view geometric constraints, introducing the real-world data distribution into the training set, compensating for the domain offset of synthetic data, and enhancing the network's robustness to real-world lighting changes, noise, and blur. The two complement each other, balancing data scale and realism.

[0041] In a specific embodiment, the physically synthesized rendering data is generated in the following manner: B1. Generate the ring-shaped code template image corresponding to each binary code.

[0042] In this step, we first generate ideal circular code template images corresponding to each valid binary code, which serve as the basic material for composite rendering.

[0043] B2. Establish a virtual camera and a world coordinate system, and use the homography matrix to perform perspective projection transformation on the template image.

[0044] In this step, a virtual camera and a world coordinate system are established. The template image is treated as an image of the real marker points on the virtual camera, and then a rotation matrix is ​​used. Translation vector Apply a 3D spatial transformation to the marker template, and then pass it through the camera intrinsic parameter matrix. Projected onto the image plane, the entire coordinate transformation process is determined by the homography matrix. Unified description: in, The template image pixel coordinates, These are the pixel coordinates of the projected image. Let be any non-zero scale factor. For world coordinates on the marked plane, such as Figure 2 As shown. This transformation can simulate the perspective distortion of the ring-shaped coded mark on the imaging plane under different viewpoints and distances, allowing the synthetic data to cover a wider range of shooting postures.

[0045] B3. Embed the logo pattern after perspective projection transformation into the background image.

[0046] In this step, the logo pattern after perspective projection transformation is embedded into a background image taken from a natural scene dataset to simulate the complex background interference of the logo in a real scene.

[0047] B4. Apply at least one of grayscale transformation, Gaussian blur, and Gaussian noise to the embedded marker region to simulate imaging degradation and generate the physically synthesized rendering data.

[0048] In this step, different degrees of grayscale transformation, Gaussian blur, and Gaussian noise are applied to the marked areas to simulate imaging degradation phenomena such as brightness changes, defocus blur, and sensor noise, respectively, generating synthetic training samples in batches. Through this physically-based rendering method, training samples covering diverse imaging conditions can be generated on a large scale at low cost, effectively alleviating the problem of high costs associated with real data annotation.

[0049] In a specific embodiment, the real data for the automatic annotation of multi-view geometry is generated in the following manner: C1. Deploy ring-shaped coded markers in the scene and collect image sequences from multiple perspectives.

[0050] In this step, firstly, ring-shaped coded markers are deployed in the scene. Image sequences are acquired from multiple perspectives to obtain multiple frames of the same marker from different viewpoints, providing sufficient observation data for subsequent geometric calculations. Figure 3 As shown.

[0051] C2. Extract the coordinates and codes of the reliably observed marker pixels in each image.

[0052] In this step, a high-threshold conventional detection method is used to extract the coordinates and codes of reliable observations in each image. The high threshold setting ensures that the extracted observations have high confidence levels, serving as reliable input for subsequent bundle adjustment.

[0053] C3. Using the reliable observations as input, combine the camera intrinsic parameters, extrinsic parameters of each viewpoint, and the three-dimensional spatial coordinates of all markers, and perform bundle adjustment to obtain optimized three-dimensional marker points.

[0054] In this step, the reliable observations are used as input, combined with camera intrinsic parameters. External parameters from various perspectives , and the three-dimensional spatial coordinates of all signs The objective function model for performing bundle adjustment is as follows: in, The total number of image sequences, For the first The set of indices of observable markers in an image. For the first The first mark in the The coordinates of the observed pixels in the image are given, and u(·) is the camera projection function. Using bundle adjustment, the camera parameters and the coordinates of the 3D marker points can be jointly optimized to minimize reprojection errors, thereby obtaining high-precision 3D marker points.

[0055] C4. Reproject the optimized 3D marker points onto images from various viewpoints to obtain the image locations and numbers of unrecognized markers, and automatically complete the annotation.

[0056] In this step, after obtaining the optimized 3D marker points, they are reprojected onto images from various viewpoints to obtain the locations and numbers of marker images that could not be identified under complex imaging conditions. Then, sub-pixel-level positioning refinement is achieved through ellipse fitting, and annotation is automatically completed. This method automatically expands the annotation data using multi-view geometric constraints, solving the problems of high difficulty and poor consistency of manual annotation under imaging degradation conditions. It introduces the real data distribution into the training set, effectively compensating for the domain offset of synthetic data.

[0057] A2. Input the training image into the coding label detection network to be trained. After forward computation by the network, the predicted confidence, bounding box regression parameters and N-dimensional binary coding probability vector are output.

[0058] In this step, the training images from the mixed training dataset are input into the coding label detection network to be trained. After forward computation by the network, the confidence of each candidate target, the bounding box regression parameters, and the N-dimensional binary coding probability vector are output.

[0059] A3. For the reference label, generate all cyclically shifted versions of the reference label, and calculate the classification loss with each cyclically shifted version using the N-dimensional binary encoded probability vector, and select the minimum classification loss as the cyclic perceptual classification loss of the training sample.

[0060] In a specific embodiment, the classification loss is a binary cross-entropy loss, and the selection of the minimum classification loss includes: D1. For each cyclically shifted version of the reference label, calculate the binary cross-entropy loss between the N-dimensional binary encoded probability vector and the cyclically shifted version.

[0061] D2. Take the minimum value of the binary cross-entropy loss corresponding to all cyclic shift versions as the cyclic perception classification loss.

[0062] In this embodiment, the reference label vector for each training sample is: First, generate all of the reference tags. The cyclic shift version, the ... Each shifted version of the label vector can be represented as: in, Represents the first reference label vector Each bit is a binary digit. Then, the binary encoded probability vector predicted by the network is... Separately with this The binary cross-entropy loss is calculated for each shift label. For the first... The shifted version has a binary cross-entropy loss defined as: In all The minimum value among the possible loss values ​​is selected as the final classification loss for that sample, i.e., the recurrent perceptual classification loss. In this embodiment, the core characteristic of the circular coding label is that the same physical label can correspond to N different binary sequences depending on the starting position of the reading. Traditional fixed-label supervision forces the model to learn a specific shift version, resulting in an uneven distribution of coding bit samples, i.e., the probability of each position being 0 and 1 is unequal. The cyclic sensing dynamic matching mechanism proposed in this invention allows the model to dynamically select the one that best matches the current prediction from N equivalent shift labels during training to calculate the loss. Essentially, it relaxes the supervision from "fixed shift representation" to "cyclic equivalence class optimization". From an optimization perspective, this switching only occurs between equivalent representations of the same physical label, without introducing conflict signals. As training converges, the selected shift version naturally becomes fixed, degenerating into standard supervised learning, ensuring stable convergence; at the same time, the distribution of positive and negative samples of each coding bit tends to be balanced, effectively improving generalization ability.

[0063] like Figure 4 The diagram shown illustrates the overall process and core mechanism of a robust detection method for ring-coded markers based on cyclic sensing. Specifically, the localization loss employs the complete intersection-union loss, and the confidence loss employs the binary cross-entropy loss.

[0064] A4. Obtain the total loss by weighted summation of the recurrent perception classification loss, localization loss, and confidence loss. Update the network parameters based on the total loss until the network converges.

[0065] In this step, the total loss function is composed of a weighted sum of classification loss, localization loss, and confidence loss: in, These are the balancing weights for the three types of losses. Classification loss That is, the recurrent perception classification loss and localization loss mentioned above. A full intersection-union loss is employed, simultaneously considering the overlap area between the predicted and ground truth bounding boxes, the Euclidean distance between their center pixels, and the aspect ratio difference. The confidence loss uses a standard binary cross-entropy loss to distinguish foreground objects from background regions. Based on the total loss... Perform backpropagation to calculate the gradients of the parameters of each layer of the network, and use the optimizer to update the network parameters; iterate through steps A2 to A4 until the network converges to obtain the trained encoding label detection network.

[0066] To verify the significant effect of the technical solution of this invention, the inventors conducted the following comparative experiment. The results were compared with the currently advanced ElasticFace embedding learning method on a synthetic dataset, and are shown in Table 1. As can be seen from the table, the recurrent perceptual matching training method consistently achieves higher accuracy and recall than the ElasticFace embedding learning method. Furthermore, as the total number of categories increases, the performance of the ElasticFace embedding learning method drops sharply, while the performance of the recurrent perceptual matching method decreases significantly less. This indicates that directly utilizing the structural prior of binary codewords can avoid the optimization burden and supervisory ambiguity brought about by embedding mapping, thereby achieving robust high-precision recognition on large-scale codebases.

[0067] Table 1: Precision and recall of ElasticFace and recurrent-aware matching training methods under different total number of classes. Furthermore, to verify the effectiveness of the cyclic sensing dynamic label matching mechanism, the proposed cyclic sensing dynamic label matching was compared with the traditional minimum cyclic code value matching method. The minimum code value matching method is a traditional approach that only retains the smallest value among all binary cyclic shift versions. Figure 5 These are the recognition results of both methods for real-world landmarks not seen in the training set. Figure 6 The results show the identification results of real markers after changing to a new scene, with green indicating correct identification and red indicating incorrect identification. The statistical data from the two experiments are summarized in Table 2. The results show that the proposed cyclic aware dynamic label matching significantly outperforms the traditional minimum code value allocation method in generalization to unseen marker categories and new scenes, achieving a substantial lead in both precision and recall. This fully verifies that the proposed method effectively eliminates the inherent supervisory ambiguity of the traditional minimum cyclic code value matching scheme.

[0068] Table 2: Precision and recall of minimum code value and recurrent sensing matching training methods in unseen label categories and new scenarios. like Figure 7 As shown, this embodiment of the invention also provides a deep learning detection device for ring-coded markers, comprising: Image acquisition module 10 is used to acquire the image to be detected; The detection network module 20 is used to input the image to be detected into the trained coded marker detection network, and output the confidence of the candidate target, the bounding box regression parameters and the N-dimensional binary coding probability vector after the network forward calculation, where N is the number of binary coding bits of the ring coded marker and N is a positive integer. Binarization module 30 is used to binarize the N-dimensional binary encoding probability vector of each candidate target to obtain the N-bit binary encoding of the candidate target; The category mapping module 40 is used to convert the N-bit binary code of each candidate target into a minimum cyclic shift value, and map the minimum cyclic shift value to a category number; Clustering module 50 is used to determine the center coordinates of each candidate target based on the bounding box regression parameters, and to perform spatial clustering on the center coordinates to obtain at least one cluster. The identity determination module 60 is used to count the frequency of occurrence of the category number of all candidate targets in each cluster, and determine the category number with the highest frequency as the final identity identifier of the ring code corresponding to the cluster. The bounding box determination module 70 is used to average the bounding box regression parameters of the candidate targets belonging to the final identity identifier, and use the averaged bounding box as the final bounding box of the ring-coded mark.

[0069] In a specific embodiment, the device further includes a training module, which is used to train the encoded flag detection network, including: A training sample acquisition unit is used to acquire training samples, wherein the training samples contain an N-bit binary encoded reference label of a ring-coded flag; The training forward computation unit is used to input the training image into the encoded label detection network to be trained. After the network performs forward computation, it outputs the predicted confidence, bounding box regression parameters, and N-dimensional binary encoded probability vector. The cyclic sensing loss calculation unit is used to generate all cyclic shift versions of the reference label for the reference label, and calculate the classification loss by combining the N-dimensional binary encoded probability vector with each cyclic shift version, and select the minimum classification loss as the cyclic sensing classification loss of the training sample. The parameter update unit is used to obtain the total loss by weighted summation of the cyclic perception classification loss, localization loss and confidence loss, and update the network parameters based on the total loss until the network converges.

[0070] In a specific embodiment, the training sample acquisition unit includes: The training sample construction subunit is used to construct a hybrid training dataset, which includes physically synthesized rendering data and real data with automatic annotation of multi-view geometry.

[0071] In a specific embodiment, the training sample construction subunit includes: A physically based rendering data generation unit, used to generate the physically based rendering data, including: The template image generation subunit is used to generate the circular code marker template image corresponding to each binary code; The perspective projection transformation subunit is used to establish the virtual camera and the world coordinate system, and to perform perspective projection transformation on the template image using the homography matrix; The background embedding subunit is used to embed the logo pattern after perspective projection transformation into the background image; An imaging degradation simulation subunit is used to apply at least one of grayscale transformation, Gaussian blur, and Gaussian noise to the embedded marker region to generate the physically synthesized rendering data.

[0072] In a specific embodiment, the training sample construction subunit further includes: A multi-view geometry automatic annotation data generation unit, used to generate the real data for the multi-view geometry automatic annotation, includes: The image acquisition subunit is used to deploy ring-shaped coded markers in the scene to acquire image sequences from multiple perspectives; The reliable observation extraction subunit is used to extract the coordinates and coding numbers of the marker pixels of reliable observations in each image; The bundle adjustment subunit is used to take the reliable observations as input, combine the camera intrinsic parameters, the extrinsic parameters of each viewpoint, and the three-dimensional spatial coordinates of all markers, and perform bundle adjustment to obtain optimized three-dimensional marker points. The reprojection annotation subunit is used to reproject the optimized 3D marker points onto images from various viewpoints, obtain the image location and number of unrecognized markers, and automatically complete the annotation.

[0073] In a specific embodiment, the cyclic sensing loss calculation unit includes: A cross-entropy calculation subunit is used to calculate the binary cross-entropy loss between the N-dimensional binary encoded probability vector and the cyclically shifted version for each cyclically shifted version of the reference label. The minimum value selection sub-unit is used to take the minimum value of the binary cross-entropy loss corresponding to all cyclic shift versions as the cyclic perception classification loss. The localization loss uses the full intersection-union ratio loss, and the confidence loss uses the binary cross-entropy loss.

[0074] In a specific embodiment, the binarization module 30 includes: The threshold comparison subunit is used to compare each probability value in the N-dimensional binary encoded probability vector with a preset threshold. The first binarization subunit is used to binarize the bit to 1 when the probability value of the bit is greater than or equal to the preset threshold. The second binarization subunit is used to binarize the bit to 0 when the probability value of the bit is less than the preset threshold. The encoding constitutes a subunit, which is used to construct the N-bit binary code of the candidate target from the binarization results of all bits, wherein the preset threshold is 0.5.

[0075] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned ring-coded mark detection device and each unit can be referred to the corresponding description in the foregoing method embodiments. For the sake of convenience and brevity, it will not be repeated here.

[0076] The aforementioned ring-shaped coded marker detection device can be implemented as a computer program, which can, for example... Figure 8 It runs on the computer device shown.

[0077] Please see Figure 8 , Figure 8 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a terminal or a server. The terminal can be an electronic device with communication functions, such as a smartphone, tablet, laptop, desktop computer, personal digital assistant, or wearable device. The server can be a standalone server or a server cluster composed of multiple servers.

[0078] See Figure 8 The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0079] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform a deep learning detection method for ring-coded flags.

[0080] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0081] The internal memory 504 provides an environment for the execution of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a deep learning detection method for ring-coded flags.

[0082] This network interface 505 is used for network communication with other devices. Those skilled in the art will understand that... Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0083] The processor 502 is used to run a computer program 5032 stored in a memory to implement the circular coded marker detection method as described above.

[0084] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0085] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0086] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When executed by a processor, the program instructions cause the processor to perform the ring-coded flag detection method as described above.

[0087] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0088] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0089] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0090] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0091] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0092] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A deep learning method for detecting ring-coded markers, characterized in that, Includes the following steps: Acquire the image to be detected; The image to be detected is input into the trained coded marker detection network. After forward computation by the network, the confidence of the candidate target, the bounding box regression parameters, and the N-dimensional binary coded probability vector are output, where N is the number of bits in the binary code of the ring coded marker and N is a positive integer. The N-dimensional binary code probability vector of each candidate target is binarized to obtain the N-bit binary code of the candidate target; The N-bit binary code of each candidate target is converted into a minimum cyclic shift value, and the minimum cyclic shift value is mapped to a category number; The center coordinates of each candidate target are determined based on the bounding box regression parameters, and the center coordinates are spatially clustered to obtain at least one cluster. The frequency of occurrence of the category number of all candidate targets within each cluster is counted, and the category number with the highest frequency is determined as the final identity identifier of the corresponding ring code for that cluster. The bounding box regression parameters of the candidate targets belonging to the final identity identifier are averaged, and the averaged bounding box is used as the final bounding box of the ring-coded mark.

2. The deep learning detection method for ring-coded markers according to claim 1, characterized in that, The encoded flag detection network is trained through the following steps: Obtain training samples, wherein the training samples contain N-bit binary encoded reference labels of the ring-coded flag; The training image is input into the coding label detection network to be trained. After the network performs forward computation, the predicted confidence, bounding box regression parameters and N-dimensional binary coding probability vector are output. For the reference label, generate all cyclically shifted versions of the reference label, and calculate the classification loss with each cyclically shifted version using the N-dimensional binary encoded probability vector. Select the minimum classification loss as the cyclic perceptual classification loss for the training sample. The total loss is obtained by weighted summation of the recurrent perception classification loss, localization loss, and confidence loss. The network parameters are then updated based on the total loss until the network converges.

3. The deep learning detection method for ring-coded markers according to claim 2, characterized in that, The training samples include physically synthesized rendering data and real data with automatic multi-view geometry annotation.

4. The deep learning detection method for ring-coded markers according to claim 3, characterized in that, The physically composed rendering data is generated in the following manner: Generate ring-shaped code marker template images corresponding to each binary code; Establish a virtual camera and a world coordinate system, and use a homography matrix to perform perspective projection transformation on the template image; Embed the logo pattern after perspective projection transformation into the background image; The physical composite rendering data is generated by applying at least one of grayscale transformation, Gaussian blur, and Gaussian noise to the embedded marker region to simulate imaging degradation.

5. The deep learning detection method for ring-coded markers according to claim 3, characterized in that, The real data for the automatic annotation of multi-view geometry is generated in the following manner: Deploy ring-shaped coded markers in the scene to acquire image sequences from multiple perspectives; Extract the coordinates and codes of reliable observation marker pixels from each image; Using the reliable observations as input, combined with the camera intrinsic parameters, extrinsic parameters of each viewpoint, and the three-dimensional spatial coordinates of all markers, the bundle adjustment method is performed to obtain optimized three-dimensional marker points; The optimized 3D marker points are reprojected onto images from various viewpoints to obtain the image locations and numbers of unrecognized markers, and annotation is automatically completed.

6. The deep learning detection method for ring-coded markers according to claim 2, characterized in that, The classification loss is a binary cross-entropy loss, and selecting the minimum classification loss includes: For each cyclically shifted version of the reference label, calculate the binary cross-entropy loss between the N-dimensional binary encoded probability vector and that cyclically shifted version; The minimum value among the binary cross-entropy losses corresponding to all cyclic shift versions is taken as the cyclic perception classification loss. The localization loss uses the full intersection-union loss, and the confidence loss uses the binary cross-entropy loss.

7. The deep learning detection method for ring-coded markers according to claim 1, characterized in that, The binarization process for the N-dimensional binary encoded probability vector of each candidate target includes: Compare the probability value of each bit in the N-dimensional binary encoded probability vector with a preset threshold; If the probability value of this bit is greater than or equal to the preset threshold, then the bit is binarized to 1; If the probability value of this bit is less than the preset threshold, then the bit is binarized to 0; The N-bit binary code of the candidate target is formed by the binarization results of all bits, wherein the preset threshold is 0.5.

Images