Identification detection method and device, readable storage medium and computer program product

Abstract

The present application provides a kind of identification detection method and device, readable storage medium and computer program product.The identification detection method includes: obtaining the target region image in target object image, target region image corresponds the target region of target object;According to target feature extraction network, the similarity value of target region image and the template image of target region is determined;According to the comparison result of similarity value and target threshold value, the identification detection result in target region is determined.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a method and apparatus for identifying symbols, a readable storage medium, and a computer program product. Background Technology

[0002] Currently, common label detection methods include manual inspection, template search inspection, and target detection. However, all of these methods have certain drawbacks. For example, manual inspection is inefficient; template search inspection has high requirements for the physical environment and cannot effectively detect similar labels; target detection relies on a large amount of sample material for annotation and training, making it unsuitable for production scenarios with diverse label types and unpredictable label changes. Thus, when inspecting labels, the detection efficiency is low, and the accuracy of the detection results is reduced, failing to meet the required inspection standards. Summary of the Invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art or related art.

[0004] Therefore, the first aspect of the present invention is to provide a method for detecting identifiers.

[0005] A second aspect of the present invention is to provide an identification detection device.

[0006] A third aspect of the present invention is to provide a readable storage medium.

[0007] A fourth aspect of the present invention is to provide a computer program product.

[0008] In view of this, according to one aspect of the present invention, a marker detection method is proposed, the method comprising: acquiring a target region image in a target object image, the target region image corresponding to a target region of the target object; determining a similarity value between the target region image and a template image of the target region according to a target feature extraction network; and determining a marker detection result within the target region based on a comparison result of the similarity value and a target threshold.

[0009] The technical solution of the tag detection method provided by this invention can be implemented by an electronic device, a tag detection device, or something else depending on the actual usage requirements; no specific limitation is made here. To more clearly describe the tag detection method provided by this invention, the following description uses a tag detection device as the implementing entity.

[0010] Specifically, in the marker detection method provided by this invention, the marker detection device acquires an image of a target object, and then acquires an image of a target region to be detected from the target object image. This target region image is the image of the target region of the target object. Further, the marker detection device acquires a template image of the target region of the target object, and compares the template image and the target region image based on a comparison network. Specifically, based on a target feature extraction network, a similarity analysis is performed on the template image and the target region image to determine the similarity value between them. Based on this, the marker detection device compares a set target threshold with the determined similarity value, and then determines the marker detection result within the target region of the target object based on the comparison result. Thus, based on the target feature extraction network, the similarity value between the target region image to be detected and the template image in the target object is determined, and then the marker detection result within the target region of the target object is determined based on the similarity value. This method of marker detection based on similarity values ​​for the target region of the target object achieves marker detection based on a unified threshold, which is suitable for detecting multiple types of unknown markers, increasing application scenarios, and eliminating the need to collect a large number of samples, thereby improving marker detection efficiency and ensuring the accuracy of the detection results. Furthermore, the system automatically identifies and detects target objects based on the acquired images, reducing manual workload and further ensuring the accuracy and efficiency of the detection results.

[0011] The identification detection method of the present invention may further include the following additional technical features:

[0012] In the above technical solution, obtaining the target region image in the target object image includes: obtaining a first region image from the target object image based on the image information of the template image; comparing the template image and the first region image; obtaining the target region image from the first region image based on the comparison result and the size information of the template image; wherein the size of the first region image is larger than the size of the target region image.

[0013] In this technical solution, during the process of the marker detection device acquiring the target region image to be detected from the target object image, specifically, the marker detection device crops the target object image based on the image information of the template image of the target region, thereby obtaining a first region image to be calibrated. Further, the marker detection device performs calibration comparison on the acquired first region image and the template image, and cuts the first region image according to the size information of the template image and the comparison result, thereby obtaining the target region image to be detected. The size of the target region image is smaller than the size of the first region image. Thus, in the process of acquiring the target region image to be detected from the target object image, firstly, based on the image information of the template image of the target region, a larger first region image is cropped from the target object image, and then, based on the size information of the template image, a smaller target region image is cut from the first region image. In this way, while discarding invalid information in the target object image that cannot be used for identification detection, the loss of valid information in the target object image for identification detection is reduced, thereby improving the comprehensiveness of the valid information contained in the target region image. When the template image and the target region image are analyzed in the subsequent process to obtain the similarity value between the two, the accuracy of the determined similarity value can be guaranteed, thus ensuring the accuracy of identification detection of the target region.

[0014] In any of the above technical solutions, the size of the first region image is larger than the size of the template image, and the size of the target region image is equal to the size of the template image.

[0015] In this technical solution, the size of the template image is smaller than the size of the first region image, and the size of the template image is equal to the size of the target region image. Thus, in the process of obtaining the target region image to be detected from the target object image, a first region image with a larger image size than the template image is first cropped from the target object image, and then a target region image with the same image size as the template image is cut from the first region image. This reduces the impact of positional fluctuations caused by different imaging device models on the coordinate information of the template image and the target object image, thereby reducing the loss of effective information for identification detection in the target object image and improving the comprehensiveness of the effective information contained in the target region image. When subsequently analyzing the template image and the target region image to obtain their similarity value, the accuracy of the determined similarity value can be guaranteed, thus improving the accuracy of the identification detection result. Furthermore, since the template image and the target region image have the same size, the accuracy of the identification detection result can be further improved when performing identification detection based on the similarity analysis results of the template image and the target region image.

[0016] In any of the above technical solutions, obtaining a first region image from a target object image based on the image information of a template image includes: determining second coordinate information based on the first coordinate information of the template image; and cropping the target object image based on the second coordinate information to obtain the first region image.

[0017] In this technical solution, during the process of cropping the first region image to be calibrated from the target object image based on the image information of the template image, specifically, the marker detection device acquires the first coordinate information of the template image and expands the acquired first coordinate information to obtain second coordinate information with a larger coordinate region. Further, the marker detection device then crops the target object image based on the expanded second coordinate information to obtain a first region image with an image size larger than the template image. This reduces the impact of positional fluctuations caused by different imaging device models on the coordinate information of the template image and the target object image, thereby reducing the loss of effective information in the target object image during image cropping and improving the comprehensiveness of the effective information contained in the target region image. When subsequently analyzing the template image and the target region image to obtain their similarity values, the accuracy of the determined similarity values ​​can be guaranteed, thus improving the accuracy of the marker detection results.

[0018] In any of the above technical solutions, determining the similarity value between the target region image and the template image of the target region based on the target feature extraction network includes: normalizing the target region image and the template image; extracting feature vectors from the normalized target region image and the template image based on the target feature extraction network to obtain a first feature vector matrix and a second feature vector matrix; calculating the similarity distance between the first feature vector matrix and the second feature vector matrix; and determining the similarity value based on the similarity distance.

[0019] In this technical solution, during the process of the marker detection device performing similarity analysis on the template image and the target region image through a target feature extraction network to obtain the similarity value between them, specifically, the marker detection device normalizes the template image and the target region image to obtain a standard pattern template image and target region image. Further, the marker detection device then extracts features from the normalized template image and target region image through the target feature extraction network. Specifically, the marker detection device extracts a first feature vector of the target region image in multiple dimensions to obtain a first feature vector matrix of the target region image; further, the marker detection device extracts a second feature vector of the template image in multiple dimensions to obtain a second feature vector matrix of the template image. Further, based on the extracted second feature vector matrix and the first feature vector matrix, the marker detection device calculates the similarity distance between the second feature vector matrix and the first feature vector matrix in multiple dimensions, and then determines the similarity value between the template image and the target region image based on the calculated similarity distance. In this way, by extracting image feature vectors through a target feature extraction network, similarity analysis is performed on the template image and the target region image to obtain their similarity value. On the one hand, this ensures the accuracy of the similarity value determination, thereby ensuring the accuracy of the label detection results. On the other hand, based on the similarity value, a unified threshold detection for the label to be detected is achieved, which is applicable to the detection of multiple types of unknown labels to be detected, increasing the application scenarios and eliminating the need to collect a large number of samples, thus improving the label detection efficiency.

[0020] In any of the above technical solutions, the similarity value and the similarity distance are inversely proportional.

[0021] In this technical solution, the similarity distance between the second feature vector matrix of the template image and the first feature vector of the target region image is inversely proportional to the similarity value between the template image and the target region image. The larger the similarity distance, the higher the similarity value, and the smaller the similarity distance, the lower the similarity value.

[0022] In any of the above technical solutions, the similarity value is inversely proportional to the similarity distance as an exponential function of the target value, where the target value is the base of the natural logarithm.

[0023] In this technical solution, the similarity value between the template image and the target region image is inversely proportional to the exponential function of the similarity distance with respect to the target value.

[0024] Here, the aforementioned target value is the base of the natural logarithm. That is, the similarity value between the template image and the target region image is related to e. xThey are inversely proportional, where x is the similarity distance between the second feature vector matrix of the template image and the first feature vector of the target region image.

[0025] In any of the above technical solutions, the relationship between similarity value and similarity distance is as follows: Where sim represents the similarity value and x represents the similarity distance.

[0026] In this technical solution, the mapping relationship between the similarity value and the similarity distance can be specifically expressed by the following formula (9):

[0027]

[0028] Where sim represents the similarity value between the template image and the target region image, and x represents the similarity distance between the second feature vector matrix of the template image and the first feature vector of the target region image.

[0029] In any of the above technical solutions, the identification detection result within the target area is determined based on the comparison result between the similarity value and the target threshold, including: if the similarity value is greater than the target threshold, the identification detection result within the target area is determined to be qualified; if the similarity value is less than or equal to the target threshold, the identification detection result within the target area is determined to be unqualified.

[0030] In this technical solution, during the process of the label detection device detecting the label pasting status within a target area based on the comparison result between a set target threshold and a determined similarity value, specifically, if the set target threshold is less than the aforementioned similarity value, the label detection device determines that the label to be detected within the target area is normal, that is, the target area successfully passes the label detection. Conversely, if the set target threshold is greater than or equal to the aforementioned similarity value, the label detection device determines that the label to be detected within the target area is abnormal, that is, the target area fails the label detection. Thus, by detecting the label to be detected within the target area based on the comparison result between the similarity value and the set target threshold, a method of detecting label to be detected based on a unified threshold is achieved. This method is applicable to the detection of multiple types of unknown label to be detected, increasing application scenarios and eliminating the need to collect a large number of samples, thereby improving label detection efficiency and ensuring the accuracy of the detection results.

[0031] In any of the above technical solutions, after determining the identification detection results within the target area, the identification detection method further includes: displaying the target object image and similarity values; and displaying the target identification on the target object image based on the identification detection results, wherein the target identification is used to indicate the identification detection results within the target area.

[0032] In this technical solution, after the sign detection device completes the detection of the label pasting status within the target area of ​​the target object, it displays the aforementioned similarity value and an image of the target object to the user. Furthermore, based on the sign detection results for the target area, the sign detection device also displays a target sign on the target object image. This target sign indicates whether the sign to be detected within the target area is normal or abnormal. This allows the user to intuitively understand the sign detection status within the target area and thus take timely corrective measures when abnormalities are found in the sign to be detected within the target area.

[0033] In any of the above technical solutions, displaying a target identifier on a target object image based on the identifier detection result includes: displaying a target identifier in a first display mode on the target object image when the identifier detection result is qualified; and displaying a target identifier in a second display mode on the target object image when the identifier detection result is unqualified; wherein the display position of the target identifier corresponds to the second coordinate information.

[0034] In this technical solution, during the process of the sign detection device displaying the target sign on the target object image based on the sign detection results of the target area, specifically, when the sign detection device determines that the sign to be detected in the target area is normal, the sign detection device displays the target sign on the target object image according to a first display mode; and when the sign detection device determines that the sign to be detected in the target area is abnormal, the sign detection device displays the target sign on the target object image according to a second display mode. In this way, by displaying target signs in different display modes, the sign detection results of the target area are visually presented to the user, allowing the user to intuitively understand the sign detection status in the target area and thus take timely adjustment measures when there are abnormalities in the sign to be detected in the target area.

[0035] In any of the above technical solutions, before acquiring the target region image in the target object image, the identification detection method further includes: acquiring the barcode information of the target object; retrieving the template information of the target region based on the barcode information; retrieving the template image if the template information is successfully retrieved; and displaying a prompt message if the template information retrieval fails to retrieve, to remind the user to create a template image.

[0036] In this technical solution, before acquiring the target area image, the identification detection device scans the barcode on the target object to identify its barcode information. Then, based on the identified barcode information, it determines whether a template image of the target area is pre-stored in the storage space. Specifically, the identification detection device retrieves the template information of the target area based on the identified barcode information. If the template information is successfully retrieved, it indicates that a template image of the target area is pre-stored in the storage space. In this case, the identification detection device directly retrieves and uses this template image for subsequent identification detection. If the identification detection device fails to retrieve the template information, it indicates that a template image of the target area is not pre-stored in the storage space. In this case, the identification detection device displays a prompt to the user, reminding them to create a new template image, and then performs subsequent identification detection based on the newly created template image.

[0037] In any of the above technical solutions, the types of identifiers to be detected in the target area of ​​different target objects are different. The identifier detection results in the target area are determined based on the comparison results of similarity value and target threshold, including: determining the detection results of different types of identifiers to be detected in the target area based on the comparison results of similarity value and target threshold.

[0038] In this technical solution, the types of identifiers to be detected differ within the target areas of different target objects. Based on this, during the identifier detection process in the target area, the detection results for different types of identifiers within the target area are determined by comparing the similarity values ​​of the template image and the target area image with a set target threshold. This achieves identifier detection based on a unified threshold, applicable to detecting multiple types of unknown identifiers, expanding application scenarios, and eliminating the need for collecting large numbers of samples, thereby improving identifier detection efficiency and ensuring the accuracy of the detection results. Furthermore, automatically detecting identifiers on target objects based on the collected target object images reduces manual workload, further guaranteeing the accuracy and efficiency of the detection results.

[0039] According to a second aspect of the present invention, a marker detection device is provided, the device comprising: an acquisition unit for acquiring a target region image in a target object image, the target region image corresponding to a target region of the target object; a processing unit for determining a similarity value between the target region image and a template image of the target region based on a target feature extraction network; and the processing unit further for determining a marker detection result within the target region based on a comparison result of the similarity value and a target threshold.

[0040] The identifier detection device provided by this invention includes an acquisition unit and a processing unit. During identifier detection of a target object, the acquisition unit acquires an image of the target object, and then acquires an image of the target region to be detected from the target object image. This target region image is the image of the target region of the target object. Further, the acquisition unit acquires a template image of the target region of the target object. The processing unit compares the template image and the target region image based on a comparison network. Specifically, the processing unit performs similarity analysis on the template image and the target region image based on a target feature extraction network, thereby determining the similarity value between the two. Based on this, the processing unit compares a set target threshold with the determined similarity value, and then determines the identifier detection result within the target region of the target object based on the comparison result. Thus, based on the target feature extraction network, the similarity value between the target region image to be detected and the template image in the target object is determined, and then the identifier detection result within the target region of the target object is determined based on the similarity value. In this way, target detection based on similarity values ​​of target regions achieves a method of detecting identifiers based on a unified threshold. This is applicable to the detection of multiple types of unknown identifiers, expanding application scenarios and eliminating the need to collect a large number of samples, thereby improving identifier detection efficiency and ensuring the accuracy of detection results. Furthermore, automatic identifier detection based on the acquired target object image reduces manual workload, further guaranteeing the accuracy and efficiency of detection results.

[0041] According to a third aspect of the present invention, a readable storage medium is provided on which a program or instructions are stored, which, when executed by a processor, implement the identifier detection method as described in any of the above-described technical solutions. Therefore, the readable storage medium proposed in the third aspect of the present invention possesses all the beneficial effects of the identifier detection method in any of the technical solutions of the first aspect, and will not be elaborated further here.

[0042] According to a fourth aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the identifier detection method as described in any of the above-described technical solutions. Therefore, the computer program product proposed in the fourth aspect of the present invention possesses all the beneficial effects of the identifier detection method in any of the technical solutions of the first aspect, which will not be elaborated further here.

[0043] Additional aspects and advantages of the invention will become apparent in the following description or may be learned by practice of the invention. Attached Figure Description

[0044] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0045] Figure 1 One of the flowcharts of the identifier detection method according to an embodiment of the present invention is shown;

[0046] Figure 2 A second schematic flowchart of the identifier detection method according to an embodiment of the present invention is shown;

[0047] Figure 3 The third schematic flowchart of the identifier detection method according to an embodiment of the present invention is shown;

[0048] Figure 4 The fourth schematic flowchart of the identifier detection method according to an embodiment of the present invention is shown;

[0049] Figure 5 The fifth schematic flowchart of the identifier detection method according to an embodiment of the present invention is shown;

[0050] Figure 6 The sixth schematic flowchart of the identifier detection method according to an embodiment of the present invention is shown;

[0051] Figure 7 The seventh flowchart of the identifier detection method according to an embodiment of the present invention is shown;

[0052] Figure 8 The eighth schematic flowchart of the identifier detection method according to an embodiment of the present invention is shown;

[0053] Figure 9 A flowchart of the identifier detection method according to an embodiment of the present invention is shown in Figure 9.

[0054] Figure 10 A schematic diagram of the identifier detection method according to an embodiment of the present invention is shown;

[0055] Figure 11 One of the operation flowcharts of the identifier detection method according to an embodiment of the present invention is shown;

[0056] Figure 12 This illustrates a second operation flowchart of the identifier detection method according to an embodiment of the present invention;

[0057] Figure 13 The third flowchart of the identification detection method according to an embodiment of the present invention is shown;

[0058] Figure 14 The fourth flowchart of the identification detection method according to an embodiment of the present invention is shown;

[0059] Figure 15 The fifth flowchart of the identification detection method according to an embodiment of the present invention is shown;

[0060] Figure 16The sixth flowchart of the identification detection method according to an embodiment of the present invention is shown;

[0061] Figure 17 The seventh flowchart of the identification detection method according to an embodiment of the present invention is shown;

[0062] Figure 18 The eighth flowchart of the identification detection method according to an embodiment of the present invention is shown;

[0063] Figure 19 A structural block diagram of the identification detection device according to an embodiment of the present invention is shown;

[0064] Figure 20 A structural block diagram of an electronic device according to an embodiment of the present invention is shown.

[0065] in, Figures 11 to 18 The correspondence between the various structures and their structural designations is as follows:

[0066] 302 Target object image, 304 First template image, 306 First region image, 308 Target region image, 310 First identifier, 312 Second identifier, 314 Second template image. Detailed Implementation

[0067] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0068] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0069] The following is combined Figures 1 to 20 The identification detection method and apparatus, readable storage medium and computer program product provided in this application will be described in detail through specific embodiments and application scenarios.

[0070] In one embodiment of the present invention, such as Figure 1 As shown, the identification detection method may specifically include the following steps 102 to 106:

[0071] Step 102: Obtain the target region image from the target object image;

[0072] Step 104: Based on the target feature extraction network, determine the similarity value between the target region image and the template image of the target region;

[0073] Step 106: Determine the label detection result within the target area based on the comparison results of similarity values ​​and target thresholds;

[0074] In this context, the target region image corresponds to the target region of the target object.

[0075] The technical solution of the tag detection method provided by this invention can be implemented by an electronic device, a tag detection device, or something else depending on the actual usage requirements; no specific limitation is made here. To more clearly describe the tag detection method provided by this invention, the following description uses a tag detection device as the implementing entity.

[0076] Specifically, in the marker detection method provided by this invention, during the marker detection process of a target object, the marker detection device acquires an image of the target object, and then acquires an image of the target region to be detected from the target object image. This target region image is the image of the target region of the target object. Further, the marker detection device acquires a template image of the target region of the target object, and compares the template image and the target region image based on a comparison network. Specifically, based on a target feature extraction network, a similarity analysis is performed on the template image and the target region image to determine the similarity value between them. Based on this, the marker detection device compares a set target threshold with the determined similarity value, and then determines the marker detection result within the target region of the target object based on the comparison result. Thus, based on the target feature extraction network, the similarity value between the target region image to be detected and the template image in the target object is determined, and then the marker detection result within the target region of the target object is determined based on the similarity value. In this way, target detection based on similarity values ​​of target regions achieves a method of detecting identifiers based on a unified threshold. This is applicable to the detection of multiple types of unknown identifiers, expanding application scenarios and eliminating the need to collect a large number of samples, thereby improving identifier detection efficiency and ensuring the accuracy of detection results. Furthermore, automatic identifier detection based on the acquired target object image reduces manual workload, further guaranteeing the accuracy and efficiency of detection results.

[0077] In practical applications, the aforementioned identifiers to be detected can specifically include product labels, barcodes containing candidate information on exam answer sheets, signature information, and seal information in documents. The identifier detection method proposed in this invention can detect the affixing of labels within the target area of ​​a target object, the affixing of barcodes containing candidate information on exam answer sheets, and the detection of signature information, seal information, and other information in documents, without specific limitations.

[0078] The following example, using the product label as an example, will explain the label detection method proposed in this invention.

[0079] Specifically, when the mark to be detected is a product label, the target area can be the label pasting area in the target object, or it can be an area in the target object where pasting labels is prohibited, without specific restrictions.

[0080] Based on this, when the target area is a label pasting area, the aforementioned label detection method is used to check whether labels are correctly pasted within the target area. For example, it checks whether any labels are missing from the target area, and whether the labels already pasted in the target area are pasted upside down, incorrectly, crookedly, excessively, or damaged compared to the standard labels that should be pasted in the target area. When the target area is an area where label pasting is prohibited within the target object, the aforementioned label detection method is used to determine whether any labels have already been pasted within the target area.

[0081] Furthermore, the aforementioned template image can be a template image of the target area of ​​the target object pre-stored in the storage space. During the detection of the label pasting status on the target object, the label detection device can directly retrieve and use this template image. The aforementioned template image can also be a newly created template image by the label detection device during the detection process, without specific limitations.

[0082] Furthermore, the aforementioned similarity value is used to indicate the degree of similarity between the template image and the target region image. The similarity value is directly proportional to the degree of similarity; a larger similarity value indicates a higher degree of similarity between the template image and the target region image, while a smaller similarity value indicates a lower degree of similarity. Thus, by comparing this similarity value with a set target threshold, the labeling status within the target area of ​​the target object can be detected, thereby detecting whether there are any instances of missing, excessive, upside-down, incorrect, crooked, or damaged labels within the target area.

[0083] Furthermore, the aforementioned target threshold can specifically be a value such as 0.5, 0.55, 0.6, or 0.65. In practical applications, those skilled in the art can set the specific value of the aforementioned target threshold according to the actual situation, and no specific restrictions are imposed here.

[0084] Furthermore, in practical applications, the aforementioned target feature extraction network can specifically be a dual-path CNN (Convolutional Neural Network) feature extraction network. Specifically, the CNN feature extraction network is based on a ResNet (residual) network, such as ResNet18, for feature extraction. It removes the fully connected layers of the ResNet18 network and uses new weights formed by transfer learning of the weights pre-trained on ImageNet (a computer vision system recognition project) as the final feature extraction weights.

[0085] In addition, during the transfer training of the CNN feature extraction network, the following formula (1) can be used as the loss function for network transfer training:

[0086] Loss=(1-y)d 2 +y{max(md,0)} 2 (1)

[0087] Where Loss represents the loss value, d represents the Euclidean distance between the feature vectors of two sample images in each sample group, m is a hyperparameter that can be adjusted according to the training task, and y is used to indicate whether two sample images in each sample group belong to the same category during network transfer training. y = 0 indicates that the two sample images belong to the same category, and y = 1 indicates that the two sample images belong to different categories. Specifically, when y = 1, the optimization objective d is greater than m through the above loss function, thereby increasing the inter-class distance and decreasing the intra-class distance. Furthermore, if two sample images are two label images collected from different samples for the same type of label, the two sample images are considered to belong to the same category.

[0088] In this embodiment of the invention, further, as Figure 2 As shown, step 102 above may specifically include steps 102a to 102c as follows:

[0089] Step 102a: Obtain the first region image from the target object image based on the image information of the template image;

[0090] Step 102b: Compare the first region image with the template image;

[0091] Step 102c: Based on the size information of the template image and the comparison results, obtain the target region image from the first region image;

[0092] The size of the first region image is larger than the size of the target region image.

[0093] In this embodiment, during the process of the marker detection device acquiring the target region image to be detected from the target object image, specifically, the marker detection device crops the target object image based on the image information of the template image of the target region, thereby obtaining a first region image to be calibrated. Further, the marker detection device performs calibration comparison on the acquired first region image and the template image, and cuts the first region image according to the size information of the template image and the comparison result, thereby obtaining the target region image to be detected. The size of the target region image is smaller than the size of the first region image. Thus, in the process of acquiring the target region image to be detected from the target object image, firstly, based on the image information of the template image of the target region, a larger first region image is cropped from the target object image, and then, based on the size information of the template image, a smaller target region image is cut from the first region image. In this way, while discarding invalid information in the target object image that cannot be used for identification detection, the loss of valid information in the target object image for identification detection is reduced, thereby improving the comprehensiveness of the valid information contained in the target region image. When the template image and the target region image are analyzed in the subsequent process to obtain the similarity value between the two, the accuracy of the determined similarity value can be guaranteed, thus ensuring the accuracy of identification detection of the target region.

[0094] In the process of the identification detection device calibrating and comparing the acquired first region image and template image, the identification detection device can extract features from the first region image and template image respectively to determine the image feature information of the first region image and template image. Then, by matching and comparing the image feature information of the first region image and template image, the image content of the first region image and template image is compared, thereby determining the image region in the first region image that is closer to the image content of the template image, and obtaining the comparison result of the first region image and template image.

[0095] Furthermore, the image information of the template image may specifically include the coordinate information of the template image in the coordinate system of the shooting device, such as the camera coordinate system, when the template image is captured. The image information may also specifically include the size information of the template image, which may specifically include the height and width information of the template image.

[0096] Based on this, during the process of the marker detection device acquiring the target region image to be detected from the target object image, specifically, the marker detection device determines the size information of the first region image to be cropped and its coordinate information in the coordinate system of the current shooting device based on the coordinate information and size information of the template image, and crops the target object image to obtain the first region image. Further, based on the height and width information of the template image, and according to the calibration comparison results between the first region image and the template image, the marker detection device determines the coordinate information and size information of the target region image to be cut from the image region in the first region image that is closest to the image content of the template image, and cuts the first region image to obtain a target region image with an image size smaller than the first region image. In this way, during the process of cropping the target region image from the target object image, the influence of positional fluctuations caused by different shooting device models on the coordinate information of the template image and the target object image can be reduced, thereby reducing the loss of effective information for marker detection in the target object image, improving the comprehensiveness of the effective information contained in the target region image, and thus ensuring the accuracy of marker detection in the target region.

[0097] In this embodiment of the invention, the size of the first region image is larger than the size of the template image, and the size of the target region image is equal to the size of the template image.

[0098] In this embodiment, the size of the template image is smaller than the size of the first region image, and the size of the template image is equal to the size of the target region image.

[0099] In other words, the size relationship between the template image, the first region image, and the target region image is: the size of the template image = the size of the target region image < the size of the first region image.

[0100] Specifically, the relationship between the height and width values ​​of the template image, the first region image, and the target region image is as follows:

[0101] w c >w t h c >h t ,

[0102] w a =w t h a =h t ,

[0103] Among them, w c h represents the width value of the first region of the image. c w represents the height value of the first region of the image. th represents the width value of the template image. t w represents the height value of the template image. a h represents the width value of the target region image. a This represents the height value of the target region image.

[0104] The first region image mentioned above is obtained by cropping the target object image based on the coordinate and size information of the template image. Specifically, the identification detection device determines the size information of the first region image to be cropped and its coordinate information in the coordinate system of the current shooting device based on the coordinate and size information of the template image, and crops the target object image to obtain a first region image with an image size larger than the template image.

[0105] Furthermore, the aforementioned target region image is obtained by segmenting the first region image based on the height and width information of the template image and the calibration comparison result between the first region image and the template image. Specifically, the identification detection device then determines the coordinate and size information of the target region image to be segmented based on the height and width information of the template image and the calibration comparison result between the first region image and the template image, and segments the first region image to obtain a target region image with the same image size as the template image.

[0106] Thus, in the process of acquiring the target region image to be detected from the target object image, a first region image larger than the template image is first cropped from the target object image. Then, a target region image with the same image size as the template image is cut from the first region image. This approach reduces the impact of positional fluctuations caused by different imaging device models on the coordinate information of the template and target object images, thereby minimizing the loss of effective information for identifier detection in the target object image and improving the comprehensiveness of the effective information contained in the target region image. This ensures the accuracy of the determined similarity value when subsequently analyzing the template and target region images to obtain their similarity values, thus improving the accuracy of the identifier detection results. Furthermore, the identical size of the template and target region images further enhances the accuracy of identifier detection results when performing identifier detection based on the similarity analysis results of the template and target region images.

[0107] In this embodiment of the invention, further, as Figure 3 As shown, step 102a above may specifically include the following steps 102a1 and 102a2:

[0108] Step 102a1: Determine the second coordinate information based on the first coordinate information of the template image;

[0109] Step 102a2: Crop the target object image according to the second coordinate information to obtain the first region image.

[0110] In this embodiment, during the process of cropping the first region image to be calibrated from the target object image based on the image information of the template image, specifically, the marker detection device acquires the first coordinate information of the template image and expands the acquired first coordinate information to obtain second coordinate information with a larger coordinate region. Further, the marker detection device then crops the target object image based on the expanded second coordinate information to obtain a first region image with an image size larger than the template image. This reduces the impact of positional fluctuations caused by different imaging device models on the coordinate information of the template image and the target object image, thereby reducing the loss of effective information in the target object image during image cropping and improving the comprehensiveness of the effective information contained in the target region image. When subsequently analyzing the template image and the target region image to obtain their similarity values, the accuracy of the determined similarity values ​​can be guaranteed, thus improving the accuracy of the marker detection results.

[0111] The first coordinate information is used to indicate the position coordinates of the template image in the coordinate system of the shooting device, such as the camera coordinate system, when the template image is captured. Specifically, the first coordinate information may include the maximum horizontal coordinate value, the minimum horizontal coordinate value, the maximum vertical coordinate value, and the minimum vertical coordinate value of the template image.

[0112] Furthermore, the aforementioned second coordinate information is used to indicate the position coordinates of the first region image to be cropped in the target object image. Specifically, the second coordinate information may include the maximum horizontal coordinate value, minimum horizontal coordinate value, maximum vertical coordinate value, and minimum vertical coordinate value of the first region image.

[0113] Furthermore, in practical applications, the second coordinate information can be determined using the following formulas (2) to (7):

[0114]

[0115]

[0116]

[0117]

[0118] w t =x tmax -x tmin (6)

[0119] h t =y tmax -y tmin(7)

[0120] Where, x cmax The x-coordinate value represents the maximum x-coordinate value of the first region of the image. cmin The minimum x-coordinate value of the first region image is represented by y. cmax The y-coordinate value represents the maximum vertical coordinate of the first region of the image. cmin The minimum ordinate value of the first region image, x tmax The x-coordinate value represents the maximum x-coordinate value of the template image. tmin The minimum x-coordinate value of the template image, y tmax The maximum y-coordinate value of the template image. tmin w represents the minimum ordinate value of the template image. t h represents the width value of the template image. t This represents the height of the template image, and r is an extended parameter that can be set by those skilled in the art according to the actual situation.

[0121] In this embodiment of the invention, further, as Figure 4 As shown, step 104 above may specifically include steps 104a to 104d as follows:

[0122] Step 104a: Normalize the template image and the target region image;

[0123] Step 104b: Based on the target feature extraction network, extract the feature vectors of the template image and the target region image to obtain the second feature vector matrix and the first feature vector matrix;

[0124] Step 104c: Calculate the similarity distance between the second eigenvector matrix and the first eigenvector matrix;

[0125] Step 104d: Determine the similarity value based on the similarity distance.

[0126] In this embodiment, during the process of the identifier detection device performing similarity analysis on the template image and the target region image through a target feature extraction network to obtain the similarity value between them, specifically, the identifier detection device normalizes the template image and the target region image to obtain a standard pattern template image and target region image. Further, the identifier detection device then performs feature extraction on the normalized template image and target region image through the target feature extraction network. Specifically, the identifier detection device extracts a first feature vector of the target region image in multiple dimensions to obtain a first feature vector matrix of the target region image; further, the identifier detection device extracts a second feature vector of the template image in multiple dimensions to obtain a second feature vector matrix of the template image. Further, based on the extracted second feature vector matrix and the first feature vector matrix, the identifier detection device calculates the similarity distance between the second feature vector matrix and the first feature vector matrix in multiple dimensions, and then determines the similarity value between the template image and the target region image based on the calculated similarity distance. In this way, by extracting image feature vectors through a target feature extraction network, similarity analysis is performed on the template image and the target region image to obtain their similarity value. On the one hand, this ensures the accuracy of the similarity value determination, thereby ensuring the accuracy of the label detection results. On the other hand, based on the similarity value, a unified threshold detection for labels is achieved, which is applicable to the detection of multiple types of unknown labels, increasing the application scenarios and eliminating the need to collect a large number of samples, thus improving the label detection efficiency.

[0127] The aforementioned similarity value indicates the degree of similarity between the template image and the target region image. The similarity value is directly proportional to the degree of similarity; a higher similarity value indicates a greater similarity between the template image and the target region image, while a lower similarity value indicates a lower similarity. Thus, by comparing this similarity value with a set target threshold, the labeling status within the target area of ​​the target object can be detected, thereby detecting whether there are any issues such as missing, excessive, upside-down, incorrect, crooked, or damaged labels within the target area.

[0128] Furthermore, during the normalization process of the marker detection device on the template image and the target region image, specifically, the marker detection device can normalize the template image and the target region image using a set image mean and image variance, thereby converting the template image and the target region image into a standard pattern image. The aforementioned image mean and image variance can specifically be parameter values ​​calculated based on a massive amount of images.

[0129] Furthermore, in practical applications, the aforementioned target feature extraction network can specifically be a dual-path CNN feature extraction network. The CNN feature extraction network is based on a ResNet network, such as ResNet18, for feature extraction. It removes the fully connected layers of the ResNet18 network and uses the new weights formed by transfer learning of the weights pre-trained on ImageNet as the final feature extraction weights. Additionally, during the transfer training of the CNN feature extraction network, the aforementioned formula (1) can be used as the loss function for network transfer training.

[0130] In practical applications, those skilled in the art can also use other feature extraction networks to extract features from the template image and the target region image, without making any specific limitations here.

[0131] Furthermore, the aforementioned similarity distance can specifically be the Euclidean distance between the second eigenvector matrix and the first eigenvector matrix in multiple dimensions. In practical applications, the Euclidean distance between the second eigenvector matrix and the first eigenvector matrix can be calculated using the following formula (8):

[0132]

[0133] Where d(t,s) represents the Euclidean distance between two feature vectors in n dimensions, s1 represents the first feature vector of the target region image in the first dimension, t1 represents the second feature vector of the template image in the first dimension, s2 represents the first feature vector of the target region image in the second dimension, t2 represents the second feature vector of the template image in the second dimension, and s n t represents the first feature vector of the target region image in the nth dimension. n s represents the second feature vector of the template image in the nth dimension. i t represents the first feature vector of the target region image in the i-th dimension. i This represents the second feature vector of the template image in the i-th dimension, where n represents the image dimension, which can be a value such as 512 or 1024, and is not specifically limited here.

[0134] In practical applications, the aforementioned similarity distance can also be Manhattan distance, Chebyshev distance, Mahalanobis distance, Hamming distance, etc. Those skilled in the art can choose the aforementioned similarity distance determination algorithm according to the actual situation, without making specific restrictions here.

[0135] Furthermore, in the process of determining the similarity value between the template image and the target region image based on the calculated similarity distance, since the calculated similarity distance is not easy to intuitively display and analyze, this application sets a normalization function to map the calculated similarity distance to the range (0, 1), thereby obtaining a similarity value that can intuitively represent the degree of similarity between the template image and the target region image.

[0136] In this embodiment of the invention, the similarity value and the similarity distance are inversely proportional.

[0137] In this embodiment, the similarity distance between the second feature vector matrix of the template image and the first feature vector of the target region image is inversely proportional to the similarity value between the template image and the target region image. The larger the similarity distance, the higher the similarity value, and the smaller the similarity distance, the lower the similarity value.

[0138] In this embodiment of the invention, the similarity value is further inversely proportional to the similarity distance as an exponential function of the target value, where the target value is the base of the natural logarithm.

[0139] In this embodiment, the similarity value between the template image and the target region image is inversely proportional to the exponential function of the similarity distance with respect to the target value.

[0140] Here, the aforementioned target value is the base of the natural logarithm. That is, the similarity value between the template image and the target region image is related to e. x They are inversely proportional, where x is the similarity distance between the second feature vector matrix of the template image and the first feature vector of the target region image.

[0141] In this embodiment of the invention, the relationship between the similarity value and the similarity distance is further defined as follows: Where sim represents the similarity value and x represents the similarity distance.

[0142] In this embodiment, during the process of determining the similarity value between the template image and the target region image based on the calculated similarity distance, since the calculated similarity distance is not easy to intuitively display and analyze, this application sets a normalization function to map the calculated similarity distance to the range (0, 1), thereby obtaining a similarity value that can intuitively represent the degree of similarity between the template image and the target region image.

[0143] In practical applications, the above normalization function, that is, the mapping relationship between the similarity value and the similarity distance, can be specifically expressed by the following formula (9):

[0144]

[0145] Where sim represents the similarity value between the template image and the target region image, and x represents the similarity distance between the second feature vector matrix of the template image and the first feature vector of the target region image.

[0146] Based on this, the mapping between the aforementioned similarity values ​​and similarity distances can be specifically described as follows: Figure 10 As shown in the figure. The similarity distance is greater than or equal to 0, the similarity value is between 0 and 1, and the similarity value increases with the increase of the similarity distance. The larger the similarity distance, the higher the similarity value, and the smaller the similarity distance, the lower the similarity value.

[0147] In this embodiment of the invention, further, as Figure 5 As shown, step 106 above may specifically include steps 106a and 106b as follows:

[0148] Step 106a: If the target threshold is less than the similarity value, the identification detection result within the target area is determined to be qualified;

[0149] Step 106b: If the target threshold is greater than or equal to the similarity value, the identification detection result within the target area is determined to be unqualified.

[0150] In this embodiment, during the process of the label detection device detecting the label pasting status within a target area based on the comparison result between a set target threshold and a determined similarity value, specifically, if the set target threshold is less than the aforementioned similarity value, the label detection device determines that the label pasting status within the target area is normal, that is, the target area successfully passes the label detection. Conversely, if the set target threshold is greater than or equal to the aforementioned similarity value, the label detection device determines that the label pasting status within the target area is abnormal, that is, the target area fails the label detection. Thus, by detecting the label pasting status within the target area based on the comparison result between the similarity value and the set target threshold, a method of detecting label pasting status based on a unified threshold is achieved. This method is applicable to the detection of multiple types of unknown labels, increasing application scenarios and eliminating the need to collect a large number of samples, thereby improving label detection efficiency and ensuring the accuracy of the detection results.

[0151] The target threshold can be a value such as 0.5, 0.55, 0.6, or 0.65. In practical applications, those skilled in the art can set the specific value of the target threshold according to the actual situation, and no specific restrictions are imposed here.

[0152] In this embodiment of the invention, further, as Figure 6 As shown, after step 106 above, the above-mentioned identification detection method may further include the following steps 108 and 110:

[0153] Step 108: Display similarity values ​​and the image of the target object;

[0154] Step 110: Display the target identifier on the target object image based on the identifier detection results;

[0155] Among them, the target identifier is used to indicate the identifier detection results within the target area.

[0156] In this embodiment, after the label detection device completes the detection of the label pasting status of the target area of ​​the target object, it will also display the detection results to the user in a visual form so that the user can intuitively understand the label pasting status in the target area and make timely adjustment measures when the label pasting status in the target area is abnormal.

[0157] Specifically, in the label detection method proposed in this invention, after the label detection device completes the detection of the label pasting status within the target area of ​​the target object, it displays the aforementioned similarity value and an image of the target object to the user. Furthermore, the label detection device also displays a target label on the target object image based on the label detection results for the target area. This target label indicates whether the label pasting within the target area is normal or abnormal. This allows the user to intuitively understand the label pasting status within the target area, enabling timely adjustments when abnormal label pasting occurs.

[0158] Furthermore, in practical applications, the aforementioned target markings can be represented as squares, rectangles, or special symbols such as asterisks, without specific limitations. Based on this, the labeling status within the target area can be displayed by adjusting the display format of the target markings, such as display color and brightness.

[0159] In this embodiment of the invention, further, as Figure 7 As shown, step 110 above may specifically include steps 110a and 110b as follows:

[0160] Step 110a: If the identification detection result is qualified, display the target identification on the target object image according to the first display mode;

[0161] Step 110b: If the identification detection result is unqualified, display the target identification on the target object image according to the second display mode;

[0162] The display position of the target identifier corresponds to the second coordinate information.

[0163] In this embodiment, during the process of the label detection device displaying the target label on the target object image based on the label detection results of the target area, specifically, if the label detection device determines that the label pasting situation in the target area is normal, the label detection device displays the target label on the target object image according to a first display mode; if the label detection device determines that the label pasting situation in the target area is abnormal, the label detection device displays the target label on the target object image according to a second display mode. The target label in the first display mode indicates that the target area has successfully passed the label detection, and the target label in the second display mode indicates that the target area has failed the label detection. In this way, by displaying target labels in different display modes, the label detection results of the target area are visually presented to the user, allowing the user to intuitively understand the label pasting situation in the target area and thus take timely adjustment measures when the label pasting situation in the target area is abnormal.

[0164] The aforementioned target markings can specifically take the form of squares, rectangles, or special symbols such as asterisks. The display mode of the target markings can specifically include the display color, brightness, and line thickness, without specific limitations. For example, if the marking detection device determines that the labeling within the target area is normal, a green rectangle will appear on the target object image; conversely, if the marking detection device determines that the labeling within the target area is abnormal, a red rectangle will appear on the target object image.

[0165] Furthermore, the display position of the aforementioned target identifier corresponds to the second coordinate information. That is, the aforementioned target identifier is displayed on the first region image of the target object image. When the aforementioned target identifier is a rectangular frame, the rectangular frame can be displayed in the boundary area of ​​the first region image to present the identifier detection result to the user more intuitively.

[0166] The following example illustrates the identifier detection method proposed in this invention, using a target threshold of 0.5:

[0167] For example, such as Figure 11As shown, when the label is correctly affixed within the target area of ​​the target object, the label detection device acquires the target object image 302. Based on the image information of the first template image 304, it crops the first region image 306 from the target object image 302. Then, based on the size information of the first template image 304, it cuts the target region image 308 to be detected from the first region image 306. Further, the label detection device performs a similarity analysis on the target region image 308 and the first template image 304, obtaining a similarity value of 0.987. Since 0.987 is greater than 0.5, the label detection device determines that the label is correctly affixed within the target area. While displaying the target object image 302 and the similarity value of 0.987 to the user, it also displays the target label, i.e., the first label 310, in the first display mode.

[0168] Or, such as Figure 12 As shown, when the label is upside down in the target area, the label detection device sequentially acquires the first area image 306 and the target area image 308 based on the target object image 302, and then calculates the similarity value between the target area image 308 and the first template image 304 to be 0.000. Since 0.000 is less than 0.5, the label detection device determines that the label is abnormally pasted in the target area. While displaying the target object image 302 and the similarity value of 0.000 to the user, it also displays the target label in the second display mode, namely the second label 312.

[0169] Or, such as Figure 13 As shown, when the label is damaged within the target area, the label detection device sequentially acquires the first area image 306 and the target area image 308 based on the target object image 302. It then calculates the similarity value between the target area image 308 and the first template image 304 to be 0.275. Since 0.275 is less than 0.5, the label detection device determines that the label application within the target area is abnormal and displays the target object image 302, the similarity value 0.275, and the second label 312 to the user.

[0170] Or, such as Figure 14 As shown, when the label is misaligned within the target area, the label detection device sequentially acquires the first region image 306 and the target region image 308 based on the target object image 302. It then calculates the similarity value between the target region image 308 and the first template image 304 to be 0.420. Since 0.420 is less than 0.5, the label detection device determines that the label is misaligned within the target area and displays the target object image 302, the similarity value 0.420, and the second label 312 to the user.

[0171] Or, such as Figure 15As shown, if a label is misplaced within the target area, and the misplaced label is similar to the label that should have been affixed, the label detection device, based on the target object image 302, sequentially acquires the first area image 306 and the target area image 308, and then calculates the similarity value between the target area image 308 and the first template image 304 to be 0.379. Since 0.379 is less than 0.5, the label detection device determines that the label affixing within the target area is abnormal and displays the target object image 302, the similarity value 0.379, and the second label 312 to the user.

[0172] Or, such as Figure 16 As shown, if a label is misplaced within the target area, and the misplaced label differs significantly from the correct label, the label detection device, based on the target object image 302, sequentially acquires the first region image 306 and the target region image 308. It then calculates the similarity value between the target region image 308 and the first template image 304 to be 0.002. Since 0.002 is less than 0.5, the label detection device determines that the label placement within the target area is abnormal and displays the target object image 302, the similarity value 0.002, and the second label 312 to the user.

[0173] Or, such as Figure 17 As shown, in the case of missing labels within the target area, the label detection device sequentially acquires the first area image 306 and the target area image 308 based on the target object image 302, and then calculates the similarity value between the target area image 308 and the first template image 304 to be 0.037. Since 0.037 is less than 0.5, the label detection device determines that the label pasting within the target area is abnormal and displays the target object image 302, the similarity value 0.037, and the second label 312 to the user.

[0174] Or, such as Figure 18 As shown, when multiple labels are affixed within the target area, the label detection device sequentially acquires the first area image 306 and the target area image 308 based on the target object image 302. It then calculates the similarity value between the target area image 308 and the second template image 314 to be 0.000. Since 0.000 is less than 0.5, the label detection device determines that there are abnormally affixed labels within the target area and displays the target object image 302, the similarity value of 0.000, and the second label 312 to the user.

[0175] In addition, in practical applications, the detection results of the markings in the target area can be displayed to the user intuitively by showing detection results, playing different audio, or displaying different colored lights. No specific limitations are imposed here. For example, a green light illuminates when the labels in the target area are properly pasted, and a red light illuminates when the labels are improperly pasted.

[0176] In this embodiment of the invention, further, as Figure 8 As shown, prior to step 102 above, the above-mentioned identification detection method may further include steps 112 to 118 as follows:

[0177] Step 112: Obtain the barcode information of the target object;

[0178] Step 114: Retrieve the template information for the target area based on the barcode information;

[0179] Step 116: If the template information is successfully retrieved, retrieve the template image;

[0180] Step 118: If template information retrieval fails, display a prompt message;

[0181] The prompt message is used to remind users to create a template image.

[0182] In this embodiment, before acquiring the target area image, the identification detection device scans the barcode on the target object to identify its barcode information. Then, based on the identified barcode information, it determines whether a template image of the target area is pre-stored in the storage space. Specifically, the identification detection device retrieves the template information of the target area based on the identified barcode information. If the template information is successfully retrieved, it indicates that a template image of the target area is pre-stored in the storage space. In this case, the identification detection device directly retrieves and uses this template image for subsequent identification detection. If the identification detection device fails to retrieve the template information, it indicates that a template image of the target area is not pre-stored in the storage space. In this case, the identification detection device displays a prompt to the user, reminding them to create a new template image, and then performs subsequent identification detection based on the newly created template image.

[0183] In this embodiment of the invention, the types of the identifiers to be detected are different within the target areas of different target objects. Based on this, step 106 may specifically include the following step 106c:

[0184] Step 106c: Based on the comparison results of similarity values ​​and target thresholds, determine the detection results for different types of identifiers to be detected within the target area.

[0185] In this embodiment, the types of identifiers to be detected differ within the target areas of different target objects. Based on this, during the identifier detection process in the target area, the detection results for different types of identifiers within the target area are determined by comparing the similarity values ​​of the template image and the target area image with a set target threshold. For example, the pasting status of different types of labels within the target area is detected based on the comparison results of the similarity values ​​of the template image and the target area image with a set target threshold. This achieves identifier detection based on a unified threshold, applicable to detecting multiple types of unknown identifiers, increasing application scenarios, and eliminating the need to collect a large number of samples, thereby improving identifier detection efficiency and ensuring the accuracy of detection results. Furthermore, automatically detecting identifiers on target objects based on the collected target object images reduces manual workload, further ensuring the accuracy and efficiency of detection results.

[0186] In summary, in the embodiments of the present invention, as follows: Figure 9 As shown, taking the identification detection via a client's electronic device as an example, the identification detection method described above may specifically include the following steps 202 to 226:

[0187] Step 202: Determine whether the scanning of the product barcode was successful. If yes, proceed to step 206; otherwise, proceed to step 204.

[0188] Step 204: The client displays "Scan failed" and a red light illuminates.

[0189] Step 206: The camera captures an image and simultaneously calls the MES to obtain the barcode model.

[0190] Step 208: Determine if a template corresponding to the barcode model exists. If yes, proceed to step 214; otherwise, proceed to step 210.

[0191] Step 210: The client displays "Please model" and a red light illuminates.

[0192] Step 212, complete target modeling;

[0193] Step 214, Image preprocessing;

[0194] Step 216, comparison and detection;

[0195] Step 218, calculate the similarity score;

[0196] Step 220: Determine whether the similarity score is greater than the threshold. If yes, proceed to step 224; otherwise, proceed to step 222.

[0197] Step 222: The client displays NG and a red light illuminates;

[0198] Step 224: The client displays "OK" and the green light illuminates.

[0199] Step 226: The detection data is transmitted to the system.

[0200] In step 218, the similarity score is the similarity value mentioned above, and in step 220, the threshold is the target threshold mentioned above.

[0201] Specifically, the client's electronic device is equipped with a camera and a tri-color LED. During the identification detection process, the device scans the barcode on the target object. If the scan fails, a scan failure message is displayed on the client, and the tri-color LED illuminates red. If the scan succeeds, the tri-color LED illuminates green, and while capturing an image of the target object through the camera on the electronic device, the device calls the MES (Manufacturing Execution System) to obtain the barcode model number of the barcode on the target object. Based on this, the electronic device then determines whether a template image corresponding to the target object exists based on the obtained barcode model number. If the template image exists, the image preprocessing process begins; if the template image does not exist, the tri-color LED illuminates red, and a prompt message is displayed on the client to prompt the user to create a template image. After modeling is completed, the image preprocessing process begins again.

[0202] Further, after obtaining a detection image of the same size as the template image through image preprocessing, a comparison detection is performed between the template image and the detection image to determine the similarity score between them. Based on this, the electronic device then determines whether the determined similarity score is greater than a set threshold. If yes, "OK" is displayed on the client, the three-color indicator lights turn green, and a green box is displayed at the label position of the target object image; if no, "NG" is displayed on the client, the three-color indicator lights turn red, and a red box is displayed at the label position of the target object image. Simultaneously, the similarity score is displayed on the client, and the detection data is transmitted to the MES system. At this point, the current detection process ends, and the next detection process begins.

[0203] In one embodiment of the present invention, a label detection device is also provided. For example... Figure 19 As shown, Figure 19 A structural block diagram of a tag detection device 400 according to an embodiment of the present invention is shown. Specifically, the tag detection device 400 may include an acquisition unit 402 and a processing unit 404.

[0204] The acquisition unit 402 is used to acquire a target region image in a target object image, wherein the target region image corresponds to the target region of the target object;

[0205] Processing unit 404 is used to determine the similarity value between the template image of the target region and the target region image based on the target feature extraction network;

[0206] The processing unit 404 is also used to determine the identification detection result within the target area based on the target threshold and the comparison result of similar values.

[0207] The identifier detection device 400 provided by the present invention includes an acquisition unit 402 and a processing unit 404. During identifier detection of a target object, the acquisition unit 402 acquires an image of the target object, and then acquires an image of the target region to be detected from the target object image. This target region image is the image of the target region of the target object. Further, the acquisition unit 402 acquires a template image of the target region of the target object. The processing unit 404 compares the template image and the target region image based on a comparison network. Specifically, the processing unit 404 performs similarity analysis on the template image and the target region image based on a target feature extraction network, thereby determining the similarity value between the two. Based on this, the processing unit 404 compares a set target threshold with the determined similarity value, and then determines the identifier detection result within the target region of the target object based on the comparison result. Thus, based on the target feature extraction network, the similarity value between the target region image to be detected and the template image in the target object is determined, and then the identifier detection result within the target region of the target object is determined based on the similarity value. In this way, target detection based on similarity values ​​of target regions achieves a method of detecting identifiers based on a unified threshold. This is applicable to the detection of multiple types of unknown identifiers, expanding application scenarios and eliminating the need to collect a large number of samples, thereby improving identifier detection efficiency and ensuring the accuracy of detection results. Furthermore, automatic identifier detection based on the acquired target object image reduces manual workload, further guaranteeing the accuracy and efficiency of detection results.

[0208] In practical applications, the aforementioned identifiers to be detected can specifically be product labels, barcodes containing candidate information on exam answer sheets, signature information, seal information, etc. The identifier detection device proposed in this invention can detect the affixing of labels within the target area of ​​a target object, the affixing of barcodes containing candidate information on exam answer sheets, and the detection of signature information, seal information, etc., in documents, without specific limitations.

[0209] The following explanation uses the application scenario of the mark to be detected as a product label as an example to illustrate the mark detection device proposed in this invention.

[0210] Specifically, when the mark to be detected is a product label, the target area can be the label pasting area in the target object, or it can be an area in the target object where pasting labels is prohibited, without specific restrictions.

[0211] Based on this, when the target area is a label pasting area, the aforementioned label detection device detects whether labels are correctly pasted within the target area. For example, it detects whether any labels are missing from the target area, and whether the labels already pasted in the target area are pasted upside down, incorrectly, crookedly, excessively, or damaged compared to the standard labels that should be pasted in the target area. When the target area is an area where label pasting is prohibited within the target object, the aforementioned label detection device determines whether any labels have already been pasted within the target area.

[0212] Furthermore, the aforementioned template image can be a template image of the target area of ​​the target object pre-stored in the storage space. During the detection of the label pasting status on the target object, the acquisition unit 402 can directly retrieve and use this template image. The aforementioned template image can also be a template image newly established by the label detection device during the detection process, without specific limitations.

[0213] Furthermore, the aforementioned similarity value is used to indicate the degree of similarity between the template image and the target region image. The similarity value is directly proportional to the degree of similarity; a larger similarity value indicates a higher degree of similarity between the template image and the target region image, while a smaller similarity value indicates a lower degree of similarity. Thus, by comparing this similarity value with a set target threshold, the labeling status within the target area of ​​the target object can be detected, thereby detecting whether there are any instances of missing, excessive, upside-down, incorrect, crooked, or damaged labels within the target area.

[0214] Furthermore, the aforementioned target threshold can specifically be a value such as 0.5, 0.55, 0.6, or 0.65. In practical applications, those skilled in the art can set the specific value of the aforementioned target threshold according to the actual situation, and no specific restrictions are imposed here.

[0215] Furthermore, in practical applications, the aforementioned target feature extraction network can specifically be a dual-path CNN feature extraction network. The CNN feature extraction network is based on a ResNet network, such as ResNet18, for feature extraction. It removes the fully connected layers of the ResNet18 network and uses the new weights formed by transfer learning of the weights pre-trained on ImageNet as the final feature extraction weights. Additionally, during the transfer training of the CNN feature extraction network, the aforementioned formula (1) can be used as the loss function for network transfer training.

[0216] In this embodiment, the acquisition unit 402 is further configured to: acquire a first region image from the target object image based on the image information of the template image; compare the first region image and the template image; acquire a target region image from the first region image based on the size information of the template image and the comparison result; wherein the size of the first region image is larger than the size of the target region image.

[0217] In this embodiment, the size of the first region image is larger than the size of the template image, and the size of the target region image is equal to the size of the template image.

[0218] In this embodiment, the acquisition unit 402 is further configured to: determine the second coordinate information based on the first coordinate information of the template image; and crop the target object image based on the second coordinate information to obtain the first region image.

[0219] In this embodiment, the processing unit 404 is further configured to: normalize the template image and the target region image; extract feature vectors from the normalized template image and the target region image according to the target feature extraction network to obtain a second feature vector matrix and a first feature vector matrix; calculate the similarity distance between the second feature vector matrix and the first feature vector matrix; and determine the similarity value based on the similarity distance.

[0220] In this embodiment, the similarity value and the similarity distance are inversely proportional.

[0221] In this embodiment, the similarity value is further inversely proportional to the similarity distance as an exponential function of the target value, where the target value is the base of the natural logarithm.

[0222] In this embodiment, the relationship between the similarity value and the similarity distance is further defined as follows: Where sim represents the similarity value and x represents the similarity distance.

[0223] In this embodiment, the processing unit 404 is further configured to: determine that the identification detection result in the target area is qualified when the target threshold is less than the similarity value; and determine that the identification detection result in the target area is unqualified when the target threshold is greater than or equal to the similarity value.

[0224] In this embodiment, the marker detection device 400 further includes a display unit 406, which is specifically used to: display similarity values ​​and a target object image; display a target marker on the target object image according to the marker detection result; wherein the target marker is used to indicate the marker detection result within the target area.

[0225] In this embodiment, the display unit 406 is further configured to: display the target identifier on the target object image according to a first display mode when the identifier detection result is qualified; and display the target identifier on the target object image according to a second display mode when the identifier detection result is unqualified; wherein the display position of the target identifier corresponds to the second coordinate information.

[0226] In this embodiment, the acquisition unit 402 is further configured to: acquire barcode information of the target object; the processing unit 404 is further configured to: retrieve template information of the target area based on the barcode information; and retrieve a template image if the template information is successfully retrieved; the display unit 406 is further configured to: display a prompt message if the template information retrieval fails; wherein the prompt message is used to remind the user to create a template image.

[0227] In this embodiment, the types of identifiers to be detected in the target areas of different target objects are different. The processing unit 404 is specifically used to: determine the detection results of different types of identifiers to be detected in the target area based on the comparison results of similarity value and target threshold.

[0228] A third aspect of the present invention provides a readable storage medium having a program or instructions stored thereon that, when executed by a processor, implements the steps of the identifier detection method as described in any of the above embodiments.

[0229] The readable storage medium provided in this embodiment of the invention, when the program or instructions stored therein are executed by a processor, can implement the steps of the identifier detection method as described in any of the above embodiments. Therefore, this readable storage medium possesses all the beneficial effects of the identifier detection method in any of the above embodiments, which will not be elaborated further here.

[0230] Specifically, the aforementioned readable storage medium can include any medium capable of storing or transmitting information. Examples of readable storage media include electronic circuits, semiconductor memory devices, read-only memory (ROM), random access memory (RAM), compact disc read-only memory (CD-ROM), flash memory, erasable ROM (EROM), magnetic tape, floppy disk, optical disk, hard disk, fiber optic media, radio frequency (RF) links, optical data storage devices, etc. Code segments can be downloaded via computer networks such as the Internet and intranets.

[0231] An embodiment of the fourth aspect of the present invention provides a computer program product comprising a computer program that, when executed by a processor, implements the identifier detection method as described in any of the above-described technical solutions. Therefore, the computer program product proposed in the fourth aspect of the present invention possesses all the beneficial effects of the identifier detection method in any of the technical solutions of the first aspect, which will not be elaborated further here.

[0232] In one embodiment of the invention, an electronic device is also provided. For example... Figure 20 As shown, Figure 20 A structural block diagram of an electronic device 500 provided in an embodiment of the present invention is shown. The electronic device includes:

[0233] Memory 502, which stores programs or instructions;

[0234] The processor 504 executes the above-described program or instructions to implement the steps of the identifier detection method as described in any of the above embodiments.

[0235] The electronic device 500 provided in this embodiment includes a memory 502 and a processor 504. When the program or instructions in the memory 502 are executed by the processor 504, they implement the steps of the identification detection method as described in any of the above embodiments. Therefore, the electronic device 500 has all the beneficial effects of the identification detection method in any of the above embodiments, which will not be repeated here.

[0236] Specifically, the memory 502 and the processor 504 can be connected via a bus or other means. The processor 504 may include one or more processing units, and the processor 504 may be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other chips.

[0237] In the description of this specification, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance, unless otherwise expressly specified and limited. The terms "connection," "installation," and "fixing," etc., should be interpreted broadly. For example, "connection" can mean a fixed connection, a detachable connection, or an integral connection; it can mean a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0238] In the description of this specification, the terms "one embodiment," "some embodiments," "specific embodiment," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0239] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0240] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of identification detection, characterized in that, include: Obtain a target region image from a target object image, wherein the target region image corresponds to the target region of the target object; Based on the target feature extraction network, the similarity value between the target region image and the template image of the target region is determined; Based on the comparison results between the similarity value and the target threshold, the identification detection result within the target area is determined; The step of determining the similarity value between the target region image and the template image of the target region based on the target feature extraction network includes: The target region image and the template image are normalized. Based on the target feature extraction network, feature vectors of the normalized target region image and the template image are extracted to obtain a first feature vector matrix and a second feature vector matrix; Calculate the similarity distance between the first eigenvector matrix and the second eigenvector matrix; The similarity value is determined based on the similarity distance; Among them, the template image and the target region image are normalized by setting the image mean and image variance; The step of obtaining the target region image in the target object image includes: Based on the image information of the template image, obtain the first region image from the target object image; Compare the template image and the first region image; Based on the comparison results and the size information of the template image, the target region image is obtained from the first region image; Wherein, the size of the first region image is larger than the size of the target region image, and the image information of the template image includes the coordinate system of the template image in the shooting device when the template image is captured.

2. The identification detection method according to claim 1, characterized in that, The size of the first region image is larger than the size of the template image, and the size of the target region image is equal to the size of the template image.

3. The identification detection method according to claim 1, characterized in that, The step of obtaining a first region image from the target object image based on the image information of the template image includes: Based on the first coordinate information of the template image, determine the second coordinate information; The first region image is obtained by cropping the target object image based on the second coordinate information.

4. The identification detection method according to claim 1, characterized in that, The similarity value and the similarity distance are inversely proportional.

5. The identification detection method according to claim 1, characterized in that, The similarity value is inversely proportional to the similarity distance as an exponential function of the target value, where the target value is the base of the natural logarithm.

6. The identification detection method according to claim 1, characterized in that, The relationship between the similarity value and the similarity distance is as follows: ； Where sim represents the similarity value and x represents the similarity distance.

7. The identification detection method according to claim 3, characterized in that, Determining the identifier detection result within the target area based on the comparison result of the similarity value and the target threshold includes: If the similarity value is greater than the target threshold, the identification detection result within the target area is determined to be qualified; If the similarity value is less than or equal to the target threshold, the identification detection result within the target area is determined to be unqualified.

8. The identification detection method according to claim 7, characterized in that, After determining the sign detection result within the target area, the sign detection method further includes: Display the image of the target object and the similarity value; Based on the identification detection results, a target identifier is displayed on the target object image, and the target identifier is used to indicate the identification detection results within the target area.

9. The identification detection method according to claim 8, characterized in that, The step of displaying the target identifier on the target object image based on the identifier detection result includes: If the identification detection result is qualified, the target identification in the first display mode is displayed on the target object image; If the identification detection result is unqualified, the target identification in a second display mode is displayed on the target object image; The display position of the target identifier corresponds to the second coordinate information.

10. The identification detection method according to any one of claims 1 to 9, characterized in that, Before acquiring the target region image in the target object image, the identifier detection method further includes: Obtain the barcode information of the target object; Retrieve template information for the target area based on the barcode information; If the template information is successfully retrieved, the template image is retrieved; If the template information retrieval fails, a prompt message will be displayed to remind the user to create the template image.

11. The identification detection method according to any one of claims 1 to 9, characterized in that, The types of identifiers to be detected differ within the target regions of different target objects. Determining the identifier detection result within the target region based on the comparison result of the similarity value and the target threshold includes: Based on the comparison results of the similarity value and the target threshold, the detection results of different types of identifiers to be detected within the target area are determined.

12. An identification detection device, characterized by include: An acquisition unit is used to acquire a target region image in a target object image, wherein the target region image corresponds to the target region of the target object; The processing unit is used to determine the similarity value between the target region image and the template image of the target region based on the target feature extraction network; The processing unit is further configured to determine the identifier detection result within the target area based on the comparison result between the similarity value and the target threshold; The processing unit is specifically used to normalize the target region image and the template image; extract feature vectors from the normalized target region image and the template image according to the target feature extraction network to obtain a first feature vector matrix and a second feature vector matrix; calculate the similarity distance between the first feature vector matrix and the second feature vector matrix; and determine the similarity value based on the similarity distance. Among them, the template image and the target region image are normalized by setting the image mean and image variance; The acquisition unit is configured to acquire a first region image from the target object image based on the image information of the template image; compare the template image and the first region image; and acquire the target region image from the first region image based on the comparison result and the size information of the template image; wherein the size of the first region image is larger than the size of the target region image, and the image information of the template image includes the coordinate system of the template image in the shooting device when the template image is captured.

13. A readable storage medium, characterized by, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the identifier detection method as described in any one of claims 1 to 11.

14. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the identifier detection method as described in any one of claims 1 to 11.

Images