An abnormal identification code identification method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of data processing, in particular to an abnormal identification code identification method and device, electronic equipment and a storage medium, the method comprising the following steps: identification code identification is performed on a to-be-detected image to obtain a corresponding identification code positioning result; when the identification code positioning result contains identification code positioning information of at least one to-be-identified identification code, an abnormal classification result of the to-be-detected image is obtained based on global image features of the to-be-detected image; when the abnormal classification result is that there is no abnormality, for each to-be-identified identification code, a corresponding sub-image is cropped out from the to-be-identified image, and image content at an edge of the to-be-identified identification code in the sub-image is analyzed to obtain a corresponding abnormal identification result. In this way, by means of comprehensive evaluation identification of multiple angles, effective abnormal identification results can be obtained according to rich image information, which greatly improves the identification efficiency and identification effect of abnormal identification codes.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, electronic device and storage medium for identifying anomaly identification codes. Background Technology

[0002] With the development of information technology, QR codes enable information acquisition, identity verification, and resource transfer. Therefore, to improve the security and authenticity of data processing based on QR codes, it is necessary to perform anomaly identification on QR codes to determine abnormal QR codes.

[0003] Currently, when identifying abnormal QR codes, a classification model or detection model is usually used to perform anomaly recognition processing on the image to be detected containing the QR code, and obtain the recognition result of whether the QR code contained in the image to be detected is an abnormal QR code.

[0004] However, since the above-mentioned abnormal QR code recognition process can only rely on a single-function model to perform a general recognition of the image to be detected, the image information on which it is based is very limited, making it difficult to guarantee the recognition effect of abnormal QR codes; moreover, it is very easy to miss abnormal QR codes, thus reducing the recognition efficiency of abnormal QR codes. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for identifying anomaly identification codes, in order to improve the identification effect of anomaly identification codes.

[0006] Firstly, a method for identifying anomaly identifier codes is proposed, including:

[0007] The image to be detected is identified by its identification code to obtain the corresponding identification code location result;

[0008] When it is determined that the identification code location result contains identification code location information of at least one identification code to be identified, the matching situation of the at least one identification code to be identified and the printing status is analyzed based on the global image features of the image to be detected, and the anomaly classification result of the image to be detected is obtained.

[0009] When the anomaly classification result is determined to be that no anomaly exists, the following identification operations are performed for each of the identifier codes to be identified:

[0010] Based on an identifier code location information, a sub-image containing the corresponding identifier code is cropped from the image to be identified, and the image content at the edge of the identifier code in the sub-image is analyzed to obtain the corresponding anomaly identification result.

[0011] Secondly, an anomaly identification code recognition device is proposed, comprising:

[0012] The acquisition unit is used to perform identification code recognition on the image to be detected and obtain the corresponding identification code positioning result;

[0013] The analysis unit is used to analyze the matching situation between the at least one identifier to be identified and the printing status based on the global image features of the image to be detected when it is determined that the identifier positioning result contains identifier positioning information of at least one identifier to be identified, thereby obtaining the abnormal classification result of the image to be detected.

[0014] The execution unit is configured to perform the following identification operations for each identification code location information when the anomaly classification result is determined to be that no anomaly exists:

[0015] Based on an identifier code location information, a sub-image containing the corresponding identifier code is cropped from the image to be identified, and the image content at the edge of the identifier code in the sub-image is analyzed to obtain the corresponding anomaly identification result.

[0016] Optionally, when the identification code location result contains identification code location information for multiple identification codes to be identified, before performing the identification operation on the identification code location information corresponding to each of the identification codes to be identified in the identification code location result, the analysis unit is further configured to:

[0017] Carrier identification is performed on the image to be detected to obtain the corresponding carrier positioning result; the carrier positioning result includes at least one carrier positioning information; each carrier positioning information is used to indicate and display the position of an identification code carrier associated with the identification code in the image to be detected;

[0018] Based on the carrier positioning result and the identification code positioning result, it is determined that each identification code carrier in the image to be detected displays an identification code to be identified.

[0019] Optionally, when determining, based on the carrier positioning result and the identification code positioning result, that each identification code carrier in the image to be detected displays an identification code to be identified, the analysis unit is used to:

[0020] Based on the carrier positioning results, determine the carrier region corresponding to each identification code carrier in the image to be detected, and based on the identification code positioning results, determine the identification code region corresponding to each identification code to be identified in the image to be detected.

[0021] Based on at least one carrier region and at least one identifier code region, the total number of identifier code carriers in the image to be detected is determined to be the same as the total number of identifier codes to be identified, and each carrier region contains one identifier code region.

[0022] Optionally, the analysis unit is further configured to:

[0023] When it is determined, based on the carrier positioning result and the identification code positioning result, that there is an abnormal identification code carrier displaying multiple identification codes in the image to be detected, the identification process of the abnormal identification code ends.

[0024] The image to be detected is determined to have an abnormal identification code, and the plurality of identification codes to be identified displayed on the abnormal identification code carrier are marked as abnormal identification codes.

[0025] Optionally, the carrier localization result is obtained by the analysis unit using a trained target carrier localization model; the training process of the target carrier localization model is as follows:

[0026] Obtain each first training sample; wherein, each first training sample includes: a carrier sample image, and a ground truth value of the carrier position labeled for the carrier sample image;

[0027] Using the first training samples, the constructed initial carrier localization model is trained in multiple rounds of iterations to obtain the trained target carrier localization model. In one round of training, the model parameters are adjusted based on the content difference between the carrier position prediction value output by the initial carrier localization model based on the read carrier sample image and the corresponding carrier position ground truth value.

[0028] Optionally, after determining that the identification code location result contains identification code location information of at least one identification code to be identified, and before analyzing whether the at least one identification code to be identified is in a printed state based on the global image features of the image to be detected to obtain the anomaly classification result of the image to be detected, the device further includes a determining unit, the determining unit being used for:

[0029] Based on the identification code positioning information of the at least one identification code to be identified, the rectangular side information corresponding to the at least one identification code to be identified is determined respectively;

[0030] For the at least one identifier to be identified, the following operations are performed respectively: based on the corresponding rectangle side information and the image size information of the image to be detected, a relative size result of an identifier to be identified compared with the image to be detected is determined, and the relative size result is determined not to exceed the corresponding set result threshold.

[0031] Optionally, the determining unit is further configured to:

[0032] When the relative size of a code to be identified relative to the image to be detected exceeds the corresponding set threshold, the identification process of the abnormal code ends.

[0033] The image to be detected is identified as having an abnormal identifier code, and a prompt message indicating that the image to be detected cannot be recognized is displayed.

[0034] Optionally, when cropping a sub-image containing the corresponding identification code from the image to be identified based on an identification code location information, the execution unit is used to:

[0035] Based on the location information of an identifier code, determine the region edge information of the corresponding identifier code to be identified;

[0036] In the image to be detected, the size of the region determined based on the region edge information is expanded according to a preset expansion ratio to determine the adjusted target region. The target region is then cropped from the image to be detected to obtain the corresponding sub-image.

[0037] Optionally, when expanding the size of the region determined based on the region edge information in the image to be detected according to a preset expansion ratio, the execution unit is used to:

[0038] In the image to be detected, the corresponding region size is determined based on the region edge information, and the region size is expanded according to a preset expansion ratio. While keeping the selected base positioning point position unchanged, the target region corresponding to the expanded target size is determined.

[0039] The basic positioning point can be any one of the following: the regional center point determined based on the regional edge information, and the regional edge point determined based on the regional edge information.

[0040] Optionally, the identifier code localization result is obtained by the obtaining unit using a trained target identifier code localization model; the training process of the target identifier code localization model is as follows:

[0041] Obtain each second training sample; wherein, each second training sample includes: an identifier sample image, and the identifier position ground value annotated for the identifier sample image;

[0042] Using the second training samples, the constructed initial identifier code localization model is trained in multiple rounds to obtain the trained target identifier code localization model. In one round of training, the model parameters are adjusted based on the content difference between the identifier code position prediction value output by the initial identifier code localization model based on the read identifier code sample image and the corresponding identifier code position ground truth value.

[0043] Optionally, the anomaly classification result is obtained by the analysis unit using a trained global target classification model; the training process of the global target classification model is as follows:

[0044] Obtain each third training sample; wherein, a third training sample includes: a sample image containing at least one sample identifier code, and a classification ground truth value labeled according to whether the at least one sample identifier code contains a sample identifier code of a printing state.

[0045] Using the aforementioned third training samples, the constructed initial global classification model is trained through multiple rounds of iteration to obtain the trained target global classification model. In each round of training, the model parameters are adjusted based on the classification difference between the classification prediction value output by the initial global classification model based on the read sample image and the corresponding true classification value.

[0046] Optionally, the anomaly identification result is obtained by the execution unit using a trained target local classification model; the training process of the target local classification model is as follows:

[0047] Obtain each fourth training sample; wherein, each fourth training sample includes: a sample sub-image containing a sample identifier code, and a classification ground truth value labeled for the sample sub-image; the sample identifier code in a sample sub-image is a single identifier code or a combination code obtained by superimposing multiple identifier codes;

[0048] Using the aforementioned fourth training samples, the constructed initial local classification model is trained through multiple rounds of iteration to obtain the trained target local classification model. In each round of training, the model parameters are adjusted based on the classification difference between the classification prediction value output by the initial global classification model based on the read sample sub-image and the corresponding true classification value.

[0049] Optionally, after obtaining the corresponding identification code location result, the execution unit is further configured to:

[0050] When the identification code location result does not contain the identification code location information of the identification code to be identified, the identification process of the abnormal identification code is directly terminated, the image to be detected is determined to have an abnormal identification code, and a prompt message indicating that the identification code to be identified was not found is displayed.

[0051] Thirdly, an electronic device is proposed, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-described method when executing the computer program.

[0052] Fourthly, a computer-readable storage medium is proposed, on which a computer program is stored, which, when executed by a processor, implements the above-described method.

[0053] Fifthly, a computer program product is proposed, comprising a computer program that, when executed by a processor, implements the above-described method.

[0054] The beneficial effects of this application are as follows:

[0055] This application proposes a method, apparatus, electronic device, and storage medium for identifying abnormal identification codes. It proposes identifying identification codes in an image to be inspected to obtain corresponding identification code location results. Furthermore, when the identification code location results contain identification code location information for at least one identification code to be identified, based on the global image features of the image to be inspected, the matching situation between at least one identification code to be identified in the image and the printing status is analyzed to obtain an abnormality classification result for the image to be inspected. This enables the determination of whether an abnormality exists in the image to be inspected, based on the matching situation between at least one identification code to be identified and the printing status, even when it is certain that an identification code to be identified exists in the image to be inspected.

[0056] Subsequently, when the anomaly classification result indicates no anomaly exists, the following recognition operations are performed for each identifier location information: based on identifier location information, a sub-image containing the corresponding identifier is cropped from the image to be recognized, and the image content at the edge of the identifier in the sub-image is analyzed to obtain the corresponding anomaly recognition result. Thus, after determining that the identifier in the image to be detected does not contain the printed state from the perspective of matching with the printing state, further refined recognition can be performed on each identifier to be recognized, thereby obtaining the corresponding anomaly recognition result for each identifier. Based on this, in the anomaly identifier recognition process proposed in this application, various analyses are performed progressively based on the image to be detected, enabling comprehensive anomaly recognition of the image to be detected from the perspectives of the existence of the identifier, matching with the printing state, and individual anomaly results, thereby reducing the reliance on recognition from a single anomaly recognition perspective. Furthermore, by leveraging the comprehensive evaluation and recognition from multiple anomaly recognition perspectives, effective anomaly recognition results can be obtained based on rich image information, which greatly improves the recognition efficiency and effectiveness of anomaly identifiers. Attached Figure Description

[0057] Figure 1 This is a schematic diagram illustrating possible application scenarios in the embodiments of this application;

[0058] Figure 2 This is a schematic diagram illustrating the identification process of the anomaly identifier code in an embodiment of this application;

[0059] Figure 3 This is a schematic diagram illustrating the process of obtaining the identification code location result in the embodiments of this application;

[0060] Figure 4 This is a schematic diagram illustrating the presence of the identifier code to be identified within the image to be detected in an embodiment of this application;

[0061] Figure 5 This is a schematic diagram of the annotation results for each sample image in the embodiments of this application;

[0062] Figure 6 This is a schematic diagram illustrating the annotation effect on the carrier area in an embodiment of this application;

[0063] Figure 7 This is a schematic diagram illustrating a feasible process for expanding the target region in an implementation of this application.

[0064] Figure 8 This is a schematic diagram illustrating another feasible process for obtaining the target region in an embodiment of this application;

[0065] Figure 9 This is a schematic diagram illustrating another feasible process for obtaining the target region in an embodiment of this application;

[0066] Figure 10 This is a schematic diagram illustrating the process of labeling each sample sub-image in an embodiment of this application;

[0067] Figure 11 This is a schematic diagram illustrating the identification process of an abnormal QR code in an embodiment of this application;

[0068] Figure 12 This is a schematic diagram illustrating the identification process of another abnormal QR code in an embodiment of this application;

[0069] Figure 13 This is a schematic diagram of the logic structure of the identification device for the abnormal identifier code in the embodiments of this application;

[0070] Figure 14 This is a schematic diagram of the hardware structure of an electronic device using an embodiment of this application;

[0071] Figure 15 This is a schematic diagram of the hardware structure of another electronic device using an embodiment of this application. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this application. Obviously, the described embodiments are only some embodiments of the technical solutions of this application, and not all embodiments. Based on the embodiments recorded in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the technical solutions of this application.

[0073] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in sequences other than those illustrated or described herein.

[0074] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0075] The following explanations of some terms used in the embodiments of this application are provided to facilitate understanding by those skilled in the art.

[0076] A QR code (Quick Response Code) is a matrix-style barcode that can store and transmit various types of data. Moreover, QR codes enable identity verification, payment, and information retrieval.

[0077] Identification code: refers to a graphic symbol that can be identified to obtain various stored information; for example, identification code can specifically refer to barcode, or QR code, or various graphic symbols that store information; in addition, it should be noted that identification code has the characteristics of uniqueness, readability, and structure.

[0078] The design concept of the embodiments of this application is briefly introduced below:

[0079] In modern retail, QR codes are widely used as an important technological tool for various scenarios such as payment, identity verification, and information transmission. However, with the increasing prevalence of QR codes, the identification of abnormal QR codes has become particularly important to ensure the security and authenticity of data processing.

[0080] Currently, some feasible methods for identifying abnormal QR codes involve using classification models. In this case, normal and abnormal QR codes are first divided into two different categories, and then a classification model based on a binary classification algorithm is constructed. Specifically, the classification model can be built using any model structure such as Support Vector Machine (SVM) or Decision Tree. Then, during the specific model training process, the classification algorithm is used to learn and extract features to distinguish between normal and abnormal QR codes based on the QR code image data, and the extracted features are used for classification processing.

[0081] Other feasible implementations involve using detection models. In this case, the problem of identifying abnormal QR codes is treated as a special target detection task, and normal and abnormal QR codes are considered as two different detection categories. In actual processing, image processing techniques and machine vision algorithms are first used to analyze the image features of the QR codes in the image, and then the abnormality of the QR code is determined based on the extracted image features. Moreover, in order to optimize the accuracy and recall of detection, specific thresholds can be set to improve the accuracy of recognition.

[0082] However, in the process of identifying abnormal QR codes, whether using a classification model or a detection model, only a general identification of the image to be detected can be performed. The image information is very limited, making it difficult to guarantee the recognition effect of abnormal QR codes. Therefore, frequent model updates are required, which greatly increases the training cost of the model. Moreover, it is difficult to accurately identify abnormal QR codes in complex recognition scenarios, which can easily lead to missed detections of abnormal QR codes and reduce the recognition efficiency of abnormal QR codes.

[0083] In view of this, this application proposes a method, apparatus, electronic device, and storage medium for identifying abnormal identification codes, and proposes to identify the identification codes of the image to be detected to obtain the corresponding identification code location results; furthermore, when the identification code location results contain identification code location information of at least one identification code to be identified, based on the global image features of the image to be detected, the matching situation of at least one identification code to be identified in the image to be detected with the printing status is analyzed to obtain the abnormal classification result of the image to be detected; this can determine whether there is an abnormality in the image to be detected from the perspective of the matching situation of at least one identification code to be identified with the printing status, even if it is determined that there is an identification code to be identified in the image to be detected.

[0084] Subsequently, when the anomaly classification result indicates no anomaly exists, the following recognition operations are performed for each identifier location information: based on identifier location information, a sub-image containing the corresponding identifier is cropped from the image to be recognized, and the image content at the edge of the identifier in the sub-image is analyzed to obtain the corresponding anomaly recognition result. Thus, after determining that the identifier in the image to be detected does not contain the printed state from the perspective of matching with the printing state, further refined recognition can be performed on each identifier to be recognized, thereby obtaining the corresponding anomaly recognition result for each identifier. Based on this, in the anomaly identifier recognition process proposed in this application, various analyses are performed progressively based on the image to be detected, enabling comprehensive anomaly recognition of the image to be detected from the perspectives of the existence of the identifier, matching with the printing state, and individual anomaly results, thereby reducing the reliance on recognition from a single anomaly recognition perspective. Furthermore, by leveraging the comprehensive evaluation and recognition from multiple anomaly recognition perspectives, effective anomaly recognition results can be obtained based on rich image information, which greatly improves the recognition efficiency and effectiveness of anomaly identifiers.

[0085] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0086] See Figure 1 The diagram shown illustrates a possible application scenario in an embodiment of this application. This application scenario diagram includes a processing device 110 and a server 120.

[0087] In some feasible embodiments of this application, the processing device 110 can independently implement the identification process of the abnormal identification code; in other feasible embodiments of this application, the processing device 110 and the server 120 can jointly implement the identification process of the abnormal identification code, for example, the server 120 can implement the abnormal identification from some abnormal identification angles.

[0088] When the processing device 110 independently identifies the anomaly identification code, it can acquire the image to be detected, process it using the identification method claimed in this application, obtain the anomaly identification result, and mark the anomaly of the identification code to be identified in the image based on the anomaly identification result. Furthermore, the processing device 110 can send the anomaly identification result back to the server 120 for storage, enabling other devices to access the server 120 and obtain the anomaly identification result.

[0089] In a feasible implementation, after the processing device obtains the anomaly identification result, if it determines that there is an anomaly identification code in the image to be detected, the processing device 110 can notify the relevant processing personnel to handle the anomaly.

[0090] The processing device 110 includes, but is not limited to, mobile phones, tablets, laptops, e-book readers, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, various electronic devices capable of scanning identification codes (such as identification code scanners), and other electronic devices capable of processing functions.

[0091] Server 120 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0092] In this embodiment, the processing device 110 and the server 120 can communicate via a wired network or a wireless network. The following description focuses only on the identification process of the anomaly identifier from the perspective of the processing device 110.

[0093] The following is an illustrative explanation of the anomaly identification code recognition process, based on possible application scenarios:

[0094] Application Scenario 1: Identifying abnormal identification codes in retail scenarios.

[0095] In offline retail scenarios, it is necessary to identify abnormal identification codes to avoid problems with the identification codes on product packaging.

[0096] Specifically, the processing device can acquire a photographed image containing at least one product identification code based on the operation instructions of the target object (such as a service personnel in a retail scenario); then, using the identification method claimed in this application, obtain an abnormal identification result for the identification code in the image to be identified; subsequently, the processing device can display the abnormal identification result to the target object so that the target object can identify the identification code that has been identified as an abnormal identification code, and identify the product corresponding to the abnormal identification code.

[0097] Furthermore, the processing device can send the anomaly identification results obtained from the image to be detected to the server, so that other devices can access the server to determine the products with abnormal identification codes on the packaging; or, after determining that a product has an abnormal identification code, the processing device can send the anomaly identification results obtained from the image to be detected to the management device associated with the product, so that the management personnel using the management device can check the same type of products.

[0098] Application Scenario 2: Identifying abnormal identification codes in identification code scanning scenarios.

[0099] Scanning identification codes is involved in scenarios such as offline payment, logistics information traceability, and identity information verification.

[0100] In this case, the processing device can acquire a captured image containing the identification code to be identified before scanning the identification code for complex processing; then, it can use the identification method claimed in this application to obtain an abnormal identification result of the identification code to be identified in the image to be identified.

[0101] Furthermore, in offline payment processes, the processing device continues to execute the payment process based on the payment QR code when it confirms that the payment QR code is valid; or, in logistics information traceability scenarios, the processing device continues to execute the traceability process based on the logistics QR code when it confirms that the logistics QR code is valid; or, in identity information verification scenarios, the processing device continues to execute the verification process based on the identity QR code when it confirms that the identity QR code is valid, such as verifying travel tickets or concert tickets.

[0102] Application Scenario 3: Identifying anomaly codes in information acquisition scenarios.

[0103] In feasible implementation scenarios, web page addresses can be encoded into information QR codes, so that after the information QR code is scanned, the corresponding web page can be directly accessed and the information content on the web page can be obtained; or, publicly available information configured by relevant objects can be encoded into information QR codes, so that after the information QR code is scanned, the publicly available information configured by relevant objects can be obtained, such as the business card information of relevant objects.

[0104] Based on this, the processing device can acquire a captured image containing the information QR code before recognizing the information in the QR code; then, using the recognition method claimed in this application, obtain an abnormal recognition result of the information QR code in the image to be detected; and then, when the processing device determines that the information QR code is normal, it continues to execute the information acquisition process based on the information QR code.

[0105] In addition, it should be understood that the specific implementation of this application involves the identification process of abnormal identification codes. When the embodiments described in this application are applied to specific products or technologies, the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0106] The following, with reference to the accompanying diagram, explains the process of identifying the anomaly code from the perspective of the processing equipment:

[0107] See Figure 2 As shown, this is a schematic diagram of the identification process of the abnormal identifier code in an embodiment of this application. The following is a detailed explanation in conjunction with the attached diagram. Figure 2 The process of identifying the anomaly identifier code is explained below:

[0108] Step 201: The processing device performs identification code recognition on the image to be detected and obtains the corresponding identification code positioning result.

[0109] In this embodiment of the application, after the processing device acquires the image to be detected, it performs identification code recognition on the image to be detected and obtains the corresponding identification code positioning result. The image to be detected may be directly acquired by the processing device with the help of an associated image acquisition component; or the image to be detected may be acquired by the image acquisition device and then obtained by the processing device from the image acquisition device. This application does not impose any specific restrictions on this.

[0110] It should be noted that, in the embodiments of this application, the obtained identification code positioning result may contain identification code positioning information of at least one identification code to be identified; or, it may not contain identification code positioning information of the identification code to be identified; that is, with the help of the identification code positioning result, it is possible to determine whether the image to be detected contains the identification code to be identified; the identification code positioning information is used to indicate the pixel position of the identification code area corresponding to the identification code to be identified in the image to be detected.

[0111] Depending on the actual processing needs, an identifier location information may specifically indicate a rectangular region in the image to be detected that contains an identifier to be identified. In this case, an identifier location information may specifically correspond to the pixel positions of the four vertices of the rectangular region in the image to be detected, or an identifier location information may specifically correspond to the pixel positions of the two diagonal points of the rectangular region in the image to be detected. Furthermore, in this embodiment, the resolution of the image to be detected is not limited.

[0112] In a feasible implementation of this application, the obtained identifier code localization result can be obtained by processing a trained target identifier code localization model. The target identifier code localization model can be obtained after training an initial identifier code localization model. The initial identifier code localization model can be constructed based on any one or a combination of the following architectures: Residual Convolutional Network (RCN), Regional Convolutional Network (RCN), single-stage object detection algorithm (You Only Look Once, YOLO), multi-stage object detection framework (Cascade Region-based Convolutional Neural Network, Cascade RCNN), accelerated region-based convolutional neural network (Faster Region-based Convolutional Neural Network, Faster R-CNN), and other model structures capable of achieving object detection functionality.

[0113] It should be understood that, in the embodiments of this application, the processing device can train corresponding target identifier code localization models for different identifier code types, or it can train a general target identifier code localization model for different identifier code types. Based on this, the target identifier code localization model used to obtain the identifier code localization result can be a target identifier code localization model corresponding to a specific identifier code type, or it can be a general target identifier code localization model for all identifier code types.

[0114] When different target identifier code localization models are trained for different identifier code types, and the image to be detected is processed according to the target identifier code localization model, the processing device can obtain the matching target identifier code localization model for processing based on the target identifier code type indicated by the relevant object; optionally, the processing device can be configured with an initial identifier code type by default.

[0115] For example, the processing device can present a setting page for setting the identification code type based on the setting instructions of the relevant object, and then determine the target identification code type to be identified based on the selection instructions of the relevant object on the setting page; then, it can obtain the target identification code positioning model that matches the target identification code type for processing, wherein the relevant object may specifically be the management personnel who instruct the anomaly identification.

[0116] For example, assuming the target identifier type selected by the relevant object in the settings page is a QR code, the obtained target identifier positioning model can specifically achieve QR code positioning.

[0117] When a universal target identifier localization model is trained for different identifier types, the processing device can directly use the trained target identifier localization model to obtain the identifier localization result based on the image to be detected.

[0118] In addition, it should be understood that whether a target identifier code localization model is trained separately for different identifier code types or a target identifier code localization model that is universal for different identifier code types is trained, the same training logic can be used for model training. Specifically, an initial identifier code localization model can be constructed based on the same model structure, and then each second training sample can be constructed according to training needs. Based on each second training sample, the initial identifier code localization model is trained to obtain the target identifier code localization model.

[0119] In this embodiment of the application, the target identification code localization model can be used to achieve target detection of the identification code to be identified. Taking the identification code to be identified as a QR code as an example, the target identification code localization model trained in a targeted manner can locate the QR code region in the image to be detected. This localization process involves using advanced image processing algorithms to analyze various parts of the image to be detected, so as to accurately identify the specific location and boundary of the QR code.

[0120] In the specific process of training the target identifier code localization model, the processing device can first acquire each second training sample; each second training sample includes: an identifier code sample image, and the ground truth value of the identifier code position labeled for the identifier code sample image; then, using each second training sample, the constructed initial identifier code localization model is trained in multiple rounds to obtain the trained target identifier code localization model. In one round of training, the model parameters are adjusted according to the content difference between the identifier code position prediction value output by the initial identifier code localization model based on the read identifier code sample image and the corresponding ground truth value of the identifier code position.

[0121] It should be noted that, in this embodiment, the predicted identifier location value is used to indicate the identifier region detected for the sample identifier in the identifier sample image; the true identifier location value is specifically used to indicate the identifier region marked in the identifier sample image; the identifier region can be indicated by using a rectangular box; or by using a mask image, wherein the mask image is the same size as the identifier sample image, and the pixel value corresponding to the pixel point within the identifier region is 1, and the pixel value corresponding to the pixel point outside the identifier region is 0; the rectangular box in the identifier sample image can be obtained from the mask image.

[0122] Furthermore, the specific format of the predicted position value and the true value of the identifier position can be the pixel position of each identifier positioning point determined for the identifier region. Taking a rectangular region as an example, it can be the pixel position of the four vertices of the rectangular region in the identifier sample image; or, it can be the pixel position of the two diagonal points of the rectangular region in the identifier sample image; or, it can be the pixel position of each identifier positioning point selected for locating the rectangular region and of a fixed number; wherein, each identifier positioning point can be located on the edge of the rectangular region, selected according to the actual processing needs.

[0123] Furthermore, the convergence condition for the trained target identifier code localization model can be: the total number of training rounds reaches a first preset value; or, the number of times the model loss value is continuously lower than a second preset value reaches a third preset value. The values ​​of the first, second, and third preset values ​​are set according to actual processing needs, and this application does not impose specific restrictions on them. The model loss value obtained during one round of iterative training is calculated based on the content difference between the predicted identifier code position output by the initial identifier code localization model and the corresponding true identifier code position. The model parameters can be adjusted based on the model loss value. The loss function used to calculate the model loss value is set according to actual processing needs. For example, the loss function can be any one or a combination of the following: cross-entropy loss function, regression loss function.

[0124] In this way, by training the initial identifier localization model, the resulting target identifier localization model can learn the ability to locate the identifier to be identified from the image to be identified, thereby effectively locating the identifier to be identified from complex image backgrounds.

[0125] For example, see Figure 3 As shown, it is a schematic diagram of the process of obtaining the identification code positioning result in an embodiment of this application. According to the appendix Figure 3 As shown in the diagram, assuming the identifier to be identified is a QR code, after specifically training a target identifier localization model, when processing the image to be detected, the image can be input into the target identifier localization model to obtain the identifier localization result; moreover, using the identifier localization information in the identifier localization result, it is possible to obtain the attached... Figure 3 The two identification code areas are shown in the diagram.

[0126] Step 202: When it is determined that the identification code positioning result contains identification code positioning information of at least one identification code to be identified, the processing device analyzes the matching situation of at least one identification code to be identified with the printing status based on the global image features of the image to be detected, and obtains the anomaly classification result of the image to be detected.

[0127] In some feasible embodiments of this application, after the processing device obtains the identification code location result, if the identification code location information does not contain the identification code to be identified, the identification process of the abnormal identification code can be directly terminated, and the image to be detected can be determined to contain an abnormal identification code. Optionally, a prompt message indicating that no identification code to be identified was found can be displayed.

[0128] Specifically, when the processing device determines that there is no identification code to be identified in the image to be detected, it can determine that there is no object in the image to be detected that needs to be identified; then, the processing device can directly end the identification process of the abnormal identification code based on the image to be detected according to the actual processing needs.

[0129] In this way, by directly ending the identification process of the abnormal identification code and displaying a prompt message that no identification code was found, the reason for ending the abnormal identification based on the image to be detected can be shown, thereby achieving efficient processing of the image to be detected; moreover, by directly determining that the image to be detected contains an abnormal identification code, it is possible to avoid introducing additional judgment results in the identification process of the abnormal identification code, thereby simplifying the abnormal identification process.

[0130] In other feasible embodiments of this application, after the processing device obtains the identification code positioning result, when it determines that the identification code positioning result contains identification code positioning information of at least one identification code to be identified, the processing device may optionally determine the relative size result of the identification code to be identified in the image to be detected, and determine whether to continue the anomaly recognition process based on the obtained relative size result and the corresponding set result threshold. Furthermore, the processing device may obtain the anomaly classification result of the image to be detected based on the matching situation between at least one identification code to be identified in the image to be detected and the printing status.

[0131] When the processing device determines whether to continue the anomaly identification process based on the relative size result of the identification code to be identified in the image to be detected, after determining that the identification code location result contains the identification code location information of at least one identification code to be identified, and before analyzing and obtaining the anomaly classification result of the image to be detected, it determines the rectangular side information corresponding to at least one identification code to be identified based on the identification code location information of at least one identification code to be identified in the image to be detected; and then, for at least one identification code to be identified, it performs the following operations respectively: according to the corresponding rectangular side information and the image size information of the image to be detected, it determines the relative result size of an identification code to be identified compared with the image to be detected, and determines that the relative result size does not exceed the corresponding set result threshold.

[0132] The relative size result can refer to the ratio of the long side of the rectangular region containing the identification code to the long side of the image to be detected; or, the relative size result can refer to the ratio of the area of ​​the rectangular region containing the identification code to the area of ​​the image to be detected.

[0133] For example, see Figure 4 As shown, this is a schematic diagram illustrating the presence of the identifier to be identified within the image to be detected in an embodiment of this application. Assuming the resolution of the image to be detected is 1920×1080, and the rectangular side information of the rectangular region determined for the identifier to be identified in the image to be detected is 42×42. Therefore, when using the ratio of the longer side as the relative size result, the determined relative size result is 42 / 1920; and when using the area ratio as the relative size result, the determined relative size result is 42*42 / (1920*1080).

[0134] In this way, by determining the relative size of each identification code in the image to be detected, the size of the identification code in the image to be detected can be effectively measured, thereby effectively determining the shooting effect of the identification code in the image to be detected.

[0135] It should be understood that, in feasible embodiments of this application, after obtaining the corresponding relative size result for each identifier to be identified, when it is determined that the relative size result of an identifier to be identified relative to the image to be detected exceeds the corresponding set result threshold, the processing device can end the identification process of the abnormal identifier; at the same time, the image to be detected is determined to have an abnormal identifier, and a prompt message indicating that the image to be detected cannot be identified is displayed.

[0136] Specifically, when the processing device determines the size of each identification code to be identified in the image to be detected, it can directly end the identification process of abnormal identification codes based on the image to be detected if the relative size result of any identification code to be identified in the image to be detected exceeds the corresponding set result threshold. In order to simplify the judgment result, the image to be detected is judged to have abnormal identification codes.

[0137] In addition, in this embodiment of the application, by displaying a prompt message indicating that the image to be detected cannot be recognized, it is possible to prompt the reacquisition of the image to be detected so as to perform anomaly recognition based on a more effective image to be detected.

[0138] In this way, by comparing the relative size of the identification code to be identified with the corresponding result threshold, it is possible to detect in a timely manner the situation where the identification code to be identified in the image to be detected is captured too large, thus losing reference information. This allows the processing of the image to be detected to be terminated in a timely manner, avoiding invalid processing.

[0139] Furthermore, in the process of obtaining anomaly classification results based on the image to be detected, the processing device may employ different procedures depending on whether the image contains one or more identification codes. The following explanation illustrates the process of obtaining anomaly classification results based on the image to be detected, using examples of identification code localization results representing the presence of one and multiple identification codes:

[0140] Processing Step 1: The image to be detected contains an identifier code to be identified.

[0141] In a feasible implementation of this application, the processing device can use a trained global target classification model to classify the image to be detected and obtain anomaly classification results.

[0142] In some feasible implementations, after other devices have trained and obtained the target global classification model, the processing device can directly obtain the target global classification model; in other feasible implementations, the processing device can train and obtain the target global classification model on its own; the following only takes the processing device training and obtaining the target global classification model on its own as an example to illustrate the relevant training process.

[0143] In order to train the target global classification model, the processing device first needs to build an initial global classification model. The initial global classification model can be built based on any one or a combination of Residual Network (ResNet) series networks and other network structures that can achieve classification functions.

[0144] It should be noted that, in the embodiments of this application, a corresponding global target classification model can be specifically trained for an image to be detected containing a single identifier; or, a corresponding global target classification model can be specifically trained for an image to be detected containing multiple identifiers; or, a general global target classification model can be trained for an image to be detected containing one or more identifiers. This application does not impose any specific limitations on this.

[0145] Alternatively, to meet the processing needs of different identification code types, the processing device can train corresponding global classification models for different identification code types, based on the number of identification codes to be identified; or, it can train a general global classification model for different identification code types and different numbers of identification codes to be identified.

[0146] However, it should be understood that although at least one global classification model may be trained for different training objectives, the training logic for the global classification model is the same. It is only necessary to construct each third training sample differently according to different training objectives.

[0147] During the training process to obtain the target global classification model, the processing device can acquire each third training sample. Each third training sample includes: a sample image containing at least one sample identifier, and a classification ground value labeled according to whether the at least one sample identifier contains a sample identifier indicating a printed state. Then, using each third training sample, the constructed initial global classification model is trained iteratively in multiple rounds to obtain the trained target global classification model. In each round of training, the model parameters are adjusted based on the classification difference between the classification prediction value output by the initial global classification model based on the read sample image and the corresponding classification ground value.

[0148] It should be noted that, in some feasible implementations of constructing each third training sample, where only the target global classification model needs to output the overall global classification result of the image to be detected from the perspective of whether the identifier is in a printed state, the ground truth value for constructing the classification values ​​in each third training sample can be set as follows: sample images containing identifiers in a printed state can be labeled as having an anomaly, and the corresponding ground truth value can be set to 0; and sample images where the corresponding identifiers are all in a non-printed state can be labeled as not having an anomaly, and the corresponding ground truth value can be set to 1. In other feasible implementations, where the target global classification model needs to output the classification result corresponding to each identifier in the image to be detected from the perspective of whether the identifier is in a printed state, the processing device can label each identifier in the sample image separately when constructing the ground truth value in a third training sample. Identifiers in a printed state in the sample image can be labeled as an anomaly, and the corresponding ground truth value can be set to 0; and identifiers in a non-printed state in the sample image can be labeled as not having an anomaly, and the corresponding ground truth value can be set to 1.

[0149] In addition, when adjusting model parameters during a training round, the cross-entropy loss function can be used to calculate the model loss value based on the classification prediction value and the corresponding true classification value, and then the model parameters can be adjusted based on the model loss value.

[0150] The convergence condition for training the target global classification model can be: the total number of training rounds reaches a fourth preset value; or the number of times the model loss value is continuously lower than a fifth preset value reaches a sixth preset value; wherein, the values ​​of the fourth, fifth, and sixth preset values ​​are set according to the actual processing needs, and this application does not impose specific restrictions on them.

[0151] Alternatively, in this embodiment, during the systematic identification of abnormal identification codes, the model can be updated in real time. Training samples can be reconstructed based on newly emerging cheating methods using these codes, and the model can be updated again. This allows the system to continuously learn and adapt to new cheating techniques, maintaining the anti-cheating system's foresight and effectiveness, and achieving effective identification of abnormal identification codes. Furthermore, through multi-angle and multi-environment data training, the interference of environmental variables can be added to the training samples, enhancing the system's stability and reliability under different usage environments.

[0152] In a feasible implementation of this application, for the identified printing or non-printing state, the target global classification model can be understood as being specifically designed to identify and classify several common anomaly types: black and white printing on A4 paper and color printing; optionally, when classifying anomalies based on whether it is a printing state, the state of the identification code carrier where the identification code to be identified is located can also be combined for classification. In this case, the anomaly types that the target global classification model can identify also include: the identification code to be identified existing on the torn identification code carrier. In other words, the identification code to be identified existing on the torn identification code carrier can be regarded as the identification code in the printing state.

[0153] For example, see Figure 5 As shown, it is a schematic diagram of the annotation results of each sample image in the embodiments of this application. According to the appendix Figure 5 As shown in the diagram, if QR code processing is required, then when annotating each sample image, the image can be labeled according to whether the QR code in the sample image is printed; sample images containing printed QR codes are labeled as abnormal, and sample images not containing printed QR codes are labeled as non-existent abnormal.

[0154] In this way, by training the initial global classification model, the resulting target global classification model has the ability to classify anomalies in the image to be detected from the perspective of the matching between the identifier code to be identified and the printing status.

[0155] Furthermore, when processing based on the target global classification model, the target global classification model uses advanced image processing technology to comprehensively evaluate the texture, color depth and other key features of the image (such as surrounding reference information) to perform effective anomaly classification. This allows for the output of anomaly classification results by accurately analyzing the input image to be detected.

[0156] This not only improves the accuracy of identification but also greatly speeds up the processing and enables accurate identification of identification code fraud through means such as identification code printing, thus achieving effective anomaly classification.

[0157] Processing step two: The image to be detected contains multiple identifiers to be identified.

[0158] In feasible embodiments of this application, when the image to be detected contains multiple identification codes, considering that under normal circumstances one identification code belongs to one identification code carrier, anomaly determination can optionally be made from the perspective of the consistency between the identification code and the identification code carrier. Specifically, after obtaining the identification code location result, anomaly identification of the image to be detected can be performed according to an arbitrarily set processing order, from the perspective of whether there is a printed identification code and from the perspective of the consistency between the identification code and the identification code carrier.

[0159] The anomaly determination is made first from the perspective of the consistency between the identification code to be identified and the identification code carrier, and then from the perspective of the matching with the printing status. The premise of the anomaly determination from the perspective of the matching with the printing status is that no anomaly is identified from the perspective of the consistency between the identification code to be identified and the identification code carrier, so that the consistency between the identification code to be identified and the identification code carrier can be guaranteed in terms of quantity and position.

[0160] Specifically, the processing device can perform carrier identification on the image to be detected and obtain the corresponding carrier positioning results; wherein, the carrier positioning results include at least one carrier positioning information; each carrier positioning information is used to indicate the position of an identification code carrier with an associated identification code in the image to be detected; and then, based on the carrier positioning results and the identification code positioning results, it is determined that each identification code carrier in the image to be detected displays an identification code to be detected.

[0161] In this embodiment of the application, the carrier localization result can be obtained by processing the trained target carrier localization model. When the target carrier localization model is trained, the processing device can acquire each first training sample. Each first training sample includes: a carrier sample image and a carrier position ground value labeled for the carrier sample image. Then, each first training sample is used to perform multiple rounds of iterative training on the constructed initial carrier localization model to obtain the trained target carrier localization model. In one round of training, the model parameters are adjusted according to the content difference between the carrier position prediction value output by the initial carrier localization model based on the read carrier sample image and the corresponding carrier position ground value.

[0162] The true value of the carrier position is determined based on the carrier region marked in the carrier sample image. The carrier region's identification code region can be indicated by marking with a bounding box of a specified shape, or by marking in the form of a mask image, wherein the mask image is the same size as the carrier sample image, and the pixel value corresponding to the pixel point within the carrier region is 1, and the pixel value corresponding to the pixel point outside the carrier region is 0. The bounding box in the carrier sample image can be obtained from the mask image.

[0163] Moreover, the content format of the carrier location prediction value and the carrier location truth value can be the pixel position of each carrier site selected on the annotation box of the carrier region.

[0164] For example, see Figure 6 As shown, this is a schematic diagram of the annotation effect for the carrier area in an embodiment of this application. Figure 6 When annotating the illustrated carrier sample image, it is necessary to annotate the carrier region of each sample carrier in order to determine the true value of the carrier position.

[0165] In addition, the initial carrier localization model can be built based on any one or a combination of the following architectures: RCN, YOLO, Cascade RCNN, Faster R-CNN, and other model structures that can achieve object detection.

[0166] The convergence condition for training the target carrier localization model can be: the total number of training rounds reaches the seventh preset value; or the number of times the model loss value is continuously lower than the eighth preset value reaches the ninth preset value; wherein, the values ​​of the seventh, eighth, and ninth preset values ​​are set according to the actual processing needs, and this application does not impose specific restrictions on them; during one round of iterative training, a loss function can be used to calculate the model loss value based on the content difference between the predicted carrier position value and the corresponding true carrier position value, and then the model parameters are adjusted based on the model loss value; wherein, the loss function used to calculate the model loss value is set according to the actual processing needs, such as any one or combination of the following: cross-entropy loss function, regression loss function.

[0167] In this way, by training the initial carrier localization model, the resulting target carrier localization model can learn the ability to locate the identification code carrier from the image to be identified, thereby effectively locating the identification code carrier from complex image backgrounds.

[0168] Furthermore, with the help of the trained target carrier localization model, the quantity and location information of different types of identification code carriers can be accurately obtained in the image to be detected. Moreover, when the target carrier localization model is used for processing, the model uses advanced image analysis technology to analyze the texture, shape, color and other visual features of the image, which can achieve accurate scanning of various carriers in the image to be detected, and identify and locate each identification code carrier containing the identification code to be identified.

[0169] Moreover, by adding interference to each of the first training samples in the model training, such as the size, angle, and partial occlusion of the sample carrier, the robustness of the target carrier localization model can be improved.

[0170] Based on this, when determining that each identification code carrier in the image to be detected displays an identification code to be identified based on the carrier positioning results and the identification code positioning results, the processing steps involved are as follows: based on the carrier positioning results, determine the carrier region corresponding to each identification code carrier in the image to be detected, and based on the identification code positioning results, determine the identification code region corresponding to each identification code to be identified in the image to be detected; then, based on the determined at least one carrier region and at least one identification code region, determine that the total number of identification code carriers in the image to be detected is the same as the total number of identification codes to be identified, and that each carrier region contains one identification code region.

[0171] It should be noted that, in the embodiments of this application, when it is determined that the total number of identification code carriers in the image to be detected is the same as the total number of identification codes of the identification codes to be identified, and each carrier region contains an identification code region, it can be determined that, from the perspective of the consistency between the identification codes to be identified and the identification code carriers, no abnormality is identified in the image to be detected.

[0172] In this way, by using the carrier localization results output by the target carrier localization model, the position of the identification code carrier in the image to be detected can be determined, as well as the total number of identification code carriers in the image to be detected. This allows for the comparison of the identification code to be identified and the identification code carrier from both the perspectives of quantity and region location, ensuring the consistency between the identification code carrier and the identification code to be identified.

[0173] Specifically, in this embodiment of the application, when it is determined, based on the carrier positioning result and the identification code positioning result, that there is an abnormal identification code carrier displaying multiple identification codes to be identified in the image to be detected, the identification process of the abnormal identification code can be terminated; at the same time, the image to be detected is determined to have an abnormal identification code, and the multiple identification codes to be identified displayed on the abnormal identification code carrier are marked as abnormal identification codes.

[0174] Specifically, when a carrier region contains multiple identification code regions based on the carrier positioning results and the identification code positioning results, the identification code carrier corresponding to that carrier region can be identified as an abnormal identification code carrier, and it can be determined that there is an abnormal identification code carrier displaying multiple identification codes in the image to be detected.

[0175] This allows for the timely identification of anomalies where multiple identification codes exist on a single identification code carrier, thus promptly ending the identification process for the abnormal identification codes. This is crucial for maintaining the effectiveness and efficiency of the identification process. Furthermore, by utilizing the carrier location results and identification code location results, the specific abnormal identification code carrier can be identified, providing a basis for subsequent anomaly handling. This enables relevant personnel to address the anomalies of the abnormal identification code carrier in a targeted manner when handling anomalies.

[0176] Furthermore, when determining anomalies based on the matching status with the printing state, the processing device can use the trained global target classification model to process the image to be detected. The method of training the global target classification model is the same as the training logic involved in Process 1, except that the constructed third training samples need to be adaptively adjusted according to the actual processing needs. This application does not impose specific restrictions on this.

[0177] In summary, combining the illustrative anomaly identification angles of Process 1 and Process 2, it can be seen that in feasible embodiments of this application, the anomaly identification angles involved in steps 201-202 include the presence of the identifier to be identified in the image to be detected, its matching with the printing status, its relative size, and the consistency between the identifier to be identified and the identifier carrier. Based on actual processing needs, the anomaly identification order currently illustrated in steps 201-202 under various anomaly evaluation angles is only illustrative. It should be understood that since anomaly identification under different anomaly identification angles is performed independently, the execution order of processing under different anomaly identification angles can be determined according to actual processing needs. Moreover, the premise for performing anomaly identification under one anomaly identification angle is that no anomaly was identified under the previous anomaly identification angle.

[0178] In feasible embodiments of this application, different processing logic can be used to perform anomaly identification on images to be detected containing a single identifier code and multiple identifier codes, or the same processing logic can be used to perform anomaly identification on images to be detected containing a single identifier code and multiple identifier codes. The above processing procedure one and processing procedure two are only illustrative examples.

[0179] In feasible embodiments of this application, the processing device can provide different recognition modes to process images to be detected that contain a single identification code or multiple identification codes. For example, according to the instructions of relevant personnel, the recognition mode of using a single identification code is determined, or according to the instructions of relevant personnel, the recognition mode of using multiple identification codes is determined.

[0180] For example, in the recognition process for an image to be detected containing a code to be identified, a target global classification model can be used to make a preliminary anomaly judgment from the perspective of "matching with the printing state". If the image to be detected is determined to be free of anomalies based on the anomaly classification result, it can be determined that at least one code to be identified is in a non-printing state. That is, it is determined that the image does not contain a code to be identified in a printing state. Then, anomaly recognition can continue from the perspective of "existence". If the code to be identified is detected, it can be determined that no anomaly has been identified from the perspective of "existence". Then, by calculating the relative size result, anomaly recognition can be performed from the perspective of "relative size". When the relative size result is lower than the set result threshold, it is determined that no anomaly has been identified from the perspective of "relative result". Then, anomaly recognition can continue from the anomaly recognition perspective involved in step 203.

[0181] For example, in the recognition process configured for an image to be detected containing one or more identification codes, anomaly recognition can first be performed from the perspective of "presence". If the identification code to be recognized is detected based on the identification code location result, it can be determined that no anomaly is recognized from the perspective of "presence". Then, by calculating the relative size result, anomaly recognition is performed from the perspective of "relative size". When each relative size result is lower than the set result threshold, it can be determined that no anomaly is recognized from the perspective of "relative result". Next, anomaly recognition is performed from the perspective of "matching with printing status". If the image to be detected is determined to be free of anomalies based on the anomaly classification result, it can be determined that at least one identification code to be recognized is in a non-printing state, and it can be determined that no anomaly is recognized from the perspective of "matching with printing status". Then, based on the identification code location result, if it is determined that the number of identification codes to be recognized is 1, anomaly recognition can continue from the anomaly recognition perspective involved in step 203. If the number of identification codes to be identified exceeds 1, anomaly identification can be performed from the perspective of consistency between the identification code to be identified and the identification code carrier. If no anomaly is identified from the perspective of consistency between the identification code to be identified and the identification code carrier, anomaly identification can continue from the anomaly identification perspective involved in step 203.

[0182] Step 203: When the anomaly classification result is determined to be that there is no anomaly, the processing device performs the following recognition operations for the identification code location information of each identification code to be identified: Based on the identification code location information, a sub-image containing the corresponding identification code to be identified is cropped from the image to be identified, and the image content at the edge of the identification code to be identified in the sub-image is analyzed to obtain the corresponding anomaly recognition result.

[0183] In this embodiment of the application, when the anomaly classification result indicates that an anomaly exists, it can be determined that the image to be detected contains a printable identification code. In this case, the processing device can directly end the identification process for the anomaly identification code and directly determine that the image to be detected contains an anomaly identification code.

[0184] Conversely, when the anomaly classification result is that there is no anomaly, it indicates that at least one identification code to be identified in the image to be detected is in a non-printed state, or in other words, the image to be detected does not contain an identification code to be identified in a printed state. In this case, the processing device can perform anomaly detection on each identification code to be identified in the image to be detected and obtain the corresponding anomaly identification result.

[0185] When performing anomaly detection for each identification code to be identified, the processing device first needs to separate each identification code from the image to be detected.

[0186] Taking the process of separating a single identifier code to be identified as an example, the processing device can determine the region edge information of a corresponding identifier code based on the identifier code positioning information in the identifier code positioning result; then, in the image to be detected, the size of the region determined based on the region edge information is expanded according to a preset expansion ratio to determine the expanded target region, and the target region is cropped out in the image to be detected to obtain the corresponding sub-image.

[0187] Among them, the identification code positioning information is used to indicate the position of the identification code region of the identification code to be identified in the image to be detected; the region edge information is used to describe the size of each edge of the identification code region, that is, to determine the region size of the identification code region; in a feasible implementation, the shape of the identification code region is usually the same as the shape of the image to be detected.

[0188] Taking the image to be detected as a rectangle as an example, if the identification code area is a rectangle, then the identification code positioning information is used to indicate the position of the identification code area of ​​the rectangle in the image to be detected, thereby determining the regional edge information of the identification code area, and thus determining the length of the four rectangular sides.

[0189] When expanding to obtain the target area, the processing device can determine the corresponding area size in the image to be detected based on the area edge information, and expand the area size according to the preset expansion ratio. While keeping the selected base positioning point position unchanged, the device can determine the target area corresponding to the expanded target size. The base positioning point can be any one of the following: the area center point determined based on the area edge information, and the area edge point determined based on the area edge information.

[0190] It should be noted that when expanding the target area based on the identifier code area, the position of the basic positioning point can remain unchanged, and the target area can be expanded outward based on the identifier code area; the value of the expansion ratio is set according to the actual processing needs, and this application does not impose specific restrictions on it; moreover, the edge of the expanded target area and the identifier code area may be parallel or not parallel.

[0191] For example, see Figure 7 As shown, this is a schematic diagram of a feasible process for expanding to obtain the target region in an implementation of this application. According to the appendix... Figure 7 As illustrated, assuming the base positioning point is the center point and the expansion ratio is 0.5, then, based on the identification code positioning information, after determining the region edge information of the identification code area in the image to be detected, assuming each side length of the identification code area is 'a', then after expansion, each side length of the target area is 1.5a. Furthermore, while keeping the center point unchanged, the target area can be expanded outward so that the center point of the target area overlaps with the center point of the identification code area, wherein the region edges of the target area and the corresponding identification code area are parallel.

[0192] For example, see Figure 8 As shown, this is a schematic diagram illustrating another feasible process for expanding the target region in an embodiment of this application. According to the appendix... Figure 8 As illustrated, assuming the base positioning point is the center point and the expansion ratio is 0.5, after determining the region edge information of the identification code area in the image to be detected based on the identification code positioning information, and assuming each side of the identification code area is 'a', then after expansion, each side of the target area is 1.5a. Therefore, while keeping the center point unchanged, the expansion can achieve the desired results. Figure 8 The target area is shown in the diagram, where the edge of the target area is not parallel to the edge of the area to be identified.

[0193] For example, see Figure 9 As shown, this is a schematic diagram of another feasible process for expanding to obtain the target area in an embodiment of this application. According to the appendix... Figure 9As illustrated, assuming the base positioning point is a vertex selected on the edge of the identification code region, and the expansion ratio is 0.5, then after determining the region edge information of the identification code region in the image to be detected based on the identification code positioning information, assuming each side of the identification code region is 'a', then after expansion, each side of the target region is 1.5a. Therefore, while keeping the position of the base positioning point unchanged, the target region can be expanded outward, so that the vertices at the corresponding positions in the target region overlap with the vertices at the corresponding positions in the identification code region.

[0194] In this way, by using the selected base positioning points, the expansion direction of the identification code area can be effectively limited.

[0195] In addition, it should be understood that in the embodiments of this application, a sub-image obtained by cropping includes only one identifier code to be identified; moreover, each cropping is based on the original image to be detected, according to the actual processing needs.

[0196] In this way, by expanding the target region based on the identifier code region and cropping the sub-image corresponding to the target region, the integrity of the cropped identifier code to be identified can be guaranteed, thereby improving the analysis effect for a single identifier code to be identified.

[0197] Furthermore, when processing a single sub-image to obtain anomaly recognition results, the trained target local classification model can be used for processing.

[0198] When training the target local classification model, the processing device first acquires each fourth training sample. Each fourth training sample includes: a sample sub-image containing a sample identifier code, and a classification ground truth value labeled for the sample sub-image. The sample identifier code in a sample sub-image is a single identifier code or a combination code obtained by superimposing multiple identifier codes. Then, using each fourth training sample, the constructed initial local classification model is trained in multiple rounds of iteration to obtain the trained target local classification model. During each round of training, the model parameters are adjusted based on the classification difference between the classification prediction value output by the initial global classification model based on the read sample sub-image and the corresponding classification ground truth value.

[0199] In this embodiment of the application, in order to train the target local classification model, the processing device first needs to construct an initial local classification model. The initial global classification model can be constructed based on ResNet series networks, as well as any one or combination of other network structures that can achieve classification functions.

[0200] The processing device can train corresponding target local classification models for different identification code types; or, it can train a general target local classification model for different identification code types. This application does not impose any specific restrictions on this.

[0201] In addition, when adjusting model parameters during a training round, the cross-entropy loss function can be used to calculate the model loss value based on the classification prediction value and the corresponding true classification value, and then the model parameters can be adjusted based on the model loss value.

[0202] The convergence condition for training the target local classification model can be: the total number of training rounds reaches the tenth preset value; or the number of times the model loss value is continuously lower than the eleventh preset value reaches the twelfth preset value; wherein, the values ​​of the tenth, eleventh, and twelfth preset values ​​are set according to the actual processing needs, and this application does not impose specific restrictions on them.

[0203] Furthermore, in this embodiment, during the systematic identification of abnormal identification codes, the model can be updated in real time. Training samples are reconstructed based on newly emerging cheating methods using these codes, and the model is updated again. This allows the system to continuously learn and adapt to new cheating techniques, maintaining the anti-cheating system's foresight and effectiveness, and achieving effective identification of abnormal identification codes. In addition, through multi-angle and multi-environment data training, the interference of environmental variables can be added to the training samples, enhancing the system's stability and reliability under different usage environments.

[0204] See Figure 10 As shown, this is a schematic diagram of the process of annotating each sample sub-image in an embodiment of this application. According to the appendix... Figure 10 As shown in the diagram, assuming the goal of training the target local anomaly detection model is to identify matting and texturing as anomalies, then after cropping a sample sub-image containing a sample identifier, the corresponding classification ground truth can be labeled based on whether the sample identifier in the sample sub-image has matting or texturing. A classification ground truth value of 1 indicates that the sample identifier does not have matting or texturing, i.e., it is classified as: no anomaly (or normal); a classification ground truth value of 1 indicates that the sample identifier has matting or texturing, i.e., it is classified as: anomaly.

[0205] In this way, by training the initial local classification model, the resulting target local classification model has the ability to classify sub-images as anomalies based on whether the identifier code to be identified has been pasted or cut out.

[0206] In this embodiment, the target local classification model can effectively classify two types of aberrations in the identification code: image cutout cheating and image pasting cheating. Image cutout cheating refers to cutting out the original identification code from its position and covering it with a printed identification code or an identification code displayed on an electronic device such as a mobile phone or tablet. Image pasting cheating involves directly pasting a printed identification code onto the original identification code. To effectively identify these cheating behaviors, the target local classification model first relies on the target identification code model to accurately locate and identify the identification code in the image. This means that before processing with the target local classification model, a sub-image needs to be cropped from the image to be detected for more detailed analysis of the characteristics of the identification code. Furthermore, when processing with the target local classification model, the model further analyzes the details of the sub-image to analyze the image content at the edges of the identification code, such as the cleanliness of the edges, color consistency, and other possible aberrations, thereby identifying whether anomalies exist.

[0207] In this way, with the help of the target local classification model, abnormal situations such as the presence or absence of the identification code to be identified can be identified in a timely manner.

[0208] The following illustration, with reference to the accompanying diagram, demonstrates the anomaly identification process using the example of identifying unusual QR codes on product packaging in a retail scenario:

[0209] See Figure 11 This is a schematic diagram illustrating the identification process of an abnormal QR code in an embodiment of this application. The following is a description of the process in conjunction with the attached diagram. Figure 11 The following explains the anomaly recognition process involved in the processing equipment's differential processing of images containing a single QR code and multiple QR codes:

[0210] Step 1101: The processing device acquires the image to be detected.

[0211] Step 1102: The processing device determines the total number of QR codes to be recognized in the image to be detected based on the QR code positioning results output by the target QR code positioning model for the image to be detected.

[0212] Step 1103: The processing device determines whether the total number of QR codes to be recognized is more than one. If yes, proceed to step 1112; otherwise, proceed to step 1104.

[0213] Specifically, when the total number of QR codes to be recognized is no more than one, the processing device executes the processing steps 1104-1111; conversely, when the total number of QR codes to be recognized is more than one, the processing device executes the processing steps 1112-1120.

[0214] Step 1104: The processing device determines whether the total number of QR codes to be recognized is 1. If yes, proceed to step 1105; otherwise, proceed to step 1111.

[0215] Step 1105: The processing device uses a global target classification model to obtain anomaly classification results from the perspective of whether the QR code entity in the image to be detected is in a printed state.

[0216] Step 1106: Does the processing device determine whether the anomaly classification result indicates the existence of a QR code in a printed state? If yes, proceed to step 1111; otherwise, proceed to step 1107.

[0217] Step 1107: The processing device calculates the relative size of the QR code region to be recognized compared to the image to be detected based on the QR code positioning result.

[0218] Step 1108: The processing device determines whether the relative size result exceeds the set result threshold. If yes, proceed to step 1111; otherwise, proceed to step 1109.

[0219] Step 1109: The processing device crops out a sub-image containing the QR code to be recognized from the image to be detected.

[0220] Step 1110: The processing device uses a target local classification model to output anomaly recognition results based on sub-images.

[0221] Step 1111: The processing device ends the anomaly identification process and determines that there is an abnormal QR code in the image to be detected.

[0222] Step 1112: The processing device calculates the relative size of the QR code region to be recognized compared to the image to be detected based on the QR code positioning result.

[0223] Step 1113: The processing device determines whether the relative size result exceeds the set result threshold. If yes, proceed to step 1120; otherwise, proceed to step 1114.

[0224] Step 1114: The processing device uses a global target classification model to obtain anomaly classification results from the perspective of whether the QR code entity in the image to be detected is in a printed state.

[0225] Step 1115: The processing device determines whether the anomaly classification result indicates the existence of a QR code in a printing state. If yes, proceed to step 1120; otherwise, proceed to step 1116.

[0226] Step 1116: The processing device uses the target carrier localization model to output the carrier localization result based on the image to be detected.

[0227] Step 1117: The processing device determines whether the number of carriers in the image to be detected is the same as the number of QR codes to be recognized. If yes, proceed to step 1118; otherwise, proceed to step 1120.

[0228] Step 1118: The processing device crops out the corresponding sub-image for each QR code to be recognized in the image to be detected.

[0229] Step 1119: The processing device uses a target local classification model to output anomaly recognition results based on each sub-image.

[0230] Step 1120: The processing device determines that there is an abnormal QR code in the image to be detected.

[0231] See Figure 12 As shown, this is a schematic diagram of the identification process of another abnormal QR code in an embodiment of this application. The following is a detailed explanation in conjunction with the attached diagram. Figure 12 The following explanation addresses the anomaly recognition process when the processing device does not specifically distinguish between images containing a single QR code and images containing multiple QR codes:

[0232] Step 1201: The processing device acquires the image to be detected.

[0233] Step 1202: The processing device determines the total number of QR codes to be recognized in the image to be detected based on the QR code positioning results output by the target QR code positioning model for the image to be detected.

[0234] Step 1203: The processing device calculates the relative size of the QR code region to be recognized compared to the image to be detected based on the QR code positioning result.

[0235] Step 1204: The processing device determines whether the relative size result exceeds the set result threshold. If yes, proceed to step 1211; otherwise, proceed to step 1205.

[0236] Step 1205: The processing device uses a global target classification model to obtain anomaly classification results from the perspective of whether the QR code entity in the image to be detected is in a printed state.

[0237] Step 1206: The processing device determines whether the anomaly classification result indicates the existence of a QR code in a printing state. If yes, proceed to step 1211; otherwise, proceed to step 1207.

[0238] Step 1207: The processing device uses the target carrier localization model to output the carrier localization result based on the image to be detected.

[0239] Step 1208: The processing device determines whether the number of carriers in the image to be detected is the same as the number of QR codes to be recognized. If yes, proceed to step 1209; otherwise, proceed to step 1211.

[0240] Step 1209: The processing device crops out the corresponding sub-image for each QR code to be recognized in the image to be detected.

[0241] Step 1210: The processing device uses a target local classification model to output anomaly recognition results based on each sub-image.

[0242] Step 1211: The processing device determines that there is an abnormal QR code in the image to be detected.

[0243] It should be noted that in this embodiment, the processing device can utilize real-time data analysis and dynamic learning mechanisms to update the global and local classification models of the target in real time, in order to counter emerging identification code cheating methods. Overall, with the continuous advancement of identification code technology and the evolution of cheating methods, developing and implementing effective anti-cheating strategies is particularly important. By continuously optimizing these technical solutions, a safer and more reliable identification code usage environment can be provided for retail businesses, protecting the interests of merchants and consumers from harm.

[0244] In summary, this application proposes a systematic method for identifying fraudulent identification codes. This method enables the identification of abnormal identification codes in retail scenarios (such as QR code usage). By comprehensively utilizing advanced image analysis techniques and machine learning algorithms, it can effectively identify and distinguish normal identification codes from tampered or forged abnormal codes. This ensures high accuracy and recall in detecting fraudulent activities (such as image cutout, overlaying, and printing), thereby effectively preventing potential damage to retail operations caused by fraudulent behavior. Furthermore, the technical solution proposed in this application not only improves security but also optimizes user experience and operational efficiency by reducing false positives and false negatives. Therefore, the systematic abnormal identification code identification method proposed in this application is a highly valuable tool for retailers who need to ensure business security and data authenticity.

[0245] Furthermore, by utilizing the technical solution provided in this application, retailers can significantly improve the security and reliability of identification codes, effectively preventing and reducing economic losses and reputational risks caused by abnormal identification codes. In addition, it helps enhance the consumer experience, strengthens consumer trust in the retail platform, and thus promotes the healthy development of the retail business.

[0246] Furthermore, in a retail environment, each image containing the identification code to be identified can be analyzed and evaluated using the abnormal identification code recognition process proposed in this application. This process utilizes advanced image processing and machine learning technologies to conduct in-depth analysis of the input image from multiple anomaly evaluation perspectives, ultimately outputting the identification result for the abnormal identification code. This process not only significantly improves the efficiency of abnormal identification code recognition in retail scenarios but also enhances the accuracy of recognition. Through real-time processing and intelligent judgment, it can help retailers reduce the potential risks and economic losses caused by identification code fraud (i.e., abnormal identification codes), ensuring the security of business processes and the authenticity of data. In summary, this application effectively addresses the shortcomings of existing technologies in dealing with advanced fraud techniques and environmental changes by providing a smarter and more adaptive solution, offering stronger protection for the secure use of QR codes in retail businesses.

[0247] Based on the same inventive concept, see [reference] Figure 13 As shown, this is a schematic diagram of the logical structure of the anomaly identification code recognition device in this application embodiment. The anomaly identification code recognition device 1300 includes an acquisition unit 1301, an analysis unit 1302, and an execution unit 1303, wherein...

[0248] The obtaining unit 1301 is used to perform identification code recognition on the image to be detected and obtain the corresponding identification code positioning result;

[0249] The analysis unit 1302 is used to analyze the matching situation between at least one identification code and the printing status based on the global image features of the image to be detected when the identification code positioning result is determined to contain identification code positioning information of at least one identification code to be identified, thereby obtaining the abnormal classification result of the image to be detected.

[0250] Execution unit 1303 is configured to perform the following identification operations for each identification code location information when the anomaly classification result is determined to be that no anomaly exists:

[0251] Based on the location information of an identifier code, a sub-image containing the corresponding identifier code is cropped from the image to be identified, and the image content at the edge of the identifier code in the sub-image is analyzed to obtain the corresponding anomaly identification result.

[0252] Optionally, when the identification code location result indicates the existence of multiple identification codes to be identified, before performing the identification operation on the identification code location information corresponding to each identification code to be identified in the identification code location result, the analysis unit 1302 is further configured to:

[0253] Carrier identification is performed on the image to be detected to obtain the corresponding carrier localization result; the carrier localization result includes at least one carrier localization information; each carrier localization information is used to indicate and display the position of an identification code carrier associated with the identification code in the image to be detected;

[0254] Based on the carrier positioning results and the identification code positioning results, it is determined that each identification code carrier in the image to be detected displays an identification code to be identified.

[0255] Optionally, when it is determined, based on the carrier positioning result and the identification code positioning result, that each identification code carrier in the image to be detected displays an identification code to be identified, the analysis unit 1302 is used to:

[0256] Based on the carrier positioning results, determine the carrier region corresponding to each identification code carrier in the image to be detected, and based on the identification code positioning results, determine the identification code region corresponding to each identification code to be identified in the image to be detected.

[0257] Based on at least one carrier region and at least one identifier code region, the total number of identifier code carriers in the image to be detected is the same as the total number of identifier codes to be identified, and each carrier region contains one identifier code region.

[0258] Optionally, the analysis unit 1302 is also used for:

[0259] When the carrier positioning result and the identification code positioning result determine that there is an abnormal identification code carrier displaying multiple identification codes in the image to be detected, the abnormal identification code recognition process ends.

[0260] The image to be detected is determined to contain anomaly codes, and multiple unidentified codes displayed on the anomaly code carrier are marked as anomaly codes.

[0261] Optionally, the carrier localization result is obtained by the analysis unit 1302 using the trained target carrier localization model; the training process of the target carrier localization model is as follows:

[0262] Obtain each first training sample; wherein, each first training sample includes: a carrier sample image, and ground truth values ​​of the carrier position labeled for the carrier sample image;

[0263] Using each first training sample, the constructed initial carrier localization model is trained in multiple rounds of iteration to obtain the trained target carrier localization model. In one round of training, the model parameters are adjusted based on the content difference between the carrier location prediction value output by the initial carrier localization model based on the read carrier sample image and the corresponding ground truth value of the carrier location.

[0264] Optionally, after determining that the identification code location result contains identification code location information of at least one identification code to be identified, and before analyzing whether the at least one identification code to be identified is in a printed state based on the global image features of the image to be detected, and before obtaining the anomaly classification result of the image to be detected, the device further includes a determining unit, the determining unit 1304 being used for:

[0265] Based on the identification code positioning information of at least one identification code to be identified, the rectangular side information corresponding to at least one identification code to be identified is determined respectively;

[0266] For at least one identifier to be identified, perform the following operations respectively: based on the corresponding rectangle side information and the image size information of the image to be detected, determine the relative size result of the identifier to be identified compared with the image to be detected, and determine that the relative size result does not exceed the corresponding set result threshold.

[0267] Optionally, the determining unit 1304 is also used for:

[0268] When the relative size of a code to be identified relative to the image to be detected exceeds the corresponding set threshold, the identification process of the abnormal code ends.

[0269] The image to be detected is identified as having an abnormal identifier code, and a prompt message is displayed indicating that the image to be detected cannot be recognized.

[0270] Optionally, when cropping a sub-image containing the corresponding identification code from the image to be identified based on an identification code location information, the execution unit 1303 is used to:

[0271] Based on the location information of an identifier code, determine the region edge information of the corresponding identifier code to be identified;

[0272] In the image to be detected, the size of the region determined based on the region edge information is expanded according to a preset expansion ratio to determine the adjusted target region. The target region is then cropped out from the image to be detected to obtain the corresponding sub-image.

[0273] Optionally, when expanding the size of a region determined based on region edge information in the image to be detected according to a preset expansion ratio, the execution unit 1303 is used to:

[0274] In the image to be detected, the corresponding region size is determined based on the region edge information, and the region size is expanded according to a preset expansion ratio. While keeping the selected base positioning point position unchanged, the target region corresponding to the expanded target size is determined.

[0275] The basic positioning point can be any one of the following: the regional center point determined based on regional edge information, or the regional edge point determined based on regional edge information.

[0276] Optionally, the identifier code localization result is obtained by processing the target identifier code localization model using the trained target identifier code localization model in unit 1301; the training process of the target identifier code localization model is as follows:

[0277] Obtain each second training sample; wherein, each second training sample includes: an identifier sample image, and the ground truth value of the identifier position labeled for the identifier sample image;

[0278] Using each second training sample, the constructed initial identifier code localization model is trained in multiple rounds to obtain the trained target identifier code localization model. In one round of training, the model parameters are adjusted based on the content difference between the identifier code position prediction value output by the initial identifier code localization model based on the read identifier code sample image and the corresponding identifier code position ground truth value.

[0279] Optionally, the anomaly classification result is obtained by the analysis unit 1302 using a trained global target classification model; the training process of the global target classification model is as follows:

[0280] Obtain each third training sample; wherein, a third training sample includes: a sample image containing at least one sample identifier, and a classification ground truth value labeled according to whether the at least one sample identifier contains a sample identifier indicating a printing state.

[0281] Using each third training sample, the constructed initial global classification model is trained in multiple rounds of iteration to obtain the trained target global classification model. In each round of training, the model parameters are adjusted based on the classification prediction value output by the initial global classification model based on the read sample image and the corresponding true classification value.

[0282] Optionally, the anomaly detection result is obtained by the execution unit 1303 using a trained target local classification model; the training process of the target local classification model is as follows:

[0283] Obtain each fourth training sample; wherein, each fourth training sample includes: a sample sub-image containing a sample identifier code, and a classification ground truth value labeled for the sample sub-image; the sample identifier code in a sample sub-image is a single identifier code or a combination code obtained by superimposing multiple identifier codes;

[0284] Using each fourth training sample, the constructed initial local classification model is trained in multiple rounds of iteration to obtain the trained target local classification model. In each round of training, the model parameters are adjusted based on the classification difference between the classification prediction value output by the initial global classification model based on the read sample sub-image and the corresponding classification ground truth value.

[0285] Optionally, after obtaining the corresponding identification code location result, the execution unit 1303 is further used to:

[0286] When the identification code location result does not contain the identification code location information of the identification code to be identified, the identification process of the abnormal identification code is directly terminated, the image to be detected is determined to contain an abnormal identification code, and a prompt message indicating that no identification code to be identified was found is displayed.

[0287] For ease of description, the above sections are divided into modules (or units) according to their functions and described separately. Of course, in implementing this application, the functions of each module (or unit) can be implemented in one or more software or hardware components.

[0288] Having introduced the method and apparatus for identifying anomaly identifiers according to exemplary embodiments of this application, we will now introduce an electronic device according to another exemplary embodiment of this application.

[0289] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0290] Based on the same inventive concept as the above-described method embodiments, this application also provides an electronic device. See reference... Figure 14 As shown, it is a schematic diagram of the hardware composition structure of an electronic device applying an embodiment of this application. In one embodiment, the electronic device may be... Figure 1 The server 120 is shown. In this embodiment, the structure of the electronic device can be as follows: Figure 14 As shown, it includes a memory 1401, a communication module 1403, and one or more processors 1402.

[0291] The memory 1401 is used to store computer programs executed by the processor 1402. The memory 1401 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and programs required to run instant messaging functions, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0292] Memory 1401 may be volatile memory, such as random-access memory (RAM); memory 1401 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 1401 may be any other medium capable of carrying or storing a desired computer program having the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1401 may be a combination of the above-described memories.

[0293] Processor 1402 may include one or more central processing units (CPUs) or digital processing units, etc. Processor 1402 is used to implement methods related to the identification of the aforementioned exception identifier code when calling computer programs stored in memory 1401.

[0294] The communication module 1403 is used to communicate with the processing device and the server.

[0295] This application embodiment does not limit the specific connection medium between the memory 1401, communication module 1403, and processor 1402. This application embodiment... Figure 14 The memory 1401 and the processor 1402 are connected via a bus 1404, and the bus 1404 is in Figure 14 The diagram uses thick lines to describe the connections between other components; these are for illustrative purposes only and should not be considered limiting. The 1404 bus can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 14 It is described using only a thick line, but does not indicate that there is only one bus or one type of bus.

[0296] The memory 1401 stores a computer storage medium containing computer-executable instructions for implementing the method related to the identification of the above-described anomaly identifier code in this embodiment. The processor 1402 is used to execute the method related to the identification of the above-described anomaly identifier code.

[0297] In another embodiment, the electronic device may also be other electronic devices, see [reference]. Figure 15 As shown, it is a schematic diagram of the hardware composition structure of another electronic device applying the embodiments of this application. The electronic device may specifically be... Figure 1 The processing device 110 is shown. In this embodiment, the electronic device can be structured as follows: Figure 15As shown, it includes components such as: communication component 1510, memory 1520, display unit 1530, camera 1540, sensor 1550, audio circuit 1560, Bluetooth module 1570, processor 1580, etc.

[0298] The communication component 1510 is used to communicate with the server. In some embodiments, it may include a Circuit-Based Wireless Fidelity (WiFi) module, which is a short-range wireless transmission technology. Electronic devices can use the WiFi module to help users send and receive information.

[0299] The memory 1520 can be used to store software programs and data. The processor 1580 executes various functions of the processing device 110 and data processing by running the software programs or data stored in the memory 1520. In this application, the memory 1520 can store the operating system and various application programs, and can also store computer programs that identify the anomaly identifier codes in the embodiments of this application.

[0300] The display unit 1530 can also be used to display information input by the user or information provided to the user, as well as a graphical user interface (GUI) of various menus of the processing device 110. Specifically, the display unit 1530 may include a display screen 1532 disposed on the front of the processing device 110. The display unit 1530 can be used to display pages, etc.

[0301] The display unit 1530 can also be used to receive input digital or character information and generate signal inputs related to user settings and function control of the processing device 110. Specifically, the display unit 1530 may include a touch screen 1531 disposed on the front of the processing device 110, which can collect touch operations of the user on or near it.

[0302] The touchscreen 1531 can be placed over the display screen 1532, or the touchscreen 1531 and the display screen 1532 can be integrated to realize the input and output functions of the processing device 110. After integration, it can be referred to as a touch display screen. In this application, the display unit 1530 can display the application program and the corresponding operation steps.

[0303] Camera 1540 can be used to capture still images, which users can then post comments on via an application. An object is projected onto a photosensitive element through a lens, generating an optical image. This photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to processor 1580 for conversion into a digital image signal.

[0304] The processing device may also include at least one sensor 1550, such as an accelerometer 1551, a distance sensor 1552, a fingerprint sensor 1553, and a temperature sensor 1554. The processing device may also be equipped with other sensors such as a gyroscope, barometer, hygrometer, thermometer, infrared sensor, light sensor, and motion sensor.

[0305] Audio circuitry 1560, speaker 1561, and microphone 1562 provide an audio interface between the user and processing device 110. Audio circuitry 1560 converts received audio data into electrical signals and transmits them to speaker 1561, where speaker 1561 converts them into sound signals for output. Conversely, microphone 1562 converts collected sound signals into electrical signals, which are then received by audio circuitry 1560, converted back into audio data, and output to communication component 1510 for transmission to, for example, another processing device 110, or to memory 1520 for further processing.

[0306] The Bluetooth module 1570 is used to exchange information with other Bluetooth devices that have Bluetooth modules via the Bluetooth protocol.

[0307] The processor 1580 is the control center of the processing device, connecting various parts of the terminal via various interfaces and lines. It executes software programs stored in the memory 1520 and calls data stored in the memory 1520 to perform various functions of the processing device and process data. In some embodiments, the processor 1580 may include at least one processing unit; the processor 1580 may also integrate an application processor and a baseband processor. In this application, the processor 1580 can run an operating system, applications, user interface display and touch response, and execute the abnormal identification code identification method of the embodiments of this application, such as... Figure 2 As shown. Additionally, the processor 1580 is coupled to the display unit 1530.

[0308] In some possible implementations, various aspects of the anomaly identification code recognition method provided in this application can also be implemented in the form of a program product, which includes a computer program. When the program product is run on an electronic device, the computer program causes the electronic device to perform the steps in the anomaly identification code recognition method according to the various exemplary embodiments of this application described above. For example, the electronic device can perform actions such as... Figure 2 The steps are shown in the figure.

[0309] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0310] The program product of the embodiments of this application may employ a portable compact disc read-only memory (CD-ROM) and include a computer program, and may run on an electronic device. However, the program product of this application is not limited thereto. In this document, the readable storage medium may be any tangible medium that contains or stores a program that may be used by or in conjunction with a command execution system, apparatus, or device.

[0311] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a readable computer program. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with a command execution system, apparatus, or device.

[0312] Computer programs contained on readable media can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0313] Computer programs for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The computer program can execute entirely on the user's electronic device, partially on the user's electronic device, as a standalone software package, partially on the user's electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In cases involving remote electronic devices, the remote electronic device can be connected to the user's electronic device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external electronic device (e.g., via the Internet using an Internet service provider).

[0314] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0315] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0316] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing a computer-usable computer program.

[0317] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0318] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0319] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for identifying anomaly identifier codes, characterized in that, include: The image to be detected is identified by its identification code to obtain the corresponding identification code location result; When it is determined that the identification code location result contains identification code location information of at least one identification code to be identified, the matching situation of the at least one identification code to be identified and the printing status is analyzed based on the global image features of the image to be detected, and the anomaly classification result of the image to be detected is obtained. When the anomaly classification result is determined to be that no anomaly exists, the following identification operations are performed for each of the identifier codes to be identified: Based on an identifier code location information, a sub-image containing the corresponding identifier code is cropped from the image to be identified, and the image content at the edge of the identifier code in the sub-image is analyzed to obtain the corresponding anomaly identification result.

2. The method as described in claim 1, characterized in that, When the identification code location result contains identification code location information for multiple identification codes to be identified, before performing the identification operation on the identification code location information corresponding to each identification code to be identified in the identification code location result, the method further includes: Carrier identification is performed on the image to be detected to obtain the corresponding carrier positioning result; the carrier positioning result includes at least one carrier positioning information; each carrier positioning information is used to indicate and display the position of an identification code carrier associated with the identification code in the image to be detected; Based on the carrier positioning result and the identification code positioning result, it is determined that each identification code carrier in the image to be detected displays an identification code to be identified.

3. The method as described in claim 2, characterized in that, The step of determining that each identification code carrier in the image to be detected displays an identification code to be identified based on the carrier positioning result and the identification code positioning result includes: Based on the carrier positioning results, determine the carrier region corresponding to each identification code carrier in the image to be detected, and based on the identification code positioning results, determine the identification code region corresponding to each identification code to be identified in the image to be detected. Based on at least one carrier region and at least one identifier code region, the total number of identifier code carriers in the image to be detected is determined to be the same as the total number of identifier codes to be identified, and each carrier region contains one identifier code region.

4. The method as described in claim 2, characterized in that, The method further includes: When it is determined, based on the carrier positioning result and the identification code positioning result, that there is an abnormal identification code carrier displaying multiple identification codes in the image to be detected, the identification process of the abnormal identification code ends. The image to be detected is determined to have an abnormal identification code, and the plurality of identification codes to be identified displayed on the abnormal identification code carrier are marked as abnormal identification codes.

5. The method as described in claim 2, characterized in that, The carrier localization result is obtained by processing the target carrier localization model after training; the training process of the target carrier localization model is as follows: Obtain each first training sample; wherein, each first training sample includes: a carrier sample image, and a ground truth value of the carrier position labeled for the carrier sample image; Using the first training samples, the constructed initial carrier localization model is trained in multiple rounds of iterations to obtain the trained target carrier localization model. In one round of training, the model parameters are adjusted based on the content difference between the carrier position prediction value output by the initial carrier localization model based on the read carrier sample image and the corresponding carrier position ground truth value.

6. The method as described in claim 1, characterized in that, After determining that the identification code location result contains identification code location information of at least one identification code to be identified, and before analyzing whether the at least one identification code to be identified is in a printed state based on the global image features of the image to be detected, and before obtaining the anomaly classification result of the image to be detected, the method further includes: Based on the identification code positioning information of the at least one identification code to be identified, the rectangular side information corresponding to the at least one identification code to be identified is determined respectively; For the at least one identifier to be identified, the following operations are performed respectively: based on the corresponding rectangle side information and the image size information of the image to be detected, a relative size result of an identifier to be identified compared with the image to be detected is determined, and the relative size result is determined not to exceed the corresponding set result threshold.

7. The method as described in claim 6, characterized in that, The method further includes: When the relative size of a code to be identified relative to the image to be detected exceeds the corresponding set threshold, the identification process of the abnormal code ends. The image to be detected is identified as having an abnormal identifier code, and a prompt message indicating that the image to be detected cannot be recognized is displayed.

8. The method according to any one of claims 1-7, characterized in that, The step of cropping a sub-image containing the corresponding identification code from the image to be identified based on an identification code location information includes: Based on the location information of an identifier code, determine the region edge information of the corresponding identifier code to be identified; In the image to be detected, the size of the region determined based on the region edge information is expanded according to a preset expansion ratio to determine the adjusted target region. The target region is then cropped from the image to be detected to obtain the corresponding sub-image.

9. The method as described in claim 8, characterized in that, The step of expanding the size of the region determined based on the region edge information in the image to be detected according to a preset expansion ratio includes: In the image to be detected, the corresponding region size is determined based on the region edge information, and the region size is expanded according to a preset expansion ratio. While keeping the selected base positioning point position unchanged, the target region corresponding to the expanded target size is determined. The basic positioning point can be any one of the following: the regional center point determined based on the regional edge information, and the regional edge point determined based on the regional edge information.

10. The method according to any one of claims 1-7, characterized in that, The identifier code localization result is obtained by processing the trained target identifier code localization model; the training process of the target identifier code localization model is as follows: Obtain each second training sample; wherein, each second training sample includes: an identifier sample image, and the identifier position ground value annotated for the identifier sample image; Using the second training samples, the constructed initial identifier code localization model is trained in multiple rounds to obtain the trained target identifier code localization model. In one round of training, the model parameters are adjusted based on the content difference between the identifier code position prediction value output by the initial identifier code localization model based on the read identifier code sample image and the corresponding identifier code position ground truth value.

11. The method according to any one of claims 1-7, characterized in that, The anomaly classification result is obtained by processing a trained global target classification model; the training process of the global target classification model is as follows: Obtain each third training sample; wherein, a third training sample includes: a sample image containing at least one sample identifier code, and a classification ground truth value labeled according to whether the at least one sample identifier code contains a sample identifier code of a printing state. Using the aforementioned third training samples, the constructed initial global classification model is trained through multiple rounds of iteration to obtain the trained target global classification model. In each round of training, the model parameters are adjusted based on the classification difference between the classification prediction value output by the initial global classification model based on the read sample image and the corresponding true classification value.

12. The method according to any one of claims 1-7, characterized in that, The anomaly identification result is obtained by processing a trained target local classification model; the training process of the target local classification model is as follows: Obtain each fourth training sample; wherein, each fourth training sample includes: a sample sub-image containing a sample identifier code, and a classification ground truth value labeled for the sample sub-image; the sample identifier code in a sample sub-image is a single identifier code or a combination code obtained by superimposing multiple identifier codes; Using the aforementioned fourth training samples, the constructed initial local classification model is trained through multiple rounds of iteration to obtain the trained target local classification model. In each round of training, the model parameters are adjusted based on the classification difference between the classification prediction value output by the initial global classification model based on the read sample sub-image and the corresponding true classification value.

13. The method according to any one of claims 1-7, characterized in that, After obtaining the corresponding identification code location result, the method further includes: When the identification code location result does not contain the identification code location information of the identification code to be identified, the identification process of the abnormal identification code is directly terminated, the image to be detected is determined to have an abnormal identification code, and a prompt message indicating that the identification code to be identified was not found is displayed.

14. A device for identifying anomaly codes, characterized in that, include: The acquisition unit is used to perform identification code recognition on the image to be detected and obtain the corresponding identification code positioning result; The analysis unit is used to analyze the matching situation between the at least one identifier to be identified and the printing status based on the global image features of the image to be detected when it is determined that the identifier positioning result contains identifier positioning information of at least one identifier to be identified, thereby obtaining the abnormal classification result of the image to be detected. The execution unit is configured to perform the following identification operations for each identification code location information when the anomaly classification result is determined to be that no anomaly exists: Based on an identifier code location information, a sub-image containing the corresponding identifier code is cropped from the image to be identified, and the image content at the edge of the identifier code in the sub-image is analyzed to obtain the corresponding anomaly identification result.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-13.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the method as described in any one of claims 1-13.

17. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-13.

Images