Character verification method based on image recognition and related device

Abstract

The invention provides a character verification method based on image recognition and a related device, and relates to the technical field of computers. The method comprises the following steps: acquiring a target image, and inputting the target image into a pre-trained first character extraction model to obtain a first character extraction result; performing character verification based on the first character extraction result and a preset character corresponding to a verification task matched with the target image; under the condition that verification based on the first character extraction result fails, inputting the target image into a pre-trained second character extraction model to obtain a second character extraction result; wherein compared with the first character extraction model, the second character extraction model is high in accuracy and long in time consumption; performing character verification based on the second character extraction result and the preset character; and outputting a verification result. According to the technical scheme provided by the invention, the problem of how to improve the character verification efficiency and verification reliability in the prior art can be solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, in particular to a character verification method based on image recognition and related devices. BACKGROUND

[0002] In the scene of national subsidy declaration audit, etc., it is necessary to take pictures of commodities and commodity packaging for audit, and one of the focuses of the audit is to audit and verify the serial number (SN) of the product, the international mobile equipment identity (IMEI) and other number information of the commodity, which requires extracting character information such as commodity number from the photographed image, and then performing audit and verification. How to improve the efficiency and reliability of character verification has become a problem to be solved. SUMMARY

[0003] Based on the defects and deficiencies of the prior art, the present application provides a character verification method based on image recognition and related devices, which can solve the problem of how to improve the efficiency and reliability of character verification in the prior art.

[0004] According to a first aspect of an embodiment of the present application, a character verification method based on image recognition is provided, the method comprising: obtaining a target image and inputting the target image into a pre-trained first character extraction model to obtain a first character extraction result; performing character verification based on the first character extraction result and a preset character corresponding to a verification task matched with the target image; in the case of failure of verification based on the first character extraction result, inputting the target image into a pre-trained second character extraction model to obtain a second character extraction result; wherein the second character extraction model has higher accuracy but longer time consumption than the first character extraction model; performing character verification based on the second character extraction result and the preset character; outputting a verification result.

[0005] According to a second aspect of an embodiment of the present application, a character verification device based on image recognition is provided, the device comprising: a first obtaining module configured to obtain a target image and input the target image into a pre-trained first character extraction model to obtain a first character extraction result; a first verification module configured to perform character verification based on the first character extraction result and a preset character corresponding to a verification task matched with the target image; The second obtaining module is configured to input the target image into a second character extraction model that is pre-trained, and obtain a second character extraction result, in a case where the verification based on the first character extraction result fails; the second character extraction model has higher accuracy but longer time consumption than the first character extraction model; The second verification module is configured to perform character verification based on the second character extraction result and the preset character. The output module is configured to output a verification result.

[0006] According to a third aspect of the embodiments of the present application, an electronic device is provided, which includes a memory and a processor. The memory is connected to the processor, and is configured to store a program. The processor is configured to realize the character verification method based on image recognition according to the first aspect by running the program in the memory.

[0007] According to a fourth aspect of the embodiments of the present application, a storage medium is provided, which stores a computer program. When the computer program is run by a processor, the character verification method based on image recognition according to the first aspect is realized.

[0008] According to a fifth aspect of the embodiments of the present application, a computer program product or a computer program is provided. The computer program product includes the computer program. When the processor executes the computer program, the steps in the character verification method based on image recognition according to the first aspect are realized.

[0009] In the technical solutions provided in the present application, a first character extraction model that is pre-trained can be used to extract characters from a target image. The character extraction model is an artificial intelligence model, which has strong global context modeling capability. It can not only look at characters themselves, but also make comprehensive reasoning by combining character context information, so it can realize more accurate character recognition and extraction. In addition, the model has high computing power, and can quickly output character extraction results, so it is beneficial to improve the efficiency and reliability of character verification. In addition, the second character extraction model can be used to extract characters from the target image in a case where the verification based on the first character extraction result fails, and character verification can be performed based on the new character extraction result. Although the second character extraction model has relatively long time consumption compared with the first character extraction model, it has higher character extraction accuracy. Using the second character extraction model as an alternative solution for character extraction can further improve the reliability of character extraction, and thus improve the reliability of character verification. BRIEF DESCRIPTION OF DRAWINGS

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0011] FIG. 1 This is a flowchart illustrating a character verification method based on image recognition, provided as an embodiment of this application.

[0012] FIG. 2 This is a schematic diagram of an example flow of a character verification method based on image recognition, provided for an embodiment of this application.

[0013] FIG. 3 This is a block diagram of a character verification device based on image recognition, provided in an embodiment of this application.

[0014] FIG. 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] Exemplary method This application also provides a character verification method based on image recognition, which is applied to electronic devices, such as terminal devices, mobile phones, computers, etc., or servers.

[0017] The method is described in detail below through some embodiments. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0018] like FIG. 1 As shown, the character verification method based on image recognition may include steps 101 to 103, as described below.

[0019] Step 101: Obtain the target image.

[0020] The target image mentioned here can be an image acquired by an image acquisition device, such as an image captured by a camera or an image obtained by scanning a barcode scanner; or it can be a rendered image generated by a computer device, such as a two-dimensional rendered image, a three-dimensional rendered image, or an AI image.

[0021] In this embodiment of the application, the target image can be obtained from the target image database.

[0022] In the scenario of national subsidy application review, the target image can be an order review image uploaded to the order review system. This order review image includes the device (such as a mobile phone, platform computer, etc.) corresponding to the order to be reviewed, as well as the device's corresponding serial number information (such as SN code, IMEI code, etc.). For example, if a user purchases a laptop eligible for national subsidies on an online shopping platform, and applying for the subsidy requires verification, the courier can take photos of the SN code on the laptop and the SN code on the laptop packaging when picking up the goods from the merchant, and upload the photos to the order review system. These photos constitute the verified order review image. In this embodiment, the target image can be obtained from the image database of the order review system based on order information (such as the order number).

[0023] Step 102: Input the target image into the pre-trained first character extraction model to obtain the first character extraction result.

[0024] In this embodiment of the application, the desired target character can be extracted from the target image using a pre-trained first character extraction model.

[0025] The target character can be a string containing multiple characters. This string can include at least one of the following: Chinese characters, numbers, and letters. For example, the string could be a license plate number composed of Chinese characters, numbers, and letters; a tracking number composed of letters and numbers; a serial number (SN) composed of letters and / or numbers; or an IMEI code composed of numbers. It is understood that the target character is not limited to the above and can be set according to actual needs.

[0026] The target image may have problems such as blurriness, tilt, uneven lighting, low resolution, or background interference, leading to errors such as missed character detection and misidentification during character recognition. For example, when the image has noise interference or the character edges are not clear enough, some correct characters may be misidentified as other similar characters, or some character content may not be accurately identified at all. When the character recognition result is inaccurate, it will affect the accuracy of the subsequent character extraction results.

[0027] The first character extraction model is an artificial intelligence model with powerful global context modeling capabilities. It can not only learn the features of the target character, but also read and understand the entire text, thereby performing comprehensive reasoning. Therefore, it can achieve more accurate character recognition and extraction.

[0028] Optionally, the first character extraction model can be a Visual Language Model (VLM), a type of multimodal generative artificial intelligence system that combines a large language model (LLM) with a visual encoder to achieve integrated image and text processing capabilities.

[0029] If the first character extraction result indicates that the target character has been extracted, step 103 can be executed; if the first character extraction result indicates that the target character has not been extracted, step 104 can be executed.

[0030] Step 103: Perform character verification based on the first character extraction result and the preset character corresponding to the verification task that matches the target image.

[0031] The preset characters mentioned here are the correct characters that have been set in advance.

[0032] Verification tasks can include checking whether the extracted first character matches a preset character. For example, in a national subsidy application review scenario, the extracted first character is the serial number (SN) of a laptop extracted from the order review image, and the preset character is the device identification number pre-stored in the order review system corresponding to the order to be reviewed. Specifically, this could be the correct SN uploaded by the merchant to the order review system. The verification task could be to check whether the SN extracted from the order review image is the same as the correct SN uploaded by the merchant to the order review system, to prevent subsidy fraud. As another example, if the target image is a photograph of a vehicle license plate, and the preset character is the license plate number of a known vehicle, the verification task could be to check whether the license plate number in the photograph matches the license plate number of a known vehicle, to determine whether the vehicle in the image is a known vehicle.

[0033] If the extracted first character matches the preset character, the character verification is successful; if the extracted first character does not match the preset character, the character verification fails.

[0034] The matching mentioned here can refer to the character extraction result being the same as a preset character, or the similarity between the character extraction result and the preset character being greater than a similarity threshold. This similarity threshold can be set according to actual needs, for example, it can be set to 98%. Different matching strategies can be adopted for different application scenarios.

[0035] If the character verification is successful, proceed to step 106; the verification result will then be considered successful. If the character verification fails, proceed to step 104.

[0036] Step 104: If the verification based on the first character extraction result fails, input the target image into the pre-trained second character extraction model to obtain the second character extraction result.

[0037] The second character extraction model is more accurate but takes longer than the first character extraction model.

[0038] In this embodiment of the application, if the first character extraction result indicates extraction failure, that is, if the target character is not extracted, or if the first character extraction result is different from the preset character, the target image can be input into the pre-trained second character extraction model to obtain the second character extraction result.

[0039] The second character extraction model described here is more complex and has higher accuracy than the first model, but it is also more time-consuming. Using the second model as an alternative can further improve the reliability of character extraction.

[0040] Optionally, the second character extraction model can be a commercial large model. Such large models are designed and optimized for enterprise-level applications and can solve specific enterprise problems. Compared with ordinary visual language large models, they have better data processing performance. However, due to the complexity of the model, the data processing time will be slightly longer. Therefore, considering the efficiency of character extraction, the first character extraction model can be used first, followed by the second character extraction model.

[0041] If the second character extraction result indicates that the target character has been extracted, step 105 can be executed; if the second character extraction result indicates that the target character has not been extracted, step 106 can be executed, and the corresponding verification result is verification failure.

[0042] Step 105: Perform character verification based on the second character extraction result and the preset characters.

[0043] In other words, it determines whether the extracted second character matches the preset character. If the extracted second character matches the preset character, the character verification is successful, and step 106 is executed. The verification result at this time is a successful verification. If the extracted second character does not match the preset character, the character verification fails, and step 106 is executed. The verification result at this time is a failed verification.

[0044] Step 106: Output the verification result.

[0045] The verification result is used to indicate whether the verification was successful or failed.

[0046] Optionally, in the national subsidy application review scenario, if the verification result indicates verification failure, it means the review has not passed, and the courier can be reminded to take photos again. If the number of failed reviews reaches a preset number (e.g., 3 times), a verification failure message can be output, and the courier will no longer be prompted to take photos again. If the verification result indicates verification success, it means the review has passed.

[0047] In some alternative embodiments, after "acquiring the target image" in step 101, the method may further include steps A1 to A3, as described below: Step A1: Perform character recognition on the target image using a character recognition algorithm, and then perform target character matching on the character recognition results to obtain the character matching results.

[0048] In this embodiment of the application, during the process of extracting target characters from the target image, the target image can first be recognized by a character recognition algorithm (such as an optical character recognition (OCR) algorithm), and the target character can be matched in the character recognition result based on a preset matching algorithm to obtain the character matching result.

[0049] The character matching result can indicate whether the target character was matched (i.e., the match was successful) or not matched (i.e., the match failed). If the character matching result indicates a successful match, step A2 can be executed; if the character matching result indicates a failed match, step 101 can be executed by "inputting the target image into the pre-trained first character extraction model to obtain the first character extraction result".

[0050] When performing target character matching, the preset matching algorithm used can be a full match algorithm or a regular expression matching algorithm, etc. A full match algorithm requires that the character recognition result contains exactly the target character to be considered a successful match; otherwise, the match fails. A regular expression matching algorithm, on the other hand, performs matching based on a preset regular expression. If the character recognition result contains character content that matches the regular expression, the match is considered successful; otherwise, the match fails.

[0051] Step A2: If the character matching result indicates that the target character has been matched, extract the target character according to the character matching result to obtain the third character extraction result, and perform character verification based on the third character extraction result and the preset character.

[0052] If the character matching result obtained in step A1 indicates that the target character has been matched, the matched target character can be extracted to obtain the third character extraction result. Then, it is determined whether the third character extraction result matches the preset character. If they match, step 106 can be executed, and the corresponding verification result is successful; if they do not match, as described in step A3, step 101 can be executed to "input the target image into the pre-trained first character extraction model to obtain the first character extraction result".

[0053] Step A3: If the character matching result indicates that the target character is not matched, perform the step of inputting the target image into the pre-trained first character extraction model to obtain the first character extraction result.

[0054] In this embodiment, considering the potential for illusion problems in character recognition by the first character extraction model (such as inferring incorrect character content based on prior information), while the character recognition algorithm performs actual character recognition without illusion problems, a method combining character recognition and character matching algorithms can be prioritized for character extraction to ensure accuracy. If the target character cannot be matched during character matching, the model extraction method is used as an alternative to improve reliability.

[0055] In some optional embodiments, after obtaining the target character extraction result and before performing character verification based on the target character extraction result, the method may further include steps B1 to B3, as described below: Step B1: Determine the target region of the character in the target image from the extracted character results.

[0056] The target character extraction result can be the first character extraction result, the second character extraction result, or the third character extraction result.

[0057] Step B2: Perform a sharpness test on the target area and obtain the sharpness test results.

[0058] Step B3: If the sharpness detection result indicates that the sharpness is greater than the sharpness threshold, the target character extraction result is determined to be a valid character extraction result.

[0059] In this embodiment, if the character extraction result obtained by any method indicates that the target character has been successfully extracted, the target region of the target character in the target image can be determined. Then, the clarity of the target region is detected. If the clarity is greater than the clarity threshold, it means that the image quality meets the requirements. The extracted character extraction result is then used as the final character extraction result, i.e., the valid character extraction result. Especially when the target character is information pending review in a national subsidy application review scenario, image clarity is essential to ensure that the uploaded review image is reliable and usable. This avoids the problem of blurry images affecting the character extraction result, and thus affecting the review result.

[0060] In some alternative embodiments, when the first character extraction model is a large visual language model, it can be optimized and trained to improve its output performance. For example, it can be trained using supervised fine-tuning (SFT), a technique that adjusts model parameters using a small amount of labeled data.

[0061] In supervised fine-tuning training, the visual encoder part (such as VIT) in the first character extraction model can be frozen, the large language model part (i.e., LLM) can be fine-tuned to obtain an optimized model, and the optimized model can be evaluated to obtain a better first character extraction model, which is then deployed.

[0062] In some embodiments, the sample images used for training the first character extraction model can be obtained through steps C1 to C5, as described below: Step C1: Obtain the initial sample image.

[0063] The initial sample image is a sample image without ground truth annotation. The ground truth here refers to the actual target character corresponding to the initial sample image.

[0064] Step C2: Input the initial sample image into the preset model, and use the preset model to perform ground truth annotation on the initial sample image to obtain the intermediate sample image.

[0065] The intermediate sample image is a sample image with truth value annotations.

[0066] The preset model mentioned here is a pre-trained ground truth labeling model. This preset model can be a first character extraction model that has not been trained under supervision, or it can be other models that can perform ground truth labeling, such as the open-source InternVL model.

[0067] The preset model can extract target characters from the initial sample image and use the extracted target characters as ground truth to annotate the initial sample image, thereby obtaining the intermediate sample image.

[0068] Step C3: Compare the ground truth annotation results of the intermediate sample images with the corresponding real target characters to see if they are the same.

[0069] The actual target characters mentioned here are pre-set.

[0070] In this embodiment, a correspondence between each initial sample image and its corresponding real target character can be established in advance. When an intermediate sample image is obtained based on the initial sample image, a correspondence between the initial sample image and the intermediate sample image is also established. Therefore, based on the correspondence between the intermediate sample image and the initial sample image, as well as the correspondence between the initial sample image and its corresponding real target character, the real target character corresponding to the intermediate sample image can be determined.

[0071] After obtaining intermediate sample images by ground truth annotation of the initial sample images using a preset model, it can be determined whether the ground truth annotation results of the intermediate sample images are the same as the corresponding real target characters. If they are the same, proceed to step C4; if they are different, proceed to step C5.

[0072] Step C4: Use the intermediate sample images as sample images for training the first character extraction model.

[0073] If the ground truth labeling result of the intermediate sample image is the same as the corresponding real target character, it indicates that the model labeling result is accurate, and the intermediate sample image can be used as a sample image for training the first character extraction model.

[0074] Step C5: Replace the ground truth labeling results with the corresponding real target characters to obtain sample images used to train the first character extraction model.

[0075] If the ground truth labeling result of the intermediate sample image is different from the corresponding real target character, it indicates that the model labeling result is incorrect. In this case, the ground truth labeling result of the intermediate sample image can be replaced with the corresponding real target character to obtain the final sample image.

[0076] In this embodiment, ground truth annotation of sample images is performed using a model, which, compared to manual annotation, enables rapid annotation of ground truth and improves the efficiency of ground truth annotation. Furthermore, this embodiment provides a verification step to check the accuracy of the ground truth annotation results and automatically corrects any errors. Therefore, the above ground truth annotation method improves both efficiency and accuracy.

[0077] As an alternative to the above-mentioned truth labeling method, the sample images used for training the first character extraction model can also be obtained through steps D1 to D5, as described below: Step D1: Obtain the initial sample image.

[0078] The initial sample images are sample images that have not been labeled with ground truth values.

[0079] Step D2: Input the initial sample image into the preset model, and use the preset model to perform ground truth annotation on the initial sample image to obtain the intermediate sample image.

[0080] Step C3: Compare the ground truth annotation results of the intermediate sample images with the corresponding real target characters to see if they are the same.

[0081] After obtaining intermediate sample images by ground truth annotation of the initial sample images using a preset model, it can be determined whether the ground truth annotation results of the intermediate sample images are the same as the corresponding real target characters. If they are the same, proceed to step D4; if they are different, proceed to step D5.

[0082] Step D4: Identify the intermediate sample image with the first marker.

[0083] The first marker is used to indicate that the ground truth annotation result of the intermediate sample image is the same as the corresponding real target character. In other words, it is used to indicate that the annotation result of the preset model is accurate. Then, the annotator can know based on the first marker that there is no need for further processing of the intermediate sample image by human intervention.

[0084] This intermediate sample image is the final sample image used to train the first character extraction model.

[0085] Step D5: Identify the intermediate sample image with the second marker.

[0086] The second marker indicates that the ground truth annotation of the intermediate sample image is different from the corresponding real target character, that is, it indicates that the annotation result of the preset model is incorrect. The annotator can then further process the intermediate sample image based on the second marker. For example, the annotator can replace the ground truth annotation of the intermediate sample image with the corresponding real target character to obtain the final sample image used to train the first character extraction model.

[0087] Optionally, if the actual target character corresponding to the intermediate sample image cannot be obtained, the method may further include: Step D6: Identify the intermediate sample image with a third label.

[0088] The third marker is used to indicate whether the ground truth labeling results of the intermediate sample images are correct after manual verification.

[0089] If the initial sample image does not have the corresponding real target character set or saved, or if the acquisition of the real target character corresponding to the intermediate sample image obtained based on the initial sample image fails due to other reasons, the intermediate sample image can be marked with a third label so that subsequent annotators can manually check whether the true value annotation result is accurate based on the third label, and modify the true value annotation result if the true value annotation result is incorrect.

[0090] The truth labeling method provided in this embodiment adds a manual verification process, which can further improve the accuracy of the truth labeling results. At the same time, since it is not necessary for all truth labeling to be done manually, it can also achieve quick truth labeling and improve the efficiency of truth labeling.

[0091] Optionally, the initial sample image in the two aforementioned truth-marking embodiments can be a historical image used to extract the target character. This historical image is stored in the target system, which also stores the actual target character corresponding to the historical image. The target system can be an order verification system.

[0092] Since the true target characters have already been obtained in the historical character extraction process of such historical images, the sample images used to train the model based on such images can more easily verify the accuracy of the ground truth labeling results.

[0093] In the scenario of national subsidy application review, the target system in the aforementioned two truth value annotation embodiments can be an order review system. This order review system can record information such as the order review image, the category of the equipment in the image, the real target character corresponding to the equipment, and the review result.

[0094] Because the required evidence for verification may differ depending on the device category, for example, for laptops, the required evidence includes the serial number (SN) on the back of the computer; while for mobile phones, the required evidence includes the SN and IMEI codes displayed on the phone. Therefore, the initial sample images obtained from the order verification system can be categorized by product type, and different prompts can be designed for different product categories to improve the accuracy of truth labeling. Similarly, for the first character extraction model and the second character extraction model, different prompts can also be designed according to the different product categories.

[0095] In addition, the initial sample images obtained from the order review system can also include initial sample images that have passed the review and initial sample images that have failed the review, so as to achieve diversified training of the model.

[0096] See last. FIG. 2 The following example illustrates the character verification method based on image recognition provided in this application: The target image is subjected to OCR recognition to obtain character recognition results. Regular expression matching is then applied to these results to extract the target character. The character extraction result indicates whether character extraction was successful or failed. If character extraction is successful, the extracted character is validated to obtain a validation result. If the validation result indicates success, it is output; if it indicates failure, the target image is input into the first character extraction model; if character extraction fails, the target image is input into the first character extraction model again, and a new character extraction result is obtained through the first character extraction model.

[0097] If the new character extraction result obtained through the first character extraction model indicates successful character extraction, then the new character extraction result is validated to obtain the validation result. If the validation result indicates successful validation, the validation result is output; if the validation result indicates failure, the target image is input into the second character extraction model for character extraction; if the new character extraction result indicates failure, the target image is input into the second character extraction model, and the new character extraction result is obtained through the second character extraction model.

[0098] If the new character extraction result obtained through the second character extraction model indicates that the character extraction was successful, then the new character extraction result is validated to obtain a validation result. Regardless of whether the validation result indicates that the validation was successful or failed, the validation result is output. If the new character extraction result indicates that the character extraction failed, then the validation result used to indicate the validation failure is output.

[0099] In summary, in this embodiment, a pre-trained first character extraction model can be used to extract characters from the target image. This character extraction model is an artificial intelligence model with powerful global context modeling capabilities. It can not only identify the characters themselves but also perform comprehensive reasoning by combining character context information. Therefore, it can achieve more accurate character recognition and extraction. Furthermore, the model's high computational power allows for rapid output of character extraction results, thus improving the efficiency and reliability of character verification. In addition, if verification based on the first character extraction result fails, a second character extraction model can be used to extract characters from the target image, and character verification can be performed based on the new extraction results. Although the second character extraction model takes slightly longer than the first, its accuracy is higher. Using the second character extraction model as an alternative can further improve the reliability of character extraction, thereby enhancing the reliability of character verification.

[0100] Exemplary apparatus Accordingly, this application also provides a character verification device based on image recognition, which is applied to electronic devices, such as terminal devices, mobile phones, computers, etc., or servers.

[0101] like FIG. 3 As shown, the device may include: The first acquisition module 301 is used to acquire the target image.

[0102] The second acquisition module 302 is used to input the target image into a pre-trained first character extraction model to obtain the first character extraction result.

[0103] The first verification module 303 is used to perform character verification based on the first character extraction result and the preset character corresponding to the verification task that matches the target image.

[0104] The third acquisition module 304 is used to input the target image into a pre-trained second character extraction model to obtain the second character extraction result if the verification based on the first character extraction result fails.

[0105] The second character extraction model is more accurate but takes longer than the first character extraction model.

[0106] The second verification module 305 is used to perform character verification based on the second character extraction result and the preset character.

[0107] Output module 306 is used to output the verification results.

[0108] Optionally, the target image is an order review image uploaded to the order review system. The order review image includes the device corresponding to the order to be reviewed and the device's corresponding number information. The preset character is the number information of the device corresponding to the order to be reviewed, which is pre-stored in the order review system.

[0109] Optionally, the device may further include: The recognition and matching module is used to perform character recognition on the target image using a character recognition algorithm, and to perform target character matching on the character recognition results to obtain character matching results; The first processing module is configured to, when the character matching result indicates that the target character is matched, extract the target character according to the character matching result to obtain a third character extraction result, and perform character verification based on the third character extraction result and the preset character; The control module is used to control the execution of the step of inputting the target image into a pre-trained first character extraction model to obtain the first character extraction result when the character matching result indicates that the target character is not matched.

[0110] Optionally, the device may further include: The region determination module is used to determine the target region of the character in the target character extraction result in the target image.

[0111] The target character extraction result is either the first character extraction result, the second character extraction result, or the third character extraction result.

[0112] The detection module is used to perform sharpness detection on the target area and obtain the sharpness detection result.

[0113] The determination module is used to determine the target character extraction result as a valid character extraction result when the sharpness detection result indicates that the sharpness is greater than the sharpness threshold.

[0114] Optionally, the device may further include: The fourth acquisition module is used to acquire the initial sample image.

[0115] The initial sample image is a sample image that has not been labeled with a ground truth value.

[0116] The first annotation module is used to input the initial sample image into a preset model and perform ground truth annotation on the initial sample image through the preset model to obtain an intermediate sample image.

[0117] The first comparison module is used to compare whether the ground truth annotation result of the intermediate sample image is the same as the corresponding real target character.

[0118] The actual target character is pre-set.

[0119] The second processing module is used to use the intermediate sample image as a sample image for training the first character extraction model if the ground truth annotation result of the intermediate sample image is the same as the corresponding real target character; if the ground truth annotation result of the intermediate sample image is not the same as the corresponding real target character, the corresponding real target character is used to replace the ground truth annotation result to obtain a sample image for training the first character extraction model.

[0120] Optionally, the device may further include: The fifth acquisition module is used to acquire the initial sample image.

[0121] The initial sample image is a sample image that has not been labeled with a ground truth value.

[0122] The second annotation module is used to input the initial sample image into a preset model and perform ground truth annotation on the initial sample image through the preset model to obtain an intermediate sample image.

[0123] The second comparison module is used to compare whether the ground truth annotation result of the intermediate sample image is the same as the corresponding real target character.

[0124] The actual target character is pre-set.

[0125] The third processing module is used to identify the intermediate sample image with a first mark when the ground truth annotation result of the intermediate sample image is the same as the corresponding real target character; and to identify the intermediate sample image with a second mark when the ground truth annotation result of the intermediate sample image is not the same as the corresponding real target character; wherein the first mark is used to indicate that the model annotation result is accurate; and the second mark is used to indicate that the model annotation result is incorrect.

[0126] Optionally, the device may further include: The fourth processing module is used to identify the intermediate sample image with a third marker when the real target character corresponding to the intermediate sample image cannot be obtained.

[0127] The third marker is used to indicate whether the ground truth labeling results of the intermediate sample images are correct after manual verification.

[0128] The character verification device based on image recognition provided in this embodiment belongs to the same concept as the character verification method based on image recognition provided in the above embodiments of this application. It can execute the character verification method based on image recognition provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects of the execution method. Technical details not described in detail in this embodiment can be found in the specific processing content of the character verification method based on image recognition provided in the above embodiments of this application, and will not be repeated here.

[0129] It should be understood that the modules in the indicator light display control device described above can be implemented by a processor calling software. For example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to realize the functions of each unit in the device. The processor can be a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. By designing the hardware circuits, some or all of the unit functions can be realized. The hardware circuit can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through a configuration file, thereby realizing the functions of some or all of the above units. All units of the above device can be implemented entirely by a processor calling software, entirely by hardware circuits, or partially by a processor calling software with the remaining parts implemented by hardware circuits.

[0130] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0131] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0132] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a System-on-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0133] Exemplary electronic device This application also provides an electronic device, such as... FIG. 4 As shown, the electronic device includes a memory 400 and a processor 410.

[0134] The memory 400 is connected to the processor 410 and is used to store programs.

[0135] The processor 410 is used to implement the character verification method based on image recognition in the above embodiments by running the program stored in the memory 400.

[0136] Specifically, the aforementioned electronic device may also include: a communication interface 420, an input device 430, an output device 440, and a bus 450.

[0137] The processor 410, memory 400, communication interface 420, input device 430, and output device 440 are interconnected via a bus. Among them: Bus 450 may include a pathway for transmitting information between various components of a computer system.

[0138] The processor 410 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0139] Processor 410 may include a main processor, as well as a baseband chip, modem, etc.

[0140] The memory 400 stores a program that executes the technical solution of this invention, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 400 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0141] Input device 430 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0142] Output device 440 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0143] The communication interface 420 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0144] The processor 410 executes the program stored in the memory 400 and calls other devices, which can be used to implement the various steps of the character verification method based on image recognition provided in the above embodiments of this application.

[0145] Exemplary computer program product and storage medium In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the character verification method based on image recognition described in the embodiments of this application.

[0146] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0147] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0148] Furthermore, embodiments of this application may also be storage media storing a computer program, which is executed by a processor of the steps in the character verification method based on image recognition described in the embodiments of this application.

[0149] Furthermore, embodiments of this application may also be chips, which include processors and data interfaces. The processor reads instructions stored in the memory through the data interface to execute the steps in the character verification method based on image recognition described in the embodiments of this application.

[0150] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0151] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0152] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0153] The modules and sub-modules in the devices and terminals in the various embodiments of this application can be merged, divided, and deleted according to actual needs.

[0154] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0155] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0156] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0157] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0158] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0159] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. An image recognition-based character verification method, characterized by, The method comprises: obtaining a target image and inputting the target image into a pre-trained first character extraction model to obtain a first character extraction result; performing character verification based on the first character extraction result and preset characters corresponding to a verification task matched with the target image; in a case where the character verification based on the first character extraction result fails, inputting the target image into a pre-trained second character extraction model to obtain a second character extraction result, wherein the second character extraction model has higher accuracy but longer time consumption than the first character extraction model; performing character verification based on the second character extraction result and the preset characters; outputting a verification result.

2. The character verification method of claim 1, wherein, The target image is an order verification image uploaded to an order verification system, the order verification image includes a device corresponding to an order to be verified and number information of the device, and the preset characters are number information of the device corresponding to the order to be verified pre-stored in the order verification system.

3. The character verification method of claim 1, wherein, After the target image is obtained, the method further comprises: performing character recognition on the target image through a character recognition algorithm, performing target character matching on a character recognition result, and obtaining a character matching result; in a case where the character matching result indicates that the target character is matched, performing extraction of the target character according to the character matching result to obtain a third character extraction result, and performing character verification based on the third character extraction result and the preset characters; in a case where the character matching result indicates that the target character is not matched, performing the step of inputting the target image into the pre-trained first character extraction model to obtain a first character extraction result.

4. The character verification method of claim 3, wherein, After the target character extraction result is obtained and before the character verification based on the target character extraction result is performed, the method further comprises: determining a target region of a character in the target character extraction result in the target image, wherein the target character extraction result is the first character extraction result, the second character extraction result or the third character extraction result; performing clarity detection on the target region to obtain a clarity detection result; in a case where the clarity detection result indicates that the clarity is greater than a clarity threshold, determining that the target character extraction result is a valid character extraction result.

5. The character verification method of claim 1, wherein, Before the first character extraction model is trained, the method further comprises: obtaining an initial sample image, wherein the initial sample image is a sample image without true value labeling; inputting the initial sample image into a preset model and performing true value labeling on the initial sample image through the preset model to obtain an intermediate sample image; comparing whether the true value labeling result of the intermediate sample image and a corresponding real target character are the same, wherein the real target character is pre-set; if the same, taking the intermediate sample image as a sample image for training the first character extraction model; if not the same, replacing the true value labeling result with the corresponding real target character to obtain a sample image for training the first character extraction model.

6. The character verification method of claim 1, wherein, Before training the first character extraction model, the method further comprises: obtaining an initial sample image; wherein the initial sample image is a sample image without true value annotation; inputting the initial sample image into a preset model, and performing true value annotation on the initial sample image through the preset model to obtain an intermediate sample image; comparing whether the true value annotation result of the intermediate sample image is the same as the corresponding real target character; wherein the real target character is pre-set; if the same, marking the intermediate sample image with a first mark; wherein the first mark is used to indicate that the model annotation result is accurate; if not the same, marking the intermediate sample image with a second mark; wherein the second mark is used to indicate that the model annotation result is wrong.

7. The character verification method of claim 6, wherein, In the case that the real target character corresponding to the intermediate sample image cannot be obtained, the method further comprises: marking the intermediate sample image with a third mark; wherein the third mark is used to indicate whether the true value annotation result of the intermediate sample image is correct.

8. An electronic device, comprising: comprise: a memory and a processor; the memory is connected with the processor, and is used for storing a program; the processor is used for realizing the character verification method based on image recognition according to any one of claims 1 to 7 by running the program in the memory.

9. A storage medium, characterized by The storage medium has a computer program stored thereon, and the computer program is run by the processor to realize the character verification method based on image recognition according to any one of claims 1 to 7.

10. A computer program product, characterised in that, The computer program product has a computer program stored thereon, and the computer program is run by the processor to realize the character verification method based on image recognition according to any one of claims 1 to 7.

Images