Printing defect repairing method and device, terminal equipment and storage medium
Through the combination of optical character recognition technology and preset defect comparison detection model and diffusion generation model, the problem of insufficient complexity and accuracy of traditional printing defect repair methods is solved, and the accurate repair of printed text defects is achieved, thus reducing printing repair complexity.
Patent Information
- Application Number
- CN202411039069.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-12-06
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional printing defect repair methods are complex and difficult to accurately repair specific font defects, which cannot effectively reduce the complexity of printing repair.
The optical character recognition technology combines the preset defect comparison detection model and the diffusion generation model. The text image is detected and recognized through the optical character recognition technology, and the defects of text characters are judged and repaired. The preset diffusion generation model is used for fine-tuning processing. Generate perfect words.
It realizes accurate repair of various types of printed text defects, reduces the complexity of printing repair, and improves the accuracy and efficiency of the repair process.
Smart Images

Figure CN119090776A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of printing, and in particular to a printing defect repair method, apparatus, terminal device and storage medium. Background Art
[0002] Generally speaking, printing defects exist in two parts. The first part is that the document itself may have missing text (such as photocopies). Another common situation is that the printing defects are caused by problems with the printing equipment. Traditional printing defect repair often repairs printing defects through pre-set features or rules. However, this method may fail to correctly repair specific fonts, or the pre-set process is too complicated, making it difficult to put into actual engineering use. Summary of the invention
[0003] In view of this, the present application provides a printing defect repair method, apparatus, terminal device and storage medium, which can perform various types of repairs on defects generated by various types of printed texts, thereby reducing the complexity of the entire printing repair.
[0004] A printing defect repairing method, comprising:
[0005] Detecting and recognizing the text image by optical character recognition technology to obtain the predicted text character content and the predicted position of the predicted text character content in the text image;
[0006] Obtain the corresponding text character image according to the predicted position;
[0007] The text character image and the standard character corresponding to the text character prediction content are sent to a preset defect comparison detection model in pairs for recognition to determine whether the text character in the text character image has defects;
[0008] If yes, the defective text character image and the text character prediction content are sent to the preset diffusion generation model for fine-tuning to obtain the required perfect character;
[0009] Repair defects in text images based on perfect words and predicted positions.
[0010] In one embodiment, before the step of pairing the text character image with the standard character corresponding to the text character prediction content into a preset defect comparison detection model for recognition to determine whether the text character in the text character image has defects, the step also includes:
[0011] Establish an initial defect comparison detection model based on a preset lightweight classification network;
[0012] The initial defect contrast detection model is trained to obtain a preset defect contrast detection model.
[0013] In one embodiment, the initial defect contrast detection model includes a first input channel, a second input channel, and a third input channel. The step of training the initial defect contrast detection model to obtain a preset defect contrast detection model includes:
[0014] Obtain image datasets of single words in preset standard fonts, image datasets of defective words in various fonts, and image datasets of perfect words;
[0015] Inputting the image data of each single word in the preset standard font into the first input channel;
[0016] Inputting defective word image data or perfect word image data of various fonts of each single word into a second input channel;
[0017] Inputting the input pixel difference between the first input channel and the second input channel to the third input channel;
[0018] The first input channel, the second input channel and the third input channel are spliced, and the spliced three-channel format is sent to a preset lightweight classification network for classification to obtain a classification result;
[0019] It is determined whether there is a defective word in the input content of the second input channel according to the classification result, so as to train and optimize the initial defect contrast detection model and obtain a preset defect contrast detection model.
[0020] In one embodiment, the channel attention weight of the initial defect contrast detection model is calculated using the following formula:
[0021]
[0022] A ij is the input of the first input channel, B ij is the input of the second input channel, C ij is the input of the third input channel, H and W are both positive integers, w_A is the global average of the first input channel, w_B is the global average of the second input channel, w_C is the global average of the third input channel, softmax is the activation function, w′ is the channel attention weight, w_A′ is the attention weight of the third input channel, w_B′ is the attention weight of the third input channel, and w_C′ is the attention weight of the third input channel.
[0023] In one embodiment, the grouped convolutional head structures of the input channel and the output channel of the initial defect contrast detection model each adopt a channel unbalanced attention mechanism, and the loss function of the initial defect contrast detection model is as follows:
[0024]
[0025] L_t is the loss function, L_c is the standard classification loss, L_a is the L2 regularization of the attention weight of the third input channel, w_C′ is the attention weight of the third input channel, is the mean of the attention weights w_C′ of the third input channel, and λ is a hyperparameter for weighing the loss.
[0026] In one embodiment, the hyperparameter λ of the trade-off loss is set to a value of 0.01 to 0.02.
[0027] In one embodiment, the training method of the preset diffusion generation model is as follows:
[0028] Re-encoding the defective text character image through the first input structure to obtain first encoded character data;
[0029] Re-encode the text character prediction content according to the second input result to obtain second encoded character data;
[0030] Obtaining a correct word image of a defective text character image as a label value;
[0031] The initial diffusion generation model is trained according to the first encoded character data, the second encoded character data and the label value to obtain a preset diffusion generation model.
[0032] In addition, a printing defect repairing device is also provided, comprising:
[0033] A detection and recognition device, used to detect and recognize the text image by optical character recognition technology to obtain the predicted content of the text characters and the predicted position of the predicted content of the text characters in the text image;
[0034] An image acquisition unit, used to acquire a corresponding text character image according to the predicted position;
[0035] A defect judgment unit is used to send the text character image and the standard character corresponding to the text character prediction content in pairs to a preset defect comparison detection model for recognition, so as to judge whether the text character in the text character image has defects;
[0036] A perfect character generation unit is used for sending the defective text character image and the text character prediction content to a preset diffusion generation model for fine-tuning processing when there are defects in the text character in the text character image, so as to obtain the desired perfect character;
[0037] The repair unit is used to repair defects in the text image based on the perfect word and the predicted position.
[0038] In addition, a terminal device is also provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the above-mentioned printing defect repairing method are implemented.
[0039] In addition, a storage medium is also provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned printing defect repairing method are implemented.
[0040] The above-mentioned printing defect repair method includes detecting and identifying the text image through optical character recognition technology to obtain the text character prediction content and the predicted position of the text character prediction content in the text image, obtaining the corresponding text character image according to the predicted position, and sending the text character image and the standard character corresponding to the text character prediction content into a preset defect comparison detection model for recognition in pairs to determine whether the text character in the text character image has defects. If so, the defective text character image and the text character prediction content are sent to the preset diffusion generation model for fine-tuning to obtain the required perfect characters, and the defects in the text image are repaired according to the perfect characters and the predicted positions. Through the mutual cooperation of the optical character recognition technology, the preset defect comparison detection model and the preset diffusion generation model, various types of defects generated by text printing characters can be accurately repaired, reducing the complexity of printing repair. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 A schematic diagram of a process for repairing a printing defect provided in an embodiment of the present application;
[0042] Figure 2 A schematic diagram of a process of a printing defect repairing method provided by another embodiment of the present application;
[0043] Figure 3 A schematic flow chart of a method for training an initial defect contrast detection model to obtain a preset defect contrast detection model provided in an embodiment of the present application;
[0044] Figure 4 A schematic diagram of a method flow for generating a preset diffusion generation model provided in an embodiment of the present application;
[0045] Figure 5 A schematic block diagram of the structure of a printing defect repairing device provided in one embodiment of the present application;
[0046] Figure 6 A schematic block diagram of the internal structure of a terminal device provided in one embodiment of the present application.
[0047] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0048] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.
[0049] In addition, if the description of "first", "second", etc. is involved in this application, it is only used for descriptive purposes (such as for distinguishing the same or similar elements), and cannot be understood as indicating or suggesting its relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.
[0050] Under current technology, the model of OCR text recognition usually combines the technology of long short-term memory network, which enables it to ignore some subtle errors when performing text image recognition and give more accurate answers through contextual relationships. For example, due to printing (or electronic archives) problems, "The weather is very good today" is printed as "The weather is very good today", but because in the training process, OCR recognition technology has usually seen many correct word relationships that conform to grammar and speech logic, it can automatically recognize and predict the correct text "The weather is very good today" according to the maximum possible probability. Through this feature, OCR recognition technology can predict what the correct words and sentences in the user or original text should be. However, OCR itself cannot determine where the defective word is in the image on the specific image content (in fact, OCR correctly predicts the six words of "The weather is very good today", but it does not know which word is defective on the original image, because it only outputs the text content with the maximum probability according to the image, but does not know that the original very is written as Gen in the image).
[0051] Based on the above problems, Figure 1 As shown, a printing defect repair method is provided, comprising:
[0052] Step 110 , detecting and recognizing the text image by using optical character recognition technology to obtain the predicted text character content and the predicted position of the predicted text character content in the text image.
[0053] Among them, Optical Character Recognition technology is Optical Character Recognition, referred to as OCR recognition technology.
[0054] Step 120: Obtain the corresponding text character image according to the predicted position.
[0055] Step 130, the text character image and the standard character corresponding to the text character prediction content are sent to a preset defect comparison detection model in pairs for recognition to determine whether the text character in the text character image has defects. If there are defects, enter step S140 for processing.
[0056] In this embodiment, the preset defect comparison detection model is a classification and recognition model based on a neural network.
[0057] Step S140, sending the defective text character image and the text character prediction content to a preset diffusion generation model for fine-tuning processing to obtain the desired perfect character.
[0058] Step S150, repairing defects in the text image according to the perfect characters and the predicted positions.
[0059] Through the coordination of optical character recognition technology, preset defect comparison detection model and preset diffusion generation model, various types of defects generated by text printing characters can be accurately repaired, reducing the complexity of printing repair.
[0060] In one embodiment, Figure 2 As shown, before step S130, the following steps are also included:
[0061] Step S160, establishing an initial defect comparison detection model based on a preset lightweight classification network.
[0062] Step S170, training the initial defect contrast detection model to obtain a preset defect contrast detection model.
[0063] In this embodiment, the preset lightweight classification network may use mobilenetv4 for classification.
[0064] In one embodiment, the initial defect contrast detection model includes a first input channel, a second input channel and a third input channel, such as Figure 3 As shown, step S170 includes:
[0065] Step S171, obtaining an image dataset of single words in a preset standard font, an image dataset of defective words in various fonts, and an image dataset of perfect words.
[0066] In this embodiment, the image data set of the single characters in the preset standard font can be generated by all Chinese characters, and the image size can be 64*64 pixels.
[0067] In this embodiment, a data augmentation method is used to generate defective characters in various fonts (including handwriting generated using an adversarial generative model). For example, a local erasure method is used to erase some pixels on the original text image content. Note that the erased pixels should not exceed 50% of the valid pixels in the text content.
[0068] In this embodiment, perfect characters of various fonts (including handwriting generated using an adversarial generation model) are generated.
[0069] Step S172, inputting the image data of each single word in the preset standard font into the first input channel.
[0070] Step S173, inputting the defective character image data or the perfect character image data of each single character in various fonts into the second input channel.
[0071] Step S174, inputting the input pixel difference between the first input channel and the second input channel into the third input channel;
[0072] Step S175, splicing the first input channel, the second input channel and the third input channel, and sending the spliced three-channel format to a preset lightweight classification network for classification to obtain a classification result.
[0073] Step S176, judging whether there is a defective word in the input content of the second input channel according to the classification result, so as to train and optimize the initial defect contrast detection model to obtain a preset defect contrast detection model.
[0074] In this embodiment, the preset lightweight classification network may use mobilenetv4 for classification.
[0075] In one embodiment, the channel attention weight of the initial defect contrast detection model is calculated using the following formula:
[0076]
[0077] A ij is the input of the first input channel, B ij is the input of the second input channel, C ijis the input of the third input channel, H and W are both positive integers, w_A is the global average of the first input channel, w_B is the global average of the second input channel, w_C is the global average of the third input channel, softmax is the activation function, w′ is the channel attention weight, w_A′ is the attention weight of the third input channel, w_B′ is the attention weight of the third input channel, and w_C′ is the attention weight of the third input channel.
[0078] In one embodiment, the grouped convolutional head structures of the input channel and the output channel of the initial defect contrast detection model each adopt a channel unbalanced attention mechanism, and the loss function of the initial defect contrast detection model is as follows:
[0079]
[0080] L_t is the loss function, L_c is the standard classification loss, L_a is the L2 regularization of the attention weight of the third input channel, w_C′ is the attention weight of the third input channel, is the mean of the attention weights w_C′ of the third input channel, and λ is a hyperparameter for weighing the loss.
[0081] In this embodiment, since the third input channel is a difference channel, by setting the hyperparameter λ of the trade-off loss, the entire detection model can automatically learn the attention of the third input channel, and try to occupy more than half of the entire channel, so that the entire detection model has better performance, that is, greatly improving the classification performance of the entire defect comparison detection model.
[0082] In one embodiment, the hyperparameter λ of the trade-off loss is set to a value of 0.01 to 0.02.
[0083] In one embodiment, Figure 4 As shown, the training method of the preset diffusion generation model is as follows:
[0084] Step S180, re-encoding the defective text character image through the first input structure to obtain first encoded character data.
[0085] Step S181, re-encode the text character prediction content through the second input result to obtain second encoded character data.
[0086] Step S182, obtaining the correct character image of the defective text character image as a label value.
[0087] Step S183, training the initial diffusion generation model according to the first coded character data, the second coded character data and the label value to obtain a preset diffusion generation model.
[0088] In addition, if Figure 5As shown, a printing defect repairing device 200 is also provided, comprising:
[0089] The detection and recognition unit 210 is used to detect and recognize the text image by using the optical character recognition technology to obtain the predicted content of the text characters and the predicted position of the predicted content of the text characters in the text image.
[0090] The image acquisition unit 220 is used to acquire the corresponding text character image according to the predicted position.
[0091] The defect judgment unit 230 is used to send the text character image and the standard character corresponding to the text character prediction content in pairs to a preset defect comparison detection model for recognition to determine whether the text character in the text character image has defects.
[0092] The perfect character generating unit 240 is used to send the defective text character image and the text character prediction content to the preset diffusion generation model for fine-tuning processing to obtain the desired perfect character when there are defects in the text character in the text character image.
[0093] The repair unit 250 is used to repair defects in the text image according to the perfect characters and the predicted positions.
[0094] In addition, a terminal device is also provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the above-mentioned printing defect repairing method when executed by the processor.
[0095] In addition, a storage medium is also provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned printing defect repairing method are implemented.
[0096] In addition, a terminal device is also provided in an embodiment of the present application. The internal structure of the terminal device can be as follows: Figure 6 As shown. The terminal device includes a processor, a memory, a communication interface and a database connected via a system bus. The processor is used to provide computing and control capabilities. The memory of the terminal device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the terminal device is used to store data called by the computer program. The communication interface of the terminal device is used to communicate data with an external terminal. The input device of the terminal device is used to receive a signal input by an external device. When the computer program is executed by the processor, a printing defect repair method as described in the above embodiment is implemented.
[0097] Those skilled in the art will understand that Figure 6 The structure shown in is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the terminal device to which the solution of the present application is applied.
[0098] In addition, the present application also proposes a storage medium, the storage medium includes a computer program, and the computer program, when executed by a processor, implements the steps of the printing defect repair method described in the above embodiment. It can be understood that the storage medium in this embodiment can be a volatile storage medium or a non-volatile storage medium.
[0099] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0100] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.
[0101] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A printing defect repair method, characterized in that: include: Detecting and recognizing the text image by optical character recognition technology to obtain text character prediction content and the predicted position of the text character prediction content in the text image; Acquire a corresponding text character image according to the predicted position; Sending the text character image and the standard character corresponding to the text character prediction content in pairs to a preset defect comparison detection model for recognition to determine whether the text character in the text character image has defects; If yes, the defective text character image and the text character prediction content are sent to a preset diffusion generation model for fine-tuning to obtain the desired perfect character; The defects in the text image are repaired according to the perfect word and the predicted position.
2. The printing defect repairing method according to claim 1, characterized in that: Before the step of pairing the text character image with the standard character corresponding to the text character prediction content into a preset defect comparison detection model for recognition to determine whether the text character in the text character image has defects, the step also includes: Establish an initial defect comparison detection model based on a preset lightweight classification network; The initial defect contrast detection model is trained to obtain the preset defect contrast detection model.
3. The printing defect repairing method according to claim 2, characterized in that: The initial defect contrast detection model includes a first input channel, a second input channel and a third input channel. The step of training the initial defect contrast detection model to obtain the preset defect contrast detection model includes: Obtain image datasets of single words in preset standard fonts, image datasets of defective words in various fonts, and image datasets of perfect words; Inputting the image data of each single word in the preset standard font into the first input channel; Inputting defective word image data or perfect word image data of various fonts of each single word into the second input channel; Inputting the input pixel difference between the first input channel and the second input channel to the third input channel; splicing the first input channel, the second input channel and the third input channel, and sending the spliced three-channel format to the preset lightweight classification network for classification to obtain a classification result; It is determined whether there is a defective word in the input content of the second input channel according to the classification result, so as to train and optimize the initial defect contrast detection model to obtain the preset defect contrast detection model.
4. The printing defect repairing method according to claim 3, characterized in that: The channel attention weight of the initial defect contrast detection model is calculated using the following formula: A ij is the input of the first input channel, B ij is the input of the second input channel, C ij is the input of the third input channel, H and W are both positive integers, wA is the global average of the first input channel, wB is the global average of the second input channel, w_C is the global average of the third input channel, softmax is the activation function, w′ is the channel attention weight, w_A′ is the attention weight of the third input channel, wB′ is the attention weight of the third input channel, and w_C′ is the attention weight of the third input channel.
5. The printing defect repairing method according to claim 3, characterized in that: The grouped convolutional head structures of the input channel and the output channel of the initial defect contrast detection model both adopt a channel unbalanced attention mechanism. The loss function of the initial defect contrast detection model is as follows: L_t is the loss function, L_c is the standard classification loss, L_a is the L2 regularization of the attention weight of the third input channel, w_C′ is the attention weight of the third input channel, is the mean of the attention weights w_C′ of the third input channel, and λ is a hyperparameter for weighing the loss.
6. The printing defect repairing method according to claim 5, characterized in that: The hyperparameter λ of the trade-off loss is set to a value of 0.01 to 0.
02.
7. The printing defect repairing method according to claim 1, characterized in that: The training method of the preset diffusion generation model is as follows: Re-encoding the defective text character image through a first input structure to obtain first encoded character data; Re-encode the text character prediction content according to the second input result to obtain second encoded character data; Obtaining a correct character image of the defective text character image as a label value; The initial diffusion generation model is trained according to the first encoded character data, the second encoded character data and the label value to obtain a preset diffusion generation model.
8. A printing defect repairing device, characterized in that: include: A detection and recognition device for detecting and recognizing a text image by using an optical character recognition technology to obtain text character prediction content and a predicted position of the text character prediction content in the text image; An image acquisition unit, used to acquire a corresponding text character image according to the predicted position; A defect judgment unit, used for sending the text character image and the standard character corresponding to the text character prediction content in pairs to a preset defect comparison detection model for recognition, so as to judge whether the text character in the text character image has defects; A perfect character generation unit, for sending the defective text character image and the text character prediction content to a preset diffusion generation model for fine-tuning processing to obtain a desired perfect character when there are defects in the text character in the text character image; A repairing unit is used to repair defects in the text image according to the perfect word and the predicted position.
9. A terminal device, characterized in that: The terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the printing defect repairing method as described in any one of claims 1 to 7.
10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the printing defect repairing method according to any one of claims 1 to 7 are implemented.
