Model training method and apparatus, data processing method and apparatus, electronic device, storage medium, and program

Abstract

To provide a model training method and apparatus, a data processing method and apparatus, an electronic device, a storage medium, and a program.SOLUTION: The model training method includes: acquiring a collection of history process flow cards in a spinning process; extracting a handwritten area from each history process flow card; classification-processing the handwritten area to obtain an image block of handwritten numbers and an image block of a handwritten text; on the basis of the image block of the handwritten numbers, forming a number identification model for different handwriting categories of numbers, to extract a target number from a newly added process flow card; and on the basis of the image block of the handwritten text, forming a text identification model for different handwriting categories of texts, to extract a target text from the newly added process flow card. The target number and target text are used to form a process flow database of the spinning process.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to the field of data processing, and in particular to technical fields such as large scale models, image processing, and computer vision. [Background technology]

[0002] In the industrial scenario of the spinning process, the spinning process flow is long and complicated. Currently, some important parts need to be recorded manually. The related process data recorded manually also needs to be manually uploaded to the system to digitize and store the spinning process flow. However, manual work is resource-intensive and inefficient. Summary of the Invention [Problem to be solved by the invention]

[0003] The present disclosure provides a model training method and apparatus, a data processing method and apparatus, an electronic device, a storage medium, and a program to solve or alleviate one or more technical problems in the prior art. [Means for solving the problem]

[0004] In a first aspect, the present disclosure provides a model training method, the method comprising: obtaining a set of historical process flow cards for a spinning process; extracting a handwritten region from each historical process flow card in the collection of historical process flow cards; classifying the handwritten region to obtain image blocks of handwritten digits and image blocks of handwritten text; Building a digit identification model for different cursive categories of digits to extract target digits from the newly added process flow card based on the image blocks of handwritten digits; Building a text identification model for different script categories of text to extract target text from the newly added process flow card based on the image block of handwritten text, where the target numbers and target text are used to build a process flow database of the spinning process.

[0005] In a second aspect, the present disclosure provides a data processing method, the method comprising: When it is determined that the content of the newly added process flow card is complete, performing a classification process on the newly added process flow card to obtain a print area and a cursive area in the newly added process flow card; performing character identification on the print area using a character extraction technique; identifying a cursive category in the cursive domain; When the cursive region includes a first target cursive category of a number, performing character identification based on a number identification model corresponding to the first target cursive category; If the cursive region includes a second target cursive category of text, performing character identification based on a text identification model corresponding to the second target cursive category, and the identified characters are used to construct a process flow database for the spinning process.

[0006] In a third aspect, the present disclosure provides a model training apparatus, comprising: an acquisition module for acquiring a set of historical process flow cards of the spinning process; an extraction module for extracting handwritten regions from each historical process flow card in the collection of historical process flow cards; a classification module for classifying the handwritten region to obtain image blocks of handwritten digits and image blocks of handwritten text; a digit identification module for constructing a digit identification model of different cursive categories of digits to extract target digits from the newly added process flow card based on the image block of handwritten digits; a text identification module for constructing a text identification model of different script categories of text to extract target text from the newly added process flow card based on the image block of handwritten text, wherein the target numbers and target text are used to construct a process flow database of the spinning process.

[0007] In a fourth aspect, the present disclosure provides a data processing apparatus, comprising: a classification module for performing a classification process on the newly added process flow card when it is determined that the content of the newly added process flow card is complete, and obtaining a print area and a cursive area in the newly added process flow card; a first identification module for performing character identification on the print area using a character extraction technique; a second identification module for identifying a cursive category in the cursive region; a third identification module for performing character identification based on a number identification model corresponding to the first target cursive category when the cursive region includes the first target cursive category of number; and a fourth identification module for performing character identification based on a text identification model corresponding to the second target cursive category when the cursive region includes the second target cursive category of text, wherein the identified characters are used to construct a process flow database of the spinning process.

[0008] In a fifth aspect, the present disclosure provides an electronic device, the device comprising: at least one processor; a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, cause the implementation of any one of the methods in the embodiments of the present disclosure.

[0009] In a sixth aspect, there is provided a non-transitory computer-readable storage medium having stored thereon computer instructions for causing a computer to perform any one of the methods in the embodiments of the present disclosure.

[0010] In a seventh aspect, there is provided a program, which, when executed by a processor, implements any one of the methods in the embodiments of the present disclosure.

[0011] According to the method provided by the embodiments of the present disclosure, the identification accuracy can be improved, thereby obtaining accurate process flow data.

[0012] It should be understood that the contents described herein are not intended to describe key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will be better understood through the following specification.

[0013] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the accompanying drawings indicate the same or similar components or elements. The accompanying drawings are not necessarily drawn to scale. It should be understood that the drawings illustrate only some examples provided by the present disclosure and should not be considered as limiting the scope of the present disclosure. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a schematic diagram illustrating a flow of a model training method according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a schematic diagram illustrating a process flow card according to one embodiment of the present disclosure. [Figure 3a] FIG. 2 is a schematic diagram illustrating a first sub-image block according to an embodiment of the present disclosure. [Figure 3b] FIG. 10 is a schematic diagram illustrating a second sub-image block according to an embodiment of the present disclosure. [Figure 4a] FIG. 1 is a schematic diagram illustrating a masking process according to an embodiment of the present disclosure. [Figure 4b] FIG. 1 is a schematic diagram illustrating a text overlay on text according to one embodiment of the present disclosure. [Figure 4c] FIG. 10 is a schematic diagram illustrating tampering of text in a second sub-image block according to an embodiment of the present disclosure. [Figure 5] FIG. 2 is a schematic diagram illustrating the flow of a data processing method according to an embodiment of the present disclosure. [Figure 6] FIG. 1 is a schematic diagram illustrating a configuration of a model training device according to an embodiment of the present disclosure. [Figure 7] 1 is a schematic diagram illustrating a configuration of a data processing device according to an embodiment of the present disclosure. [Figure 8] FIG. 1 is a block diagram of an electronic device for implementing a model training method and a data processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0015] The present disclosure will now be described in more detail with reference to the accompanying drawings, in which like reference numerals represent like or similar elements and in which various aspects of the embodiments are shown, and which, unless otherwise noted, are not necessarily drawn to scale.

[0016] Furthermore, in order to better explain the present disclosure, many specific details are described in the following specific embodiments. Those skilled in the art should understand that the present disclosure can be similarly implemented without some details. In some examples, methods, means, components, circuits, etc. that are well known to those skilled in the art are not described in detail so that the gist of the present disclosure is clear.

[0017] Digitization of information helps improve the efficiency of information management, especially in the spinning process, which is long and complex. Digitizing each step in the spinning process flow and archiving it in a system helps track each step in the spinning process, improves process efficiency, and provides effective data support for identifying problems. However, currently, some of the process data for the spinning process must be recorded manually and uploaded to the system, which is inefficient and prone to errors.

[0018] Therefore, in order to accurately and efficiently identify the information in the process flow card and facilitate the construction of a spinning process database, the embodiments of the present disclosure combine the advantages of the neural network model with the characteristics of the spinning process to propose a scheme for using a neural network model to identify different handwriting styles. Below, we first describe the model training method in this scheme, and then describe the process of automatically identifying the process flow card using the trained model.

[0019] As shown in FIG. 1 , an embodiment of the present disclosure proposes a model training method, in which the trained model is used to extract content in a process flow card, and the method includes the following steps:

[0020] In S101, a set of historical process flow cards of the spinning process is obtained.

[0021] Here, the process flow data may include equipment data, weighing data for individual wound yarn packages, fineness inspection data, woven sock dyeing assessment data, visual inspection data, automatic packaging line data, laboratory chemical inspection data, wound yarn package trolley transport data, warehouse management data, supply chain data, etc. Therefore, any process flow may have a process flow card template.

[0022] Here, an example of a process flow card may include a printed area and a handwritten area that a worker needs to fill in, as shown in Figure 2 (Figure 2 shows characters in the printed area).

[0023] Here, the number may be used to determine which process the process flow card belongs to, and other contents may be changed according to the actual process, and the embodiments of the present disclosure are not limited thereto.

[0024] Embodiments of the present disclosure focus on identifying cursive writing, as print is generally clear and easy to identify.

[0025] In S102, a handwritten region is extracted from each historical process flow card in the collection of historical process flow cards.

[0026] In S103, the handwritten region is classified to obtain image blocks of handwritten numbers and image blocks of handwritten text.

[0027] In the spinning process, the workers are complex, and there are differences in the handwriting styles of different workers, so it is not practical to train models individually for different workers. Therefore, in the embodiment of the present disclosure, classification is performed for different handwriting categories, and training is performed to obtain a discriminative model suitable for each handwriting category. See S104 and S105 for details.

[0028] In S104, based on the image blocks of handwritten digits, a digit identification model of different cursive categories of digits is constructed to extract target digits from the newly added process flow card.

[0029] In S105, a text identification model for different script categories of text is constructed based on the image block of handwritten text to extract target text from the newly added process flow card, and the target numbers and target text are used to construct a process flow database for the spinning process.

[0030] The execution timing of S104 and S105 is not limited.

[0031] In an embodiment of the present disclosure, handwritten regions are extracted from each historical process flow card in a collection of historical process flow cards, and the handwritten regions are classified to obtain image blocks of handwritten digits and image blocks of handwritten text. Corresponding digit recognition models are trained for the cursive categories of the image blocks of handwritten digits, and corresponding text recognition models are trained for the cursive categories of the image blocks of handwritten text, so as to obtain target digits and target text for use in building a process flow database. This method enables automatic uploading of relevant data in the spinning process flow, saving human resources. Furthermore, because different people have different handwriting styles for numbers and text, and the visual effects of the same digits and text are different, classification processing can be performed on the cursive characters to sort out the characteristics of cursive characters in the same category. The corresponding recognition models are trained to complete the recognition operation, improving recognition accuracy and thereby obtaining accurate process flow data.

[0032] In some embodiments, multiple classification models are created for different handwriting types to improve classification accuracy.

[0033] 1) We build a digit identification model for different cursive categories of digits.

[0034] In some embodiments, building different cursive categories for numbers can be performed as follows.

[0035] In step A1, the image block of handwritten numbers is divided into first sub-image blocks, each corresponding to a cell in the history process flow card.

[0036] As shown in Figure 3a, there may be handwritten numbers and handwritten text in one cell, and each number and text in the cell may be divided separately and sequentially, and the image block belonging to the handwritten numbers may be determined as the first sub-image block. Of course, there may be cells that only contain handwritten numbers, and the image block in the cell may be determined as the first sub-image block.

[0037] In step A2, a feature extraction operation is performed on each of the first sub-image blocks to obtain a first feature vector for each of the first sub-image blocks.

[0038] A relatively basic convolutional neural network can be used to perform feature extraction operations on each of the first sub-image blocks to obtain a first feature vector for each of the first sub-image blocks. Modified convolutional neural networks can also be used. Modified networks include, for example, a pre-trained backbone network for image feature extraction and a Transformer network incorporating an attention mechanism. Any method capable of obtaining a first feature vector for a first sub-image block can be applied to the embodiments of the present disclosure.

[0039] In step A3, a clustering analysis is performed on the first feature vector of each first sub-image block to obtain different cursive categories of numbers.

[0040] Since writing styles may vary from person to person, performing clustering analysis on the first feature vector of each first sub-image block is equivalent to clustering fonts that belong to the same writing style and distinguishing fonts that have different writing styles, thereby obtaining different cursive categories for numbers.

[0041] In the embodiments of the present disclosure, feature extraction is performed on a first sub-image block in an image block of handwritten digits to obtain a first feature vector of the first sub-image block, and then a clustering operation is performed on the first feature vector to obtain different cursive categories, which can then support accurate identification.

[0042] After obtaining the different cursive categories of numbers, building a number identification model for the different cursive categories of numbers based on the image blocks of handwritten numbers can be implemented as follows.

[0043] In step B1, a first large scale model is trained based on the corresponding labeling data of each handwritten digit image block, so that the first large scale model can identify the digit content in each historical process flow card.

[0044] The labeling data is a specific numerical value corresponding to the first sub-image block of the image block of the handwritten digits, for example, the first sub-image block 1 in FIG. 3a has the corresponding labeling data 2346-7.

[0045] Here, the first large-scale model may be a large-scale model such as the Zeus model of the ERNIE3.0 large-scale model version 3.0. ERNIE3.0 Zeus uses prompt information during training. The large-scale model has the ability to learn based on prompts. Simply put, prompt learning involves input information for the first large-scale model that includes a task description, example samples, and target samples, where the task description is the prompt.

[0046] Here, the target sample is an image block of handwritten digits, the task description is "identify digits in the target sample", and the reference example can be "image block 1 of handwritten digits, predicted result 2346-7". In the training process, the first large-scale model can predict the target sample based on the task description and the reference example to obtain a prediction result, compare the prediction result with the labeling result to obtain a first loss value, and adjust the first large-scale model based on the first loss value to obtain a first large-scale model that satisfies the first convergence condition.

[0047] Here, the first convergence condition is that the first loss value is stable or that a preset number of iterations is satisfied, but the embodiment of the present disclosure is not limited thereto.

[0048] In step B2, the following operations are performed for each number cursive category.

[0049] In step B21, a first model awaiting optimization corresponding to the cursive writing category is distilled from the first large-scale model.

[0050] Using the knowledge distillation (KD) method, a first awaiting optimization model corresponding to the cursive writing category can be distilled from the first large-scale model. That is, a small model with fewer parameters, i.e., a first awaiting optimization model, is constructed based on the model structure of the first large-scale model.

[0051] Of course, it is also possible to obtain a model ready for first optimization using distillation techniques such as lightweight model architecture, pruning, and quantization.

[0052] In step B22, image blocks of handwritten digits in the cursive category are used to construct a first training sample set for a first model awaiting optimization.

[0053] Here, the image blocks of handwritten digits can be divided, rotated, masked, occluded, contaminated (e.g., sketched, falsified valid information), etc. to construct a first training sample set.

[0054] In step B23, a first model-to-be-optimized is trained based on the first training sample set.

[0055] Here, the first model to be optimized is a region convolutional neural network (Region-CNN, R-CNN), which includes a first network, a second network, and a third network. Here, the first network is used to extract feature information of each handwritten digit image block in the first training sample set. The second network is used to determine a target region of each handwritten digit image block. The third network is used to identify content in the region.

[0056] During implementation, a first training sample set is input to a first network to obtain feature information for each image block of handwritten digits in the first training sample set, and this feature information is input to a second network to obtain a set of region candidate frames and a set of foreground candidate frames.

[0057] Furthermore, the foreground and background of the handwritten digit image block are divided into pre-defined regions to obtain a set of foreground frame regions. The second network filters the set of candidate region frames and determines the set of candidate region frames with high foreground probability as the foreground candidate frame set.

[0058] A comparison can be made based on the foreground candidate frame set and the foreground frame region set, and a screening loss used to represent the difference between the two can be determined. After determining the foreground candidate frame set based on the method described above, content in the foreground candidate frame set is identified based on a third network, an identification result is obtained, and a detection loss between the identification result and the labeling data is determined.

[0059] Based on the screening loss and the detection loss, the learnable parameters of the first model awaiting optimization are adjusted, and if the training convergence condition is met, the training of the first model awaiting optimization is terminated.

[0060] Here, the first convergence condition may be that the comparative training loss and the detection loss tend to stabilize, or may be that a preset number of iterations is satisfied, and the embodiments of the present disclosure are not limited thereto.

[0061] In step B24, if the classification accuracy of the model awaiting first optimization reaches a bottleneck and does not meet the desired accuracy, samples whose classification error rate is higher than the first error rate threshold are selected from the first training sample set to construct a small sample set.

[0062] In step B25, the first optimization-ready model is trained to learn the knowledge of the small sample, and a digit identification model corresponding to the cursive writing category is obtained.

[0063] When the classification accuracy of the first model awaiting optimization reaches a bottleneck and does not meet the expected accuracy, a small sample set is input to a first network in the model awaiting first optimization to obtain feature information of the samples in the small sample set. The feature information is input to a second network to obtain a set of region candidate frames. Each candidate frame in the set of region candidate frames is designated as a first candidate frame, and positive and negative sample pairs are constructed for each first candidate frame. Here, the positive sample pair is constructed by using a sample whose content in the candidate frame is similar to that of the first candidate frame as a positive sample, and a sample whose content is dissimilar to that of the first candidate frame as a negative sample, and a negative sample pair is constructed by using a sample whose content is dissimilar to that of the first candidate frame as a negative sample. Note that each of the candidate frames in the positive and negative sample pairs must have a partial overlap with the first candidate frame. This allows the construction of positive and negative sample pairs whose content has a certain degree of correlation with the first candidate frame. A comparative training loss can further be determined based on the feature distance between the two candidate frames in the positive sample pair and the feature distance between the two candidate frames in the negative sample pair. The first optimization waiting model is adjusted based on the comparative learning loss to obtain a first optimization waiting model that satisfies the second convergence condition.

[0064] The second convergence condition may be that the comparative learning loss tends to stabilize, or may be that a preset number of iterations is satisfied, and the embodiments of the present disclosure are not limited thereto.

[0065] It is necessary to construct a corresponding number of number identification models based on the number of number cursive categories, and one number identification model is associated with each number cursive category.

[0066] In an embodiment of the present disclosure, to achieve accurate identification of handwritten numbers, different number identification models are constructed for different cursive types.

[0067] 2) Build a text identification model for different script categories of text.

[0068] In some embodiments, building different cursive categories of text can be performed as follows.

[0069] In step C1, the image block of the handwritten text is divided into second sub-image blocks, each corresponding to a cell in the history process flow card, as a division unit.

[0070] As shown in Figure 3b, there may be handwritten numbers and text in a cell, and each number and text in the cell may be divided separately and the image block belonging to the handwritten text may be determined as the second sub-image block. Of course, there may be a cell that only contains handwritten text, and the image block in that cell may be determined as the second sub-image block.

[0071] In step C2, a feature extraction operation is performed on each of the second sub-image blocks to obtain a second feature vector for each of the second sub-image blocks.

[0072] The method for obtaining the second feature vector is the same as the method for obtaining the first feature vector, and will not be repeated in the embodiments of the present disclosure.

[0073] In step C3, a clustering analysis is performed on the second feature vector of each second sub-image block to obtain different cursive categories of the text.

[0074] In an embodiment of the present disclosure, feature extraction is performed on a second sub-image block in an image block of handwritten text to obtain a second feature vector of the second sub-image block, and then a clustering operation is performed on the second sub-image block to obtain different handwriting categories, which can then support accurate identification.

[0075] After obtaining the different cursive categories of text, building a text identification model for the different cursive categories of text based on image blocks of handwritten text can be implemented as follows.

[0076] In step D1, a second large scale model is trained so that the second large scale model can identify the text content in each historical process flow card based on the corresponding labeling data of each image block of handwritten text.

[0077] The labeling data is a specific text corresponding to the second sub-image block of the image block of handwritten text, for example, for the second sub-image block 1 in FIG. 3b, the corresponding labeling data is LDQ.

[0078] Here, the second large-scale model may be the same model as the first large-scale model, and the second large-scale model may be trained using the training method of the first large-scale model, and the embodiments of the present disclosure are not limited thereto.

[0079] In step D2, for each cursive text category of the text, the following operations are performed:

[0080] In step D21, a second optimization-ready model corresponding to the cursive writing category is distilled from the second large-scale model.

[0081] Here, the second optimization waiting model can be obtained in the same manner as the first optimization waiting model, and these will not be described again in the embodiments of the present disclosure.

[0082] In step D22, image blocks of handwritten text in the cursive category are used to construct a second training sample set for the model awaiting second optimization.

[0083] Here, the second training sample set of the model awaiting second optimization can be constructed based on the following method.

[0084] In step E1, the image block of handwritten text is divided into units of cells, and second sub-image blocks corresponding to each unit are obtained.

[0085] In step E2, at least one of predetermined operations including masking, overlaying text on text, and altering text in the second sub-image block is performed on the data in each second sub-image block to obtain an extended sample, and the second training sample set includes the extended sample and the second sub-image block.

[0086] Here, masking is a process of masking text, as shown in Figure 4a. Even if the cursive handwriting is unclear, the model can accurately identify it. Furthermore, by removing parts of the text, the model can learn sample knowledge of abbreviations and abbreviations, leading to accurate and complete representations.

[0087] Text overlay on text refers to the overlapping of multiple characters, unclear character boundaries, or the presence of continuous strokes, as shown in Figure 4b.

[0088] The text tampering in the second sub-image block can be performed when there is an error in the text, i.e., the tampering process can be performed, as shown in Figure 4c, which makes it easier for the model to learn the sample characteristics when the task is tampered and to identify the correct text content when it is tampered.

[0089] In the embodiment of the present disclosure, the extended sample is constructed by taking multiple situations into consideration, and the second training sample set obtained based on this method comprehensively takes into account multiple situations of handwritten text, so that the second optimization model can then learn multiple types of writing situations, thereby achieving improved recognition accuracy.

[0090] In step D23, a second, awaiting optimization, model is trained based on the second training sample set.

[0091] The model structure of the second model awaiting optimization may be the same as the structure of the first model awaiting optimization. The training method is also the same as that of the first model awaiting optimization. Finally, the trainable parameters of the second model awaiting optimization are adjusted based on the screening loss and the detection loss. If the training convergence condition is met, the training of the second model awaiting optimization is terminated.

[0092] In step D24, if the classification accuracy of the model awaiting second optimization reaches a bottleneck and does not meet the desired accuracy, samples whose classification error rate is higher than the second error rate threshold are selected from the second training sample set to construct a small sample set.

[0093] In step D25, the second optimization-ready model is trained to learn the knowledge of a small sample, and a text identification model corresponding to the cursive writing category is obtained.

[0094] When the classification accuracy of the second model awaiting optimization reaches a bottleneck and does not meet the expected accuracy, a small sample set is input to a first network in the second model awaiting optimization to obtain feature information of the samples in the small sample set. The feature information is input to a second network to obtain a set of region candidate frames. Each candidate frame in the set of region candidate frames is designated as a first candidate frame, and positive and negative sample pairs are constructed for each first candidate frame. Here, the positive sample pair is constructed by using a sample in the candidate frame whose content is similar to that of the first candidate frame as a positive sample, and a sample in the first candidate frame whose content is dissimilar to that of the first candidate frame as a negative sample, and a negative sample pair is constructed by using a sample in the first candidate frame as a negative sample. Note that each of the candidate frames in the positive and negative sample pairs must have a partial overlap with the first candidate frame. This allows the construction of positive and negative sample pairs whose content has a certain degree of correlation with the first candidate frame. A comparative training loss can further be determined based on the feature distance between the two candidate frames in the positive sample pair and the feature distance between the two candidate frames in the negative sample pair. The model awaiting second optimization is adjusted based on the comparative learning loss to obtain a model awaiting second optimization that satisfies the second convergence condition.

[0095] The second convergence condition may be that the comparative learning loss tends to stabilize, or may be that a preset number of iterations is satisfied, and the embodiments of the present disclosure are not limited thereto.

[0096] It should be noted that, based on the number of text cursive categories, a corresponding number of text identification models should be constructed, and each text cursive category should correspond to one text identification model.

[0097] In an embodiment of the present disclosure, to achieve accurate identification of handwritten text, different text identification models are constructed for different cursive types.

[0098] Based on the obtained text identification model and digit identification model, an embodiment of the present disclosure also provides a data processing method, which can be implemented as follows, as shown in FIG.

[0099] In S501, if it is determined that the content of the newly added process flow card is complete, a classification process is performed on the newly added process flow card to obtain the printed area and the cursive area of ​​the newly added process flow card.

[0100] As shown in Figure 2, the process flow card can be marked with a black polygonal box around it, and a neural network can be used to detect it, and if the content in the black polygonal box is detected, the completeness of the process flow card can be determined. Of course, labeling using reference points is also possible, and the embodiments of the present disclosure are not limited to this.

[0101] A two-dimensional code or barcode may also be used on the process flow card, which can not only identify an area of ​​the complete flow card, but can also be used to determine that the process flow card is a process flow card that belongs to a certain stage of a process flow.

[0102] If a complete newly added process flow card cannot be obtained or if there is ambiguity in the process flow card, the worker can be prompted to re-photograph the process flow card.

[0103] In S502, character identification is performed on the print area using a character extraction technique.

[0104] The character extraction technology may be OCR (Optical Character Recognition) technology, natural scenario text detection and identification FOTS (Fast Oriented Text Spotting), etc., and any technology that can identify characters can be applied to the embodiments of the present disclosure, and the embodiments of the present disclosure are not limited thereto.

[0105] In S503, the cursive category in the cursive region is identified.

[0106] In S504, if the first target cursive category of numerals is included in the cursive region, character recognition is performed based on the numeral recognition model corresponding to the first target cursive category.

[0107] In S505, if the cursive region contains a second target cursive category of text, character identification is performed based on a text identification model corresponding to the second target cursive category, and the identified characters are used to construct a process flow database for the spinning process.

[0108] In the embodiments of the present disclosure, by using a corresponding model based on each cursive type for identification, not only can fully automatic identification of flow data be realized, but also the accuracy of handwriting identification can be improved.

[0109] In some embodiments, based on the characters obtained based on the identification, the characters can be directly stored in the process flow database. However, because the number of identified characters is too large, directly storing them in the process flow database would require multiple reads and writes to the process flow database, which may result in read and write errors. Therefore, in embodiments of the present disclosure, intermediate storage is performed through a non-relational database to reduce the number of reads and writes to the process flow database. The solution can be implemented as follows:

[0110] In step F1, the characters identified from the print area are used as keywords (key), and the characters identified using the number identification model or the text identification model are used as values ​​(value).

[0111] In step F2, the keywords (key) and values ​​(values) are associated with each other and stored in a non-relational database.

[0112] Non-relational databases include MongoDB (a distributed document storage database) and Redis (a remote dictionary service).

[0113] The process flow cards for different stages can be stored in a non-relational database by calling out the corresponding key-value pairs.

[0114] In step F3, if it is determined that the identification of the process flow card is complete, the information in the non-relational database is updated in the process flow database.

[0115] Here, when it is determined that identification of the process flow card is complete, the information in the non-relational database is updated in the process flow database, and then the information in the non-relational database is emptied so that it can be used to read the next process flow card.

[0116] In embodiments of the present disclosure, a non-relational database is used to provide intermediate caching of read data, reducing the number of reads to the process flow database and improving the accuracy of data uploads.

[0117] After the product information in the process flow card is entered into the process flow database, the product information in the process flow card is verified based on the reference information in the process flow database, and it is possible to verify which products have failed, and mark the failed products in color to prompt the worker to process the products.

[0118] During the physical inspection stage of the process flow, i.e., when inspecting the produced spun products, multiple inspections may be performed, and each inspection may be performed by a different person. This may result in a situation where multiple people have handwritten notes on a single process flow card. Therefore, by identifying the two-dimensional code or barcode on the process flow card, it is possible to determine which stage the process flow card belongs to. For example, if the process flow card is in the physical inspection stage, it is divided into handwritten areas assigned to different work stages, and a corresponding identification model is invoked based on the font type of each area. The target numbers and target text are obtained and the corresponding key-values ​​are stored in a non-relational database. When it is determined that the identification of the process flow card is complete, the information in the non-relational database is updated to the process flow database, and the information in the non-relational database is simultaneously emptied.

[0119] In implementation, process flow cards may be marked with different step marks to facilitate identification of parameters for different steps in the same process flow card. For example, a physical inspection step may require different physical inspection items to be performed, and the different items may be completed by the same worker or by different workers. This allows the content of the handwritten text for different items to be accurately identified and applied to automated process flows.

[0120] Based on the same technical idea, an embodiment of the present disclosure also provides a model training apparatus 600, which includes: an acquisition module 601 for acquiring a set of historical process flow cards of the spinning process; an extraction module 602 for extracting handwritten regions from each historical process flow card in the collection of historical process flow cards; a classification module 603 for classifying the handwritten region to obtain image blocks of handwritten digits and image blocks of handwritten text; a digit identification module 604 for constructing a digit identification model of different cursive categories of digits to extract target digits from the newly added process flow card based on the image block of handwritten digits; a text identification module 605 for building a text identification model of different script categories of text to extract target text from the newly added process flow card based on the image block of handwritten text, where the target numbers and target text are used to build a process flow database of the spinning process.

[0121] In some embodiments, the model training device: Dividing the image block of handwritten numbers into first sub-image blocks, each corresponding to a cell in the history process flow card; performing a feature extraction operation on each of the first sub-image blocks to obtain a first feature vector for each of the first sub-image blocks; The image processing device further includes a digit construction module for performing a clustering analysis on the first feature vector of each first sub-image block to obtain different cursive categories of digits.

[0122] In some embodiments, the model training device: Dividing the image block of the handwritten text into second sub-image blocks, each corresponding to a cell in the history process flow card; performing a feature extraction operation on each of the second sub-image blocks to obtain a second feature vector for each of the second sub-image blocks; The image processing device further includes a text construction module for performing a clustering analysis on the second feature vector of each second sub-image block to obtain different script categories of the text.

[0123] In some embodiments, the numeric identification module comprises: a first training unit for training a first large-scale model based on corresponding labeling data of each handwritten digit image block, so that the first large-scale model can identify digit content in each historical process flow card; For each number cursive category, a first acquisition unit distilling a first optimization-ready model corresponding to a cursive writing category from the first large-scale model; A first construction unit constructs a first training sample set of a first optimization-ready model using image blocks of handwritten digits in a cursive style category; a second training unit training a first awaiting optimization model based on the first training sample set; A second construction unit, when the classification accuracy of the first model awaiting optimization reaches a bottleneck and does not satisfy a desired accuracy, selects samples from the first training sample set whose classification error rate is higher than a first error rate threshold, and constructs a small sample set; A first determination unit performs operations including: training a first optimization-ready model to learn small-sample knowledge to obtain a digit identification model corresponding to the cursive writing category.

[0124] In some embodiments, the text identification module: a third training unit for training the second large-scale model so that the second large-scale model can identify text content in each historical process flow card based on the corresponding labeling data of each handwritten text image block; For each cursive text category of text, a second obtaining unit distilling a second optimization-ready model corresponding to the cursive writing category from the second large-scale model; A third construction unit constructs a second training sample set for a second awaiting optimization model using image blocks of handwritten text in the cursive category; a fourth training unit training a second awaiting optimization model based on the second training sample set; a fourth construction unit, when the classification accuracy of the model awaiting second optimization reaches a bottleneck and does not satisfy a desired accuracy, selecting samples from the second training sample set whose classification error rate is higher than a second error rate threshold, and constructing a small sample set; A second determination unit performs operations including training a second optimization-ready model to learn small-sample knowledge to obtain a text identification model corresponding to the cursive writing category.

[0125] In some embodiments, the third building block is Dividing the image block of the handwritten text into units of cells to obtain second sub-image blocks corresponding to the respective units; and performing at least one of predetermined operations, including masking, overlaying text on text, and altering text in the second sub-image block, on the data in each second sub-image block to obtain an extended sample, wherein the second training sample set includes the extended sample and the second sub-image block.

[0126] Based on the same technical idea, an embodiment of the present disclosure also provides a data processing device 700, which includes: a classification module 701 for performing a classification process on the newly added process flow card when it is determined that the content of the newly added process flow card is complete, and obtaining a print area and a cursive area in the newly added process flow card; a first identification module 702 for performing character identification on the print area using character extraction techniques; a second identification module 703 for identifying a cursive category in the cursive region; a third identification module 704 for performing character identification based on a number identification model corresponding to the first target cursive category when the cursive region includes the first target cursive category of number; and a fourth identification module 705 for performing character identification based on a text identification model corresponding to the second target cursive category when the cursive region includes the second target cursive category of text, wherein the identified characters are used to construct a process flow database of the spinning process.

[0127] In some embodiments, the data processing device comprises: using the characters identified from the print area as keywords; using the character identified using the number identification model or the text identification model as the value; Associating keywords with values ​​and storing them in a non-relational database; The system further includes a cache module for updating the information in the non-relational database to the process flow database when it is determined that the identification of the process flow card is complete.

[0128] For specific functions and exemplary descriptions of each module and sub-module of the apparatus in the embodiments of the present disclosure, please refer to the relevant descriptions of the corresponding steps in the above method embodiments, and they will not be repeated here.

[0129] In the technical solution of the present disclosure, the acquisition, storage, and application of users' personal information comply with the provisions of relevant laws and regulations and do not violate public order and morals.

[0130] FIG. 8 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 8, the electronic device includes a memory 810 and a processor 820, and the memory 810 stores a computer program executable by the processor 820. The number of memories 810 and processors 820 may be one or more. The memory 810 may store one or more computer programs, which, when executed by the electronic device, cause the electronic device to perform the method provided by the above method embodiments. The electronic device may further include: a communication interface 830 for communicating with external devices and for data interaction and transmission;

[0131] When the memory 810, the processor 820, and the communication interface 830 are implemented independently, the memory 810, the processor 820, and the communication interface 830 are connected to each other via a bus to enable communication between them. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be classified into an address bus, a data bus, a control bus, and the like. For ease of explanation, only one bold line is shown in FIG. 8, but this does not represent only one bus or only one type of bus.

[0132] Optionally, in a specific implementation, when the memory 810, the processor 820, and the communication interface 830 are integrated on one chip, the memory 810, the processor 820, and the communication interface 830 can communicate with each other via an internal interface.

[0133] It should be understood that the processor may be a Central Processing Unit (CPU), or may be other general-purpose processors, Digital Signal Processing (DSP), Application Specific Integrated Circuits (ASIC), Field Programmable Gate Arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware assemblies, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0134] Additionally, the memory may optionally include read-only memory and random access memory, or may further include non-volatile random access memory. The memory may be either volatile or non-volatile memory, or may include both volatile and non-volatile memory. Here, non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which acts as an external cache. By way of example and not limitation, many forms of RAM are available. For example, static random access memory (Static RAM, SRAM), dynamic random access memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct RAMBUS RAM (DR RAM).

[0135] The above-described embodiments may be implemented, in whole or in part, in software, hardware, firmware, or any combination thereof. When implemented in software, they may be implemented, in whole or in part, in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, a process or function according to an embodiment of the present disclosure is generated, in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center via wire (e.g., coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, Bluetooth, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device including a server, a data center, etc. integrated with one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a Digital Versatile Disc (DVD)), or a semiconductor medium (e.g., a Solid State Disk (SSD)). Note that the computer-readable storage medium referred to in this disclosure may be a non-volatile storage medium, in other words, a non-transitory storage medium.

[0136] Those skilled in the art can understand that all or part of the steps for realizing the above embodiments may be implemented by hardware, or may be implemented by instructing relevant hardware by a program, and the program may be stored in a computer-readable storage medium, and the storage medium may be a read-only memory, a magnetic disk, an optical disk, etc.

[0137] In describing embodiments of the present disclosure, the use of reference terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. The described specific features, structures, materials, or characteristics may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, a person skilled in the art may combine different embodiments or examples and features of different embodiments or examples described in the present disclosure to the extent that they are not mutually inconsistent.

[0138] In the description of the embodiments of the present disclosure, unless otherwise specified, " / " means "or," for example, A / B can mean either A or B. In the present disclosure, "and / or" merely describes the related relationship of related objects and indicates that three types of relationships may exist, for example, A and / or B can indicate the following three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0139] In describing the embodiments of the present disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be interpreted as indicating or implying relative importance, nor should they be interpreted as implying the number of technical features shown. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of such features. In describing the embodiments of the present disclosure, "plurality" means two or more, unless otherwise specified.

[0140] The above are merely illustrative examples of the present disclosure, and do not limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. 1. A method for training a model, comprising: obtaining a set of historical process flow cards for a spinning process; extracting a handwritten region from each historical process flow card in the collection of historical process flow cards; classifying the handwritten region to obtain an image block of handwritten digits and an image block of handwritten text; Based on the image block of handwritten digits, construct a digit identification model of different cursive categories of digits to extract target digits from the newly added process flow card; and building a text identification model of different script categories of text based on the image block of handwritten text to extract target text from the newly added process flow card, wherein the target numbers and the target text are used to build a process flow database of a spinning process. Model training method.

2. Constructing different cursive categories for numbers is Dividing the image block of handwritten numbers into first sub-image blocks, each corresponding to a cell in the history process flow card; performing a feature extraction operation on each of the first sub-image blocks to obtain a first feature vector for each of the first sub-image blocks; performing a clustering analysis on the first feature vector of each first sub-image block to obtain different cursive categories of the digits; The method of claim 1 .

3. Constructing different cursive categories of text is Dividing the image block of the handwritten text into second sub-image blocks, each corresponding to a cell in the history process flow card; performing a feature extraction operation on each of the second sub-image blocks to obtain a second feature vector for each of the second sub-image blocks; performing a clustering analysis on the second feature vector of each second sub-image block to obtain different script categories of the text; The method of claim 1 .

4. building a digit identification model for different cursive categories of digits based on the image blocks of handwritten digits, training a first large scale model to identify numeric content in each historical process flow card based on the corresponding labeling data of each handwritten digit image block; For each number cursive category, distilling a first ready-to-optimize model corresponding to the cursive writing category from the first large-scale model; constructing a first training sample set for the first optimization-ready model using image blocks of handwritten digits in the cursive category; training the first optimization-ready model based on the first training sample set; When the classification accuracy of the first model awaiting optimization reaches a bottleneck and does not satisfy a desired accuracy, selecting samples from the first training sample set whose classification error rate is higher than a first error rate threshold to construct a small sample set; training the first optimization-ready model to learn knowledge of samples in the small sample set to obtain a digit discrimination model corresponding to the cursive category; The method of claim 1 .

5. building text identification models for different cursive categories of text based on the image blocks of handwritten text, Based on the corresponding labeling data of each image block of handwritten text, training a second large scale model so that the second large scale model can identify text content in each historical process flow card; For each cursive text category of text, distilling a second ready-to-optimize model corresponding to the cursive writing category from the second large-scale model; constructing a second training sample set for the second optimization-ready model using image blocks of handwritten text in the cursive category; training the second awaiting optimization model based on the second training sample set; When the classification accuracy of the second model awaiting optimization reaches a bottleneck and does not satisfy a desired accuracy, selecting samples from the second training sample set whose classification error rate is higher than a second error rate threshold to construct a small sample set; training the second, yet-to-be-optimized model to learn knowledge of samples in the small sample set to obtain a text identification model corresponding to the cursive category; The method of claim 1 .

6. constructing a second training sample set for the second awaiting optimization model using image blocks of handwritten text in the cursive category; Dividing the image block of the handwritten text into units of cells to obtain second sub-image blocks corresponding to the units; performing at least one of predetermined operations, including masking, overlaying text on text, and altering text in the second sub-image blocks, on the data in each second sub-image block to obtain enhanced samples, wherein the second training sample set includes the enhanced samples and the second sub-image blocks. The method of training a model according to claim 5.

7. 1. A data processing method comprising: When it is determined that the content of the newly added process flow card is complete, performing a classification process on the newly added process flow card to obtain a print area and a cursive area of ​​the newly added process flow card; performing character identification on the print area using a character extraction technique; identifying a cursive category in the cursive region; When the cursive region includes a first target cursive category of a number, performing character identification based on a number identification model corresponding to the first target cursive category; if the cursive region includes a second target cursive category of text, performing character identification based on a text identification model corresponding to the second target cursive category, and the identified characters are used to construct a process flow database of a spinning process. Data processing methods.

8. The data processing method includes: using the characters identified from the print area as a keyword; using the character identified using the number identification model or the text identification model as the value; Associating keywords with values ​​and storing them in a non-relational database; and updating the information in the non-relational database to a process flow database when it is determined that the identification of the process flow card is complete. The data processing method according to claim 7.

9. A model training device, an acquisition module for acquiring a set of historical process flow cards of the spinning process; an extraction module for extracting handwritten regions from each historical process flow card in the collection of historical process flow cards; a classification module for classifying the handwritten region to obtain an image block of handwritten digits and an image block of handwritten text; a digit identification module for constructing a digit identification model of different cursive categories of digits according to the image block of handwritten digits, so as to extract target digits from the newly added process flow card; a text identification module for constructing a text identification model of different script categories of text to extract target text from the newly added process flow card based on the image block of handwritten text, wherein the target numbers and the target text are used to construct a process flow database of a spinning process; Model training equipment.

10. The model training device comprises: Dividing the image block of handwritten numbers into first sub-image blocks, each corresponding to a cell in the history process flow card; performing a feature extraction operation on each of the first sub-image blocks to obtain a first feature vector for each of the first sub-image blocks; and performing a clustering analysis on the first feature vector of each first sub-image block to obtain different cursive categories of numbers.

10. The model training apparatus of claim 9.

11. The model training device comprises: Dividing the image block of the handwritten text into second sub-image blocks, each corresponding to a cell in the history process flow card; performing a feature extraction operation on each of the second sub-image blocks to obtain a second feature vector for each of the second sub-image blocks; and performing a clustering analysis on the second feature vector of each second sub-image block to obtain different script categories of the text.

10. The model training device of claim 9.

12. The number identification module includes: a first training unit for training a first large-scale model based on corresponding labeling data of each handwritten digit image block, so that the first large-scale model can identify digit content in each historical process flow card; For each number cursive category, a first obtaining unit distilling a first optimization-ready model corresponding to the cursive writing category from the first large-scale model; a first construction unit constructing a first training sample set of the first optimization-ready model using image blocks of handwritten digits of the cursive style category; a second training unit training the first optimization-ready model based on the first training sample set; a second construction unit, when the classification accuracy of the first model awaiting optimization reaches a bottleneck and does not satisfy a desired accuracy, selecting samples from the first training sample set whose classification error rate is higher than a first error rate threshold, and constructing a small sample set; a first determination unit performing an operation including: training the first optimization-ready model to learn knowledge of samples in the small sample set, and obtaining a digit identification model corresponding to the cursive style category; 10. The model training device of claim 9.

13. The text identification module: a third training unit for training the second large scale model so that the second large scale model can identify text content in each historical process flow card based on the corresponding labeling data of each image block of handwritten text; For each cursive text category of text, a second obtaining unit distilling a second optimization-ready model corresponding to the cursive writing category from the second large-scale model; a third construction unit constructing a second training sample set for the second optimization-ready model using image blocks of handwritten text of the cursive style category; a fourth training unit training the second optimization-ready model based on the second training sample set; a fourth construction unit, when the classification accuracy of the second model awaiting optimization reaches a bottleneck and does not satisfy a desired accuracy, selecting samples from the second training sample set whose classification error rate is higher than a second error rate threshold, and constructing a small sample set; a second determination unit performing an operation including: training the second optimization-ready model to learn knowledge of samples in the small sample set, and obtaining a text identification model corresponding to the cursive category; 10. The model training apparatus of claim 9.

14. The third building unit is Dividing the image block of the handwritten text into units of cells to obtain second sub-image blocks corresponding to the units; performing at least one of predetermined operations, including masking, overlaying text on text, and altering text in the second sub-image blocks, on the data in each second sub-image block to obtain enhanced samples, wherein the second training sample set includes the enhanced samples and the second sub-image blocks; 14. The model training apparatus of claim 13.

15. 1. A data processing device, comprising: a classification module for performing a classification process on the newly added process flow card when it is determined that the content of the newly added process flow card is complete, and obtaining a print area and a cursive area in the newly added process flow card; a first identification module for performing character identification on the print area using a character extraction technique; a second identification module for identifying a cursive category in the cursive region; a third identification module for performing character identification based on a number identification model corresponding to the first target cursive category when the cursive region includes a first target cursive category of number; a fourth identification module for performing character identification based on a text identification model corresponding to a second target cursive category of text when the cursive region includes the second target cursive category of text, the identified characters being used to construct a process flow database of a spinning process. Data processing device.

16. The data processing device includes: using the characters identified from the print area as a keyword; using the character identified using the number identification model or the text identification model as the value; Associating keywords with values ​​and storing them in a non-relational database; and updating the information in the non-relational database to the process flow database when it is determined that the identification of the process flow card is complete.

16. A data processing apparatus according to claim 15.

17. at least one processor; a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform the method of any one of claims 1 to 8. Electronic devices.

18. A non-transitory computer readable storage medium for storing instructions that cause a computer to perform the method of any one of claims 1 to 8.

19. A program for implementing the method of any one of claims 1 to 8 when executed by a processor in a computer.

Images