Model training method and apparatus, data processing method and apparatus, electronic device, storage medium, and program

Abstract

To provide a model training method and apparatus, a data processing method and apparatus, an electronic device, a storage medium, and a program.SOLUTION: The model training method includes: acquiring a collection of history process flow cards in a spinning process; extracting a handwritten area from each history process flow card; classification-processing the handwritten area to obtain an image block of handwritten numbers and an image block of a handwritten text; on the basis of the image block of the handwritten numbers, forming a number identification model for different handwriting categories of numbers, to extract a target number from a newly added process flow card; and on the basis of the image block of the handwritten text, forming a text identification model for different handwriting categories of texts, to extract a target text from the newly added process flow card. The target number and target text are used to form a process flow database of the spinning process.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to the field of data processing, and in particular to technical fields such as large scale models, image processing, and computer vision. [Background technology]

[0002] In the industrial scenario of the spinning process, the flow of the spinning process is long and complicated. At present, some important parts need to be recorded manually. The related process data recorded manually also needs to be manually uploaded to the system to digitize and store the flow of the spinning process. However, manual work is resource-intensive and inefficient. Summary of the Invention [Problem to be solved by the invention]

[0003] The present disclosure provides a model training method and apparatus, a data processing method and apparatus, an electronic device, a storage medium, and a program to solve or alleviate one or more technical problems in the prior art. [Means for solving the problem]

[0004] In a first aspect, the present disclosure provides a method for training a model, the method comprising: obtaining a set of historical process flow cards of a spinning process; extracting a handwritten region from each historical process flow card in the collection of historical process flow cards; classifying the handwritten region to obtain an image block of handwritten digits and an image block of handwritten text; Based on the image block of handwritten digits, construct a digit identification model of different cursive categories of digits to extract a target digit from the newly added process flow card; building a text identification model of different script categories of text to extract target text from the newly added process flow card based on the image block of handwritten text, where the target numbers and target text are used to build a process flow database of the spinning process.

[0005] In a second aspect, the present disclosure provides a data processing method, the method comprising: When it is determined that the content of the newly added process flow card is complete, performing a classification process on the newly added process flow card to obtain a print area and a cursive area of ​​the newly added process flow card; performing character identification on the printed area using a character extraction technique; identifying a cursive category in the cursive domain; performing character identification based on a number identification model corresponding to the first target cursive category when the cursive region includes a first target cursive category of a number; and when the cursive region includes a second target cursive category of text, performing character identification based on a text identification model corresponding to the second target cursive category, and the identified characters are used to construct a process flow database of the spinning process.

[0006] In a third aspect, the present disclosure provides a model training apparatus, comprising: an acquisition module for acquiring a set of historical process flow cards of the spinning process; an extraction module for extracting handwritten regions from each historical process flow card in the collection of historical process flow cards; a classification module for classifying the handwritten region to obtain an image block of handwritten digits and an image block of handwritten text; A digit identification module for constructing a digit identification model of different cursive categories of digits to extract a target digit from the newly added process flow card based on the image block of handwritten digits; and a text identification module for building a text identification model of different script categories of text to extract target text from the newly added process flow card based on the image block of handwritten text, where the target numbers and target text are used to build a process flow database of the spinning process.

[0007] In a fourth aspect, the present disclosure provides a data processing apparatus, comprising: a classification module for performing a classification process on the newly added process flow card when it is determined that the content of the newly added process flow card is complete, and obtaining a print area and a cursive area in the newly added process flow card; a first identification module for performing character identification on the printed matter area using a character extraction technique; a second identification module for identifying a cursive category in the cursive region; a third identification module for performing character identification based on a number identification model corresponding to the first target cursive category when the cursive region includes a first target cursive category of a number; and a fourth identification module for performing character identification based on a text identification model corresponding to the second target cursive category when the cursive region includes a second target cursive category of the text, wherein the identified characters are used to construct a process flow database of the spinning process.

[0008] In a fifth aspect, the present disclosure provides an electronic device, the device comprising: At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, cause the implementation of any one of the methods in the embodiments of the present disclosure.

[0009] In a sixth aspect, there is provided a non-transitory computer-readable storage medium having stored thereon computer instructions for causing a computer to perform any one of the methods in the embodiments of the present disclosure.

[0010] In a seventh aspect, a program is provided, which, when executed by a processor, implements any one of the methods in the embodiments of the present disclosure.

[0011] According to the method provided by the embodiments of the present disclosure, the identification accuracy can be improved, and thus accurate process flow data can be obtained.

[0012] It should be understood that the contents described herein are not intended to describe key or important features of the embodiments of the present disclosure, nor are they used to limit the scope of the present disclosure, other features of the present disclosure are facilitated by the following specification.

[0013] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the several accompanying drawings indicate the same or similar components or elements. The accompanying drawings are not necessarily drawn to scale. It should be understood that the drawings depict only some examples provided by the present disclosure and should not be considered as limiting the scope of the present disclosure. [Brief description of the drawings]

[0014] [Figure 1] FIG. 2 is a schematic diagram illustrating a flow of a model training method according to an embodiment of the present disclosure. [Diagram 2] FIG. 1 is a schematic diagram illustrating a process flow card according to one embodiment of the present disclosure. [Figure 3a] FIG. 2 is a schematic diagram illustrating a first sub-image block according to an embodiment of the present disclosure. [Figure 3b] FIG. 2 is a schematic diagram illustrating a second sub-image block according to an embodiment of the present disclosure. [Figure 4a] FIG. 2 is a schematic diagram illustrating a masking process according to an embodiment of the present disclosure. [Figure 4b] FIG. 2 is a schematic diagram illustrating a text overlay on text according to one embodiment of the present disclosure. [Figure 4c] FIG. 13 is a schematic diagram illustrating tampering with text in a second sub-image block according to an embodiment of the present disclosure. [Diagram 5] FIG. 2 is a schematic diagram showing the flow of a data processing method according to an embodiment of the present disclosure. [Figure 6] FIG. 1 is a schematic diagram illustrating a configuration of a model training device according to an embodiment of the present disclosure. [Figure 7] 1 is a schematic diagram illustrating a configuration of a data processing device according to an embodiment of the present disclosure. [Figure 8] FIG. 1 is a block diagram of an electronic device for implementing a model training method and a data processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] The present disclosure will now be described in more detail with reference to the accompanying drawings, in which like reference numbers represent the same or similar elements, and in which various aspects of the embodiments are illustrated, and which, unless otherwise noted, are not necessarily drawn to scale.

[0016] Furthermore, in order to better explain the present disclosure, many specific details are described in the following specific embodiments. Those skilled in the art should understand that the present disclosure can be similarly implemented without some details. In some embodiments, methods, means, components, circuits, etc. that are well known to those skilled in the art are not described in detail so that the gist of the present disclosure is clear.

[0017] Information digitization helps to improve the efficiency of information management, especially in the spinning process, which is long and complicated. Digitalizing each process of the spinning process flow and archiving it in the system helps to track each process of the spinning process, improves the efficiency of the process, and provides effective data support for identifying problems. However, at present, part of the process data of the spinning process needs to be recorded manually and uploaded to the system, which is inefficient and prone to errors.

[0018] Therefore, in order to facilitate accurate and efficient identification of information in the process flow card and building a spinning process database, the embodiment of the present disclosure combines the advantages of the neural network model with the characteristics of the spinning process to propose a scheme for using a neural network model to identify different cursive scripts. The following will first describe the method for training the model in this scheme, and then describe the process of automatically identifying the process flow card using the trained model.

[0019] As shown in FIG. 1 , an embodiment of the present disclosure proposes a model training method, in which the trained model is used to extract content in a process flow card, and the method includes the following steps:

[0020] In S101, a set of historical process flow cards of a spinning process is obtained.

[0021] Here, the process flow data may include equipment data, weighing data of individual wound yarn packages, fineness inspection data, woven socks dyeing judgment data, appearance inspection data, automatic packaging line data, laboratory chemical inspection data, wound yarn package trolley transport data, warehouse management data, supply chain data, etc. Therefore, any process flow may have a process flow card template.

[0022] Here, an example of a process flow card may include a printed area and a handwritten area that a worker needs to fill in, as shown in Figure 2 (Figure 2 shows characters in the printed area).

[0023] Here, the number may be used to determine which process the process flow card belongs to, and other contents may be changed according to the actual process, and the embodiments of the present disclosure are not limited thereto.

[0024] The embodiments of the present disclosure focus on identifying cursive writing, as print is generally clear and easy to identify.

[0025] At S102, a handwritten region is extracted from each historical process flow card in the collection of historical process flow cards.

[0026] In S103, the handwritten region is classified to obtain image blocks of handwritten numbers and image blocks of handwritten text.

[0027] In the spinning process, the workers are complicated, and the cursive writing style varies from worker to worker, so it is not practical to train models for different workers individually. Therefore, in the embodiment of the present disclosure, a classification model suitable for each cursive writing category is obtained by performing training after classifying different cursive writing categories. See S104 and S105 for details.

[0028] In S104, based on the image blocks of handwritten digits, a digit identification model of different cursive categories of digits is constructed to extract target digits from the newly added process flow card.

[0029] In S105, a text identification model of different script categories of text is constructed based on the image block of handwritten text to extract target text from the newly added process flow card, and the target number and the target text are used to construct a process flow database of the spinning process.

[0030] The execution timing of S104 and S105 is not limited.

[0031] In the embodiment of the present disclosure, a handwritten region is extracted from each historical process flow card in the set of historical process flow cards, and the handwritten region is classified to obtain an image block of handwritten digits and an image block of handwritten text. For the cursive category of the image block of handwritten digits, a corresponding number identification model is trained, and for the cursive category of the image block of handwritten text, a corresponding text identification model is trained, so as to obtain the target digits and target text used in constructing a process flow database. Based on this method, the relevant data in the spinning process flow can be automatically uploaded, and human resources can be saved. In addition, for digits and text, different people have different handwriting methods, and the same digits and text have different visual effects. Therefore, by performing classification processing on the cursive, the characteristics of the cursive of the same category can be sorted out, and the corresponding identification model is trained to complete the identification operation, which improves the identification accuracy, and thus the accurate process flow data can be obtained.

[0032] In some embodiments, multiple classification models are created for different handwriting types to improve the accuracy of classification.

[0033] 1) Build a digit identification model for different cursive categories of digits.

[0034] In some examples, constructing different cursive categories for numbers can be performed as follows.

[0035] In step A1, the image block of handwritten numbers is divided into first sub-image blocks, each of which corresponds to a cell in the history process flow card as a division unit.

[0036] As shown in Fig. 3a, there may be handwritten numbers and handwritten text in a cell, and each number and text in the cell may be divided separately and sequentially, and the image block belonging to the handwritten numbers may be determined as the first sub-image block. Of course, there may be a cell that only has handwritten numbers, and the image block in the cell may be determined as the first sub-image block.

[0037] In step A2, a feature extraction operation is performed on each of the first sub-image blocks to obtain a first feature vector for each of the first sub-image blocks.

[0038] A relatively basic convolutional neural network can be used to perform feature extraction operations on each of the first sub-image blocks to obtain a first feature vector for each of the first sub-image blocks. Modified convolutional neural networks can also be used. Modified networks include, for example, a pre-trained backbone network for image feature extraction and a Transformer network that introduces an attention mechanism. Any method that can obtain a first feature vector for a first sub-image block can be applied to the embodiments of the present disclosure.

[0039] In step A3, a clustering analysis is performed on the first feature vector of each first sub-image block to obtain different cursive categories of numbers.

[0040] Because writing styles may differ from person to person, performing clustering analysis on the first feature vector of each first sub-image block is equivalent to clustering fonts that belong to the same writing style and distinguishing fonts that have different writing styles, so as to obtain different cursive categories of numbers.

[0041] In an embodiment of the present disclosure, feature extraction is performed on a first sub-image block in an image block of handwritten digits to obtain a first feature vector of the first sub-image block, and then a clustering operation is performed on the first feature vector to obtain different cursive categories, which can support accurate identification.

[0042] After obtaining the different cursive categories of numbers, building a number identification model for the different cursive categories of numbers based on the image blocks of handwritten numbers can be implemented as follows.

[0043] In step B1, a first large scale model is trained based on the corresponding labeling data of each handwritten digit image block, so that the first large scale model can identify the digit content in each historical process flow card.

[0044] The labeling data is a specific numerical value corresponding to the first sub-image block of the image block of handwritten digits, for example, the first sub-image block 1 in FIG. 3a has the corresponding labeling data 2346-7.

[0045] Here, the first large-scale model may be a large-scale model such as ERNIE3.0 Zeus, a model of the large-scale model version 3.0 of the Wenxin large-scale model. ERNIE3.0 Zeus uses prompt information during training. The large-scale model has the ability to learn based on prompts. In simple terms, prompt learning is when the input information of the first large-scale model includes a task description, sample examples, and target samples, where the task description is a prompt.

[0046] Here, the target sample is an image block of handwritten digits, the task description is "identify digits in the target sample", and the reference example can be "image block 1 of handwritten digits, prediction result 2346-7". In the training process, the first large-scale model can predict the target sample based on the task description and the reference example to obtain a prediction result, compare the prediction result with the labeling result to obtain a first loss value, and adjust the first large-scale model based on the first loss value to obtain a first convergence condition.

[0047] Here, the first convergence condition is that the first loss value is stable or that a preset number of iterations is satisfied, but the embodiment of the present disclosure is not limited thereto.

[0048] In step B2, for each number cursive category, the following operations are performed:

[0049] In step B21, a first model awaiting optimization corresponding to the cursive category is distilled from the first large-scale model.

[0050] Using the knowledge distillation (KD) method, a first awaiting optimization model corresponding to the cursive category can be distilled from the first large-scale model. That is, based on the model structure of the first large-scale model, a small model with fewer parameters, i.e., the first awaiting optimization model, is constructed.

[0051] Of course, it is also possible to obtain a model awaiting first optimization using distillation techniques such as lightweight model architecture, pruning, and quantization.

[0052] In step B22, image blocks of handwritten digits in the cursive category are used to construct a first training sample set of the first optimization-ready model.

[0053] Here, the image blocks of handwritten digits can be split, rotated, masked, occluded, contaminated (e.g., sketched, falsified valid information), and other methods to construct a first training sample set.

[0054] In step B23, a first optimization-ready model is trained based on the first training sample set.

[0055] Here, the first model to be optimized is a Region Convolutional Neural Network (Region-CNN, R-CNN), which includes a first network, a second network, and a third network, where the first network is used to extract feature information of each handwritten digit image block in the first training sample set, the second network is used to determine a target region of each handwritten digit image block, and the third network is used to identify content in the region.

[0056] During implementation, a first training sample set is input to a first network, feature information of each image block of handwritten digits in the first training sample set is obtained, and the feature information is input to a second network to obtain a region candidate frame set and a foreground candidate frame set.

[0057] In addition, the foreground and background of the handwritten digit image block are divided in advance to obtain a set of foreground frame regions. The second network filters the set of region candidate frames and determines the set of region candidate frames with high foreground probability as the foreground candidate frame set.

[0058] A comparison can be made based on the foreground candidate frame set and the foreground frame region set, and a screening loss used to represent the difference between the two can be determined. After determining the foreground candidate frame set based on the method described above, the content in the foreground candidate frame set is identified based on the third network to obtain an identification result, and a detection loss between the identification result and the labeling data is determined.

[0059] Based on the screening loss and the detection loss, adjust the learnable parameters of the first model awaiting optimization, and terminate the training of the first model awaiting optimization if a training convergence condition is satisfied.

[0060] Here, the first convergence condition may be that the comparative learning loss and the detection loss tend to stabilize, or may be that a preset number of iterations is satisfied, and the embodiments of the present disclosure are not limited thereto.

[0061] In step B24, when the classification accuracy of the model awaiting first optimization reaches a bottleneck and does not satisfy the desired accuracy, samples whose classification error rate is higher than the first error rate threshold are selected from the first training sample set to construct a small sample set.

[0062] In step B25, the first optimization-ready model is trained to learn the knowledge of the small sample, and a digit discrimination model corresponding to the cursive writing category is obtained.

[0063] When the classification accuracy of the model waiting for the first optimization reaches a bottleneck and does not satisfy the expected accuracy, the small sample set is input to a first network in the model waiting for the first optimization to obtain feature information of the samples in the small sample set. The feature information is input to a second network to obtain a region candidate frame set. Any candidate frame in the region candidate frame set is set as a first candidate frame, and a positive and negative sample pair is constructed for each first candidate frame. Here, the positive sample pair is used to construct a positive sample pair by using a sample in the candidate frame whose content is similar to that of the first candidate frame as a positive sample, and a sample in the first candidate frame whose content is not similar to that of the first candidate frame as a negative sample, and is used to construct a negative sample pair. Note that each of the candidate frames of the positive and negative sample pairs must have a part of the region overlapping with the first candidate frame. This makes it possible to construct a positive and negative sample pair whose content has a certain relevance to the first candidate frame. A comparative learning loss can be further determined based on the feature distance of the two candidate frames in the positive sample pair and the feature distance of the two candidate frames in the negative sample pair. The first optimization awaiting model is adjusted based on the comparative learning loss to obtain a first optimization awaiting model that satisfies the second convergence condition.

[0064] The second convergence condition may be that the comparative learning loss tends to stabilize, or may be that a preset number of iterations is satisfied, and the embodiments of the present disclosure are not limited thereto.

[0065] In addition, it is necessary to construct a corresponding number of number identification models based on the number of number cursive categories, and correspond to one number identification model for each number cursive category.

[0066] In the embodiment of the present disclosure, to achieve accurate identification of handwritten numbers, different number identification models are constructed for different cursive types.

[0067] 2) Build a text identification model for different script categories of text.

[0068] In some implementations, constructing different cursive categories of text can be performed as follows.

[0069] In step C1, the image block of the handwritten text is divided into second sub-image blocks, each corresponding to a cell in the history process flow card as a division unit.

[0070] As shown in Fig. 3b, there may be handwritten numbers and text in a cell, and each number and text in the cell may be divided separately and sequentially, and the image block belonging to the handwritten text may be determined as the second sub-image block. Of course, there may be a cell that only has handwritten text, and the image block in the cell may be determined as the second sub-image block.

[0071] In step C2, a feature extraction operation is performed on each of the second sub-image blocks to obtain a second feature vector for each of the second sub-image blocks.

[0072] The method of obtaining the second feature vector is the same as the method of obtaining the first feature vector, and will not be repeated in the embodiments of the present disclosure.

[0073] In step C3, a clustering analysis is performed on the second feature vector of each second sub-image block to obtain different script categories of the text.

[0074] In an embodiment of the present disclosure, feature extraction is performed on a second sub-image block in an image block of handwritten text to obtain a second feature vector of the second sub-image block, and a clustering operation is further performed on the second sub-image block to obtain different cursive categories, which can then support accurate identification.

[0075] Obtaining the different cursive categories of the text, building a text identification model for the different cursive categories of the text based on the image blocks of the handwritten text can be implemented as follows.

[0076] In step D1, a second large scale model is trained so that the second large scale model can identify text content in each historical process flow card based on the corresponding labeling data of each image block of handwritten text.

[0077] The labeling data is a specific text corresponding to the second sub-image block of the image block of handwritten text, for example, in the second sub-image block 1 of FIG. 3b, the corresponding labeling data is LDQ.

[0078] Here, the second large-scale model may be the same model as the first large-scale model, and the second large-scale model may be trained using the training method of the first large-scale model, and the embodiments of the present disclosure are not limited thereto.

[0079] In step D2, for each cursive text category of the text, the following operations are performed:

[0080] In step D21, a second awaiting optimization model corresponding to the cursive category is distilled from the second large-scale model.

[0081] Here, the second optimization-awaiting model can be obtained in a similar manner to the first optimization-awaiting model, which will not be described again in the embodiments of the present disclosure.

[0082] In step D22, image blocks of handwritten text in the cursive category are used to construct a second training sample set for the second awaiting optimization model.

[0083] Here, the second training sample set of the model awaiting second optimization can be constructed based on the following method.

[0084] In step E1, an image block of handwritten text is divided into units of cells, and second sub-image blocks corresponding to each unit are obtained.

[0085] In step E2, at least one of predefined operations including masking, overlaying text on text, and altering text in the second sub-image block is performed on the data in each second sub-image block to obtain an extended sample, and the second training sample set includes the extended sample and the second sub-image block.

[0086] Here, masking is a process of masking text, as shown in Figure 4a. Even if the cursive handwriting is unclear, the model can accurately identify it. Furthermore, by removing parts of the text, the model can learn sample knowledge of abbreviations and abbreviations, and derive accurate and complete representations.

[0087] Text overlay on text refers to the overlapping of multiple characters, unclear character boundaries, and the presence of continuous strokes, as shown in Figure 4b.

[0088] The tampering of the text in the second sub-image block can be performed when the text is erroneous, i.e., the tampering process can be performed, as shown in Figure 4c, which makes it easier for the model to learn the sample characteristics when the task is tampered and easier to identify the correct text content when it is tampered.

[0089] In the embodiment of the present disclosure, an extended sample is constructed by taking into account multiple situations, and the second training sample set obtained based on this method comprehensively takes into account multiple situations of handwritten text, so that the second optimization-ready model can learn multiple types of writing situations, and thus improve the recognition accuracy.

[0090] In step D23, a second, pending optimization model is trained based on the second training sample set.

[0091] The model structure of the second optimization pending model may be the same as the structure of the first optimization pending model. The training method is also the same as that of the first optimization pending model. Finally, the learnable parameters of the second optimization pending model are adjusted based on the screening loss and the detection loss, and the training of the second optimization pending model is terminated when the training convergence condition is satisfied.

[0092] In step D24, when the classification accuracy of the model awaiting second optimization reaches a bottleneck and does not meet the desired accuracy, samples whose classification error rate is higher than the second error rate threshold are selected from the second training sample set to construct a small sample set.

[0093] In step D25, a second optimization-ready model is trained to learn the knowledge of small samples to obtain a text identification model corresponding to the cursive category.

[0094] When the classification accuracy of the model waiting for the second optimization reaches a bottleneck and does not satisfy the expected accuracy, the small sample set is input to a first network in the model waiting for the second optimization to obtain feature information of the samples in the small sample set. The feature information is input to a second network to obtain a region candidate frame set. Any candidate frame in the region candidate frame set is set as a first candidate frame, and a positive and negative sample pair is constructed for each first candidate frame. Here, the positive sample pair is used to construct a positive sample pair by using a sample in the candidate frame whose content is similar to that of the first candidate frame as a positive sample, and a sample in the first candidate frame whose content is not similar to that of the first candidate frame as a negative sample, and is used to construct a negative sample pair. Note that each of the candidate frames of the positive and negative sample pairs must have a part of the region overlapping with the first candidate frame. This makes it possible to construct a positive and negative sample pair whose content has a certain relevance to the first candidate frame. A comparative learning loss can be further determined based on the feature distance of the two candidate frames in the positive sample pair and the feature distance of the two candidate frames in the negative sample pair. The model awaiting second optimization is adjusted based on the comparative learning loss to obtain a model awaiting second optimization that satisfies the second convergence condition.

[0095] The second convergence condition may be that the comparative learning loss tends to stabilize, or may be that a preset number of iterations is satisfied, and the embodiments of the present disclosure are not limited thereto.

[0096] It should be noted that, based on the number of text cursive categories, a corresponding number of text identification models should be constructed, and each text cursive category should correspond to one text identification model.

[0097] In the embodiments of the present disclosure, to achieve accurate identification of handwritten text, different text identification models are constructed for different cursive types.

[0098] Based on the obtained text identification model and digit identification model, an embodiment of the present disclosure also provides a data processing method, which can be implemented as follows, as shown in FIG.

[0099] In S501, if it is determined that the content of the newly added process flow card is complete, a classification process is performed on the newly added process flow card to obtain the printed area and the cursive area of ​​the newly added process flow card.

[0100] As shown in Figure 2, the process flow card can be marked with a black line frame around it, and the neural network can be used to detect it, and if the content in the black line frame is detected, the completeness of the process flow card can be determined. Of course, labeling using reference points is also possible, and the embodiment of the present disclosure is not limited thereto.

[0101] A process flow card may also use a two-dimensional code or barcode that can not only identify an area of ​​the complete flow card, but can also be used to determine that the process flow card is a process flow card that belongs to a stage of a process flow.

[0102] If a complete newly added process flow card cannot be obtained or if there is ambiguity in the process flow card, the worker may be prompted to re-photograph the process flow card.

[0103] In S502, character identification is performed on the print area using a character extraction technique.

[0104] The character extraction technology may be OCR technology (Optical Character Recognition), natural scenario text detection and identification FOTS (Fast Oriented Text Spotting), etc., and any technology that can identify characters is applicable to the embodiments of the present disclosure, and is not limited thereto in the embodiments of the present disclosure.

[0105] At S503, a cursive category in the cursive region is identified.

[0106] In S504, when the first target cursive category of numbers is included in the cursive region, character recognition is performed based on the number recognition model corresponding to the first target cursive category.

[0107] In S505, if the cursive region includes a second target cursive category of text, character identification is performed based on a text identification model corresponding to the second target cursive category, where the identified characters are used to construct a process flow database of the spinning process.

[0108] In the embodiments of the present disclosure, by using a corresponding model based on each cursive type for identification, it is possible to not only realize fully automatic identification of flow data, but also improve the accuracy of handwriting identification.

[0109] In some embodiments, based on the characters obtained based on the identification, the characters can be directly stored in the process flow database. However, because the number of identified characters is too large, directly storing them in the process flow database will result in multiple reads and writes to the process flow database, which may cause read and write errors. Therefore, in the embodiments of the present disclosure, intermediate storage is performed through a non-relational database to reduce the number of reads and writes to the process flow database. The solution can be implemented as follows:

[0110] In step F1, the characters identified from the print area are used as keywords (key) and the characters identified using the number identification model or the text identification model are used as values ​​(value).

[0111] In step F2, the keywords (key) and the values ​​(values) are associated with each other and stored in a non-relational database.

[0112] Non-relational databases include MongoDB (a distributed document storage database) and Redis (a remote dictionary service).

[0113] The process flow cards for different stages can be stored in a non-relational database by recalling the corresponding key-value pairs.

[0114] In step F3, if it is determined that the identification of the process flow card is complete, the information in the non-relational database is updated in the process flow database.

[0115] Here, when it is determined that identification of the process flow card is complete, the information in the non-relational database is updated in the process flow database, and then the information in the non-relational database is emptied so that it can be used to read the next process flow card.

[0116] In an embodiment of the present disclosure, a non-relational database is used to provide intermediate caching of read data, reducing the number of reads to the process flow database and improving the accuracy of data uploads.

[0117] After the product information in the process flow card is entered into the process flow database, the product information in the process flow card is verified based on the reference information in the process flow database to verify which products have a rejected status, and the rejected products can be marked in color to prompt the worker that the product needs to be processed.

[0118] In the physical inspection stage of the process flow, that is, to inspect the produced spun products, multiple inspections can be performed, and each inspection may be performed by different people. This may lead to a situation where multiple people have handwritten notes on one process flow card. Therefore, by identifying the two-dimensional code or barcode on the process flow card, it is possible to determine which stage the process flow card belongs to. For example, when the process flow card is in the physical inspection stage, it is divided according to the handwritten areas assigned to different work stages, and the corresponding identification model is called according to the font type of each area, and the target number and the target text are obtained and the corresponding key-value value is stored in the non-relational database. When it is determined that the identification of the process flow card is completed, the information in the non-relational database is updated to the process flow database, and the information in the non-relational database is emptied at the same time.

[0119] In implementation, the process flow card may be set with marks for different steps, which facilitates identification of parameters for different steps in the same process flow card. For example, a physical inspection step may require different physical inspection items to be performed, and the different items may be completed by the same worker or by different workers. This allows the contents of the script text for different items to be accurately identified and applied to the automated process flow.

[0120] Based on a similar technical idea, an embodiment of the present disclosure also provides a model training apparatus 600, which includes: an acquisition module 601 for acquiring a set of historical process flow cards of a spinning process; an extraction module 602 for extracting handwritten regions from each historical process flow card in the collection of historical process flow cards; a classification module 603 for classifying the handwritten regions to obtain image blocks of handwritten numbers and image blocks of handwritten text; A digit identification module 604 for constructing a digit identification model of different cursive categories of digits to extract a target digit from the newly added process flow card based on the image block of handwritten digits; and a text identification module 605 for building a text identification model of different script categories of text to extract target text from the newly added process flow card based on the image block of handwritten text, where the target numbers and target text are used to build a process flow database of the spinning process.

[0121] In some embodiments, the model training device comprises: Dividing the image block of handwritten numbers into first sub-image blocks, each of which corresponds to a cell in the history process flow card as a division unit; performing a feature extraction operation on each of the first sub-image blocks to obtain a first feature vector for each of the first sub-image blocks; The method further comprises: performing a clustering analysis on the first feature vector of each first sub-image block to obtain different cursive categories of numbers; and a number construction module for use in the above.

[0122] In some embodiments, the model training device comprises: Dividing the image block of the handwritten text into second sub-image blocks, each of which corresponds to a cell in the historical process flow card as a division unit; performing a feature extraction operation on each of the second sub-image blocks to obtain a second feature vector for each of the second sub-image blocks; The method further comprises a text construction module for performing a clustering analysis on the second feature vector of each second sub-image block to obtain different script categories of the text.

[0123] In some embodiments, the numeric identification module comprises: A first training unit is provided for training a first large-scale model, so that the first large-scale model can identify digit content in each historical process flow card according to the corresponding labeling data of each handwritten digit image block; For each number cursive category, A first acquisition unit distills a first optimization-ready model corresponding to a cursive writing category from the first large-scale model; A first construction unit constructs a first training sample set of a first optimization-ready model using image blocks of handwritten digits in a cursive style category; A second training unit trains a first pending model based on the first training sample set; and A second construction unit, when the classification accuracy of the first optimization waiting model reaches a bottleneck and does not meet a desired accuracy, selects samples from the first training sample set whose classification error rate is higher than a first error rate threshold, and constructs a small sample set; A first determination unit performs operations including: training a first optimization-ready model to learn small-sample knowledge to obtain a digit discrimination model corresponding to the cursive category.

[0124] In some embodiments, the text identification module: a third training unit for training the second large-scale model according to the corresponding labeling data of each image block of handwritten text, so that the second large-scale model can identify text content in each historical process flow card; For each cursive text category of text, A second obtaining unit distills a second optimization-ready model corresponding to a cursive writing category from the second large-scale model; and A third construction unit constructs a second training sample set of a second awaiting optimization model using image blocks of handwritten text in the cursive category; A fourth training unit trains a second pending model based on the second training sample set; and A fourth construction unit, when the classification accuracy of the model waiting for the second optimization reaches a bottleneck and does not meet a desired accuracy, selects samples from the second training sample set whose classification error rate is higher than a second error rate threshold, and constructs a small sample set; A second determination unit performs operations including training a second optimization-ready model to learn small-sample knowledge to obtain a text identification model corresponding to the cursive category.

[0125] In some embodiments, the third building block is obtaining second sub-image blocks corresponding to each unit obtained by dividing the image block of the handwritten text into units of cells; and performing at least one of predetermined operations, including masking, overlaying text on text, and altering text in the second sub-image block, on the data in each second sub-image block to obtain an extended sample, wherein the second training sample set includes the extended samples and the second sub-image blocks.

[0126] Based on a similar technical idea, an embodiment of the present disclosure also provides a data processing device 700, which includes: a classification module 701 for performing a classification process on the newly added process flow card when it is determined that the content of the newly added process flow card is complete, and obtaining a print area and a cursive area in the newly added process flow card; a first identification module 702 for performing character identification on the print area using a character extraction technique; a second identification module 703 for identifying a cursive category in the cursive region; a third identification module 704 for performing character identification based on a number identification model corresponding to the first target cursive category when the cursive region includes a first target cursive category of a number; and a fourth identification module 705 for performing character identification based on a text identification model corresponding to the second target cursive category when the cursive region includes a second target cursive category of the text, the identified characters being used to build a process flow database of the spinning process.

[0127] In some embodiments, the data processing device comprises: using characters identified from the print area as keywords; using the character identified using the number identification model or the text identification model as the value; Associating keywords with values ​​and storing them in a non-relational database; The method further includes updating the information in the non-relational database to the process flow database when it is determined that the identification for the process flow card is complete, and including a cache module for use therewith.

[0128] The specific functions and exemplary descriptions of each module and sub-module of the apparatus in the embodiments of the present disclosure may refer to the relevant descriptions of the corresponding steps in the embodiments of the above method, and will not be repeated here.

[0129] In the technical solution disclosed herein, the acquisition, storage, and application of users' personal information comply with the provisions of relevant laws and regulations and do not violate public order and morals.

[0130] FIG. 8 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 8, the electronic device includes a memory 810 and a processor 820, and the memory 810 stores a computer program executable by the processor 820. The number of the memory 810 and the processor 820 may be one or more. The memory 810 may store one or more computer programs, which, when executed by the electronic device, cause the electronic device to perform the method provided by the above method embodiment. The electronic device may further include: The communication interface 830 is used for communicating with an external device and performing data interaction and transmission.

[0131] When the memory 810, the processor 820, and the communication interface 830 are implemented independently, the memory 810, the processor 820, and the communication interface 830 are connected to each other via a bus and can communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus can be classified into an address bus, a data bus, a control bus, and the like. For ease of explanation, only one thick line is shown in FIG. 8, but this does not indicate only one bus or only one type of bus.

[0132] Optionally, in a specific implementation, when the memory 810, the processor 820, and the communication interface 830 are integrated on one chip, the memory 810, the processor 820, and the communication interface 830 can communicate with each other via an internal interface.

[0133] It should be understood that the processor may be a Central Processing Unit (CPU) or may be other general purpose processors, Digital Signal Processing (DSP), Application Specific Integrated Circuits (ASIC), Field Programmable Gate Arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware assemblies, etc. The general purpose processor may be a microprocessor or any conventional processor, etc. Additionally, the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0134] Additionally, optionally, the memory may include read-only memory and random access memory, and may further include non-volatile random access memory. The memory may be either volatile or non-volatile memory, or may include both volatile and non-volatile memory. Here, non-volatile memory may include Read-Only Memory (ROM), Programmable ROM (PROM), Erasable PROM (EPROM), Electrically EPROM (EEPROM), or flash memory. Volatile memory may include Random Access Memory (RAM), which acts as an external cache. By way of example and not limitation, many forms of RAM are available. For example, static random access memory (Static RAM, SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (Double Data Rate SDRAM, DDR SDRAM), enhanced SDRAM (Enhanced SDRAM, ESDRAM), synchlink DRAM (Synchlink DRAM, SLDRAM), and direct RAMBUS RAM (DR RAM).

[0135] The above-mentioned embodiments may be implemented in whole or in part in software, hardware, firmware, or any combination thereof. When implemented in software, the whole or in part may be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, a process or function according to the embodiments of the present disclosure is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center via wire (e.g., coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, Bluetooth, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device including a server, a data center, etc. integrated with one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a Digital Versatile Disc (DVD)), or a semiconductor medium (e.g., a Solid State Disk (SSD)). Note that the computer-readable storage medium referred to in this disclosure may be a non-volatile storage medium, in other words, a non-transitory storage medium.

[0136] Those skilled in the art can understand that all or part of the steps for realizing the above embodiments may be implemented by hardware, or may be implemented by instructing relevant hardware by a program, and the program may be stored in a computer-readable storage medium, and the storage medium may be a read-only memory, a magnetic disk, an optical disk, etc.

[0137] In the description of the embodiments of the present disclosure, the reference words "one embodiment", "several embodiments", "examples", "specific examples", or "several examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. And, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. Furthermore, a person skilled in the art may combine different embodiments or examples and features of different embodiments or examples described in the present disclosure to the extent that they are not mutually inconsistent.

[0138] In the description of the embodiments of the present disclosure, unless otherwise specified, " / " means "or", for example, A / B can mean either A or B. "And / or" in the present disclosure is merely for describing the related relationship of related objects, and indicates that three kinds of relationships may exist, for example, A and / or B can indicate the following three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0139] In the description of the embodiments of the present disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying a relative importance, nor should they be construed as implying a number of technical features shown. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present disclosure, "plurality" means two or more, unless otherwise specified.

[0140] The above are merely illustrative examples of the present disclosure, and are not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure should be included within the scope of protection of the present disclosure.

Claims

1. 1. A method for training a model, comprising: obtaining a set of historical process flow cards of a spinning process; extracting a handwritten region from each historical process flow card in the collection of historical process flow cards; classifying the handwritten region to obtain an image block of handwritten numbers and an image block of handwritten text; According to the image block of handwritten digits, construct a digit identification model of different cursive categories of digits to extract a target digit from the newly added process flow card; and building a text identification model of different script categories of text based on the image block of handwritten text, to extract target text from the newly added process flow card, wherein the target numbers and the target text are used to build a process flow database of a spinning process. Model training methods.

2. Constructing different cursive categories for numbers is Dividing the image block of handwritten numbers into first sub-image blocks, each of which corresponds to a cell in the history process flow card as a division unit; performing a feature extraction operation on each of the first sub-image blocks to obtain a first feature vector for each of the first sub-image blocks; performing a clustering analysis on the first feature vector of each first sub-image block to obtain different cursive categories of the digits; The method of training a model according to claim 1 .

3. Constructing different cursive categories of text is Dividing the image block of the handwritten text into second sub-image blocks, each sub-block corresponding to each cell in the historical process flow card; performing a feature extraction operation on each of the second sub-image blocks to obtain a second feature vector for each of the second sub-image blocks; performing a clustering analysis on the second feature vector of each second sub-image block to obtain different script categories of the text; The method of training a model according to claim 1 .

4. building a digit identification model for different cursive categories of digits based on the image blocks of handwritten digits, training a first large scale model so that the first large scale model can identify numeric content in each historical process flow card based on the corresponding labeling data of each handwritten digit image block; For each number cursive category, distilling a first ready-to-be-optimized model corresponding to the cursive category from the first large-scale model; constructing a first training sample set for the first optimization-ready model using image blocks of handwritten digits in the cursive category; training the first optimization-ready model based on the first training sample set; When the classification accuracy of the first model awaiting optimization reaches a bottleneck and does not satisfy a desired accuracy, selecting samples from the first training sample set whose classification error rate is higher than a first error rate threshold, and constructing a small sample set; training the first optimization-ready model to learn knowledge of the small sample to obtain a digit discrimination model corresponding to the cursive category; The method of training a model according to claim 1 .

5. building a text identification model for different cursive categories of text based on the image blocks of handwritten text, training a second large scale model so that the second large scale model can identify text content in each historical process flow card based on the corresponding labeling data of each image block of handwritten text; For each cursive text category of text, distilling a second ready-to-be-optimized model corresponding to the cursive category from the second large-scale model; constructing a second training sample set for the second optimization-ready model using image blocks of handwritten text of the cursive category; training the second optimization-ready model based on the second training sample set; When the classification accuracy of the second model waiting for optimization reaches a bottleneck and does not satisfy a desired accuracy, selecting samples from the second training sample set whose classification error rate is higher than a second error rate threshold, and constructing a small sample set; training the second optimization-ready model to learn knowledge of the small sample to obtain a text identification model corresponding to the cursive category. The method of training a model according to claim 1 .

6. constructing a second training sample set for the second optimization-ready model using image blocks of handwritten text of the cursive category; obtaining second sub-image blocks corresponding to the units obtained by dividing the image block of the handwritten text into units of cells; performing at least one of predetermined operations, including masking, overlaying text on text, and falsifying text in the second sub-image block, on data in each second sub-image block to obtain an augmented sample, wherein the second training sample set includes the augmented sample and the second sub-image block; The method of training a model according to claim 5.

7. 1. A data processing method comprising the steps of: When it is determined that the content of the newly added process flow card is complete, performing a classification process on the newly added process flow card to obtain a print area and a cursive area of ​​the newly added process flow card; performing character identification on the printed matter region using a character extraction technique; identifying a cursive category in the cursive region; performing character identification based on a number identification model corresponding to a first target cursive category when the cursive region includes a number first target cursive category; and if the cursive region includes a second target cursive category of text, performing character identification based on a text identification model corresponding to the second target cursive category, and the identified characters are used to construct a process flow database of a spinning process. Data processing methods.

8. The data processing method includes: using the characters identified from the print area as a keyword; using the character identified using the number identification model or the text identification model as a value; Associating the keywords with the values ​​and storing them in a non-relational database; and updating the information in the non-relational database to a process flow database when it is determined that the identification of the process flow card is complete. The data processing method according to claim 7.

9. A model training device, an acquisition module for acquiring a set of historical process flow cards of the spinning process; an extraction module for extracting handwritten regions from each historical process flow card in the collection of historical process flow cards; a classification module for classifying the handwritten region to obtain an image block of handwritten digits and an image block of handwritten text; A digit identification module for constructing a digit identification model of different cursive categories of digits according to the image block of handwritten digits, so as to extract a target digit from the newly added process flow card; a text identification module for constructing a text identification model of different script categories of text to extract target text from the newly added process flow card based on the image block of handwritten text, wherein the target numbers and the target text are used to construct a process flow database of a spinning process. Model training equipment.

10. The model training device comprises: Dividing the image block of handwritten numbers into first sub-image blocks, each of which corresponds to a cell in the history process flow card as a division unit; performing a feature extraction operation on each of the first sub-image blocks to obtain a first feature vector for each of the first sub-image blocks; performing a clustering analysis on the first feature vector of each first sub-image block to obtain different cursive categories of numbers; and 10. The model training apparatus of claim 9.

11. The model training device comprises: Dividing the image block of the handwritten text into second sub-image blocks, each sub-block corresponding to each cell in the historical process flow card; performing a feature extraction operation on each of the second sub-image blocks to obtain a second feature vector for each of the second sub-image blocks; performing a clustering analysis on the second feature vector of each second sub-image block to obtain different cursive categories of the text; and 10. The model training apparatus of claim 9.

12. The number identification module includes: a first training unit for training the first large-scale model so that the first large-scale model can identify digit content in each historical process flow card according to the corresponding labeling data of each handwritten digit image block; For each number cursive category, A first acquisition unit distills a first optimization-ready model corresponding to the cursive category from the first large-scale model; A first construction unit constructs a first training sample set of the first optimization-ready model using image blocks of handwritten digits of the cursive category; a second training unit training the first optimization-ready model based on the first training sample set; A second construction unit, when the classification accuracy of the first model waiting for optimization reaches a bottleneck and does not satisfy a desired accuracy, selects samples from the first training sample set whose classification error rate is higher than a first error rate threshold, and constructs a small sample set; a first determining unit performing an operation including: training the first optimization-ready model to learn knowledge of the small sample to obtain a digit discrimination model corresponding to the cursive category; 10. The model training apparatus of claim 9.

13. The text identification module includes: a third training unit for training the second large scale model so that the second large scale model can identify text content in each historical process flow card according to the corresponding labeling data of each image block of handwritten text; For each cursive text category of text, A second acquisition unit distills a second optimization-ready model corresponding to the cursive category from the second large-scale model; and A third construction unit constructs a second training sample set of the second optimization-ready model using image blocks of handwritten text of the cursive category; a fourth training unit training the second optimization-ready model based on the second training sample set; A fourth construction unit, when the classification accuracy of the second model waiting for optimization reaches a bottleneck and does not meet a desired accuracy, selects samples from the second training sample set whose classification error rate is higher than a second error rate threshold, and constructs a small sample set; a second determining unit performing an operation including: training the second optimization-ready model to learn knowledge of the small sample to obtain a text identification model corresponding to the cursive category.

10. The model training apparatus of claim 9.

14. The third building unit is obtaining second sub-image blocks corresponding to the units obtained by dividing the image block of the handwritten text into units of cells; performing at least one of predetermined operations, including masking, overlaying text on text, and altering text in the second sub-image block, on the data in each second sub-image block to obtain an augmented sample, wherein the second training sample set includes the augmented sample and the second sub-image block; 14. A model training apparatus according to claim 13.

15. 1. A data processing device, comprising: a classification module for performing a classification process on the newly added process flow card when it is determined that the content of the newly added process flow card is complete, and obtaining a print area and a cursive area in the newly added process flow card; a first identification module for performing character identification on the printed matter region using a character extraction technique; a second identification module for identifying a cursive category in the cursive region; a third identification module for performing character identification based on a number identification model corresponding to the first target cursive category when the cursive region includes a first target cursive category of a number; a fourth identification module for performing character identification based on a text identification model corresponding to a second target cursive category of text when the cursive region includes the second target cursive category of text, the identified characters being used to build a process flow database of a spinning process. Data processing device.

16. The data processing device includes: using the characters identified from the print area as a keyword; using the character identified using the number identification model or the text identification model as a value; Associating the keywords with the values ​​and storing them in a non-relational database; and updating the information in the non-relational database to a process flow database when it is determined that the identification of the process flow card is complete.

16. A data processing apparatus according to claim 15.

17. At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform a method according to any one of claims 1 to 8. Electronic devices.

18. A non-transitory computer readable storage medium for storing instructions that cause a computer to perform the method of any one of claims 1 to 8.

19. A program for implementing the method according to any one of claims 1 to 8 when executed by a processor in a computer.

Images