Text data processing method and device, storage medium and electronic equipment
By obtaining and processing the style feature vectors of handwritten text, new style feature vectors are derived, which solves the problem of limited number of handwritten text styles in the existing technology, and achieves a diversified data training set and higher recognition accuracy.
Patent Information
- Application Number
- CN202310547592.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-16
- Publication Date
- 2024-11-22
AI Technical Summary
The existing handwritten text recognition model relies on manual collected handwritten text data, resulting in limited number of handwritten text styles, resulting in low recognition accuracy.
By obtaining the sample handwritten text, determining its style feature vector, and deriveing new style feature vectors according to preset operation rules, thereby increasing the number of handwritten text styles and forming a diverse data training set.
Save manpower and material resources, improve the generation efficiency of handwritten text data training sets, and form a more diverse handwritten text style, which improves the recognition accuracy of handwritten text recognition models.
Smart Images

Figure CN119007225A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a text data processing method, device, storage medium and electronic device. Background Art
[0002] Existing handwritten text recognition models often have a technical defect of low recognition accuracy in handwritten text recognition. The main reason for this technical defect is that the handwritten text model with high recognition accuracy is obtained by training a large number of handwritten text data training sets with different handwriting styles. However, the data training sets currently used often rely on manual extraction, such as manually collecting handwritten texts with different handwriting styles from different devices. It is conceivable that manual data collection not only consumes a lot of manpower and material resources, but also the number of styles collected is limited, resulting in the handwritten text styles in the data training set are not diverse enough, causing the trained model to have the problem of low recognition accuracy. Based on this, how to provide a method to increase the number of handwritten text styles is a technical problem that needs to be solved urgently. Summary of the invention
[0003] The embodiments of the present application provide a method, device, storage medium and electronic device for processing text data. The technical solution provided by the present application can increase the number of styles of handwritten text styles, thereby solving the technical problem of limited styles of handwritten text collected manually.
[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.
[0005] According to a first aspect of an embodiment of the present application, a method for processing text data is provided, the method comprising: obtaining a first sample handwritten text; determining a style feature vector based on the first sample handwritten text as a first style feature vector, the style feature vector being used to characterize the style of the handwritten text; operating the first style feature vector according to a preset operation rule to derive at least one second style feature vector; and determining a second sample handwritten text through the second style feature vector to increase the number of styles of the handwritten text style.
[0006] According to a second aspect of an embodiment of the present application, a device for processing text data is provided, the device comprising: an acquisition unit, for acquiring a first sample handwritten text; a first determination unit, for determining a style feature vector based on the first sample handwritten text as a first style feature vector, the style feature vector being used to characterize the style of the handwritten text; a derivation unit, for operating the first style feature vector according to a preset operation rule to derive at least one second style feature vector; and a second determination unit, for determining a second sample handwritten text through the second style feature vector to increase the number of styles of the handwritten text style.
[0007] According to a third aspect of an embodiment of the present application, a computer-readable storage medium is provided, characterized in that at least one program code is stored in the computer-readable storage medium, and the at least one program code is loaded and executed by a processor to implement the operations performed by the method described in any one of the first aspects above.
[0008] According to a fourth aspect of an embodiment of the present application, an electronic device is provided, comprising one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the operations performed by the method described in any one of the first aspects above.
[0009] The technical solution of the present application first obtains a first sample handwritten text, and then determines a style feature vector for characterizing the style of the handwritten text based on the first sample handwritten text as a first style feature vector; then operates the first style feature vector according to a preset operation rule to derive at least one second style feature vector; finally, determines the second sample handwritten text through the second style feature vector, thereby increasing the number of styles of the handwritten text style. It can be seen that through the technical solution of the present application, since the number of styles of the handwritten text style is increased based on the style feature vector of the first sample handwritten text, other styles different from the style in the first sample handwritten text can be derived. In the entire technical solution, there is no need to rely on manual collection of a large amount of data, which can save manpower and material resources and improve the generation efficiency of the handwritten text data training set. In addition, compared with the number of handwritten text styles collected manually, the number of handwritten text styles that can be obtained through the technical solution of the present application is not limited, so the handwritten text styles formed are more diverse.
[0010] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0012] Figure 1 A schematic diagram showing a flow chart of a method for processing text data according to an embodiment of the present application;
[0013] Figure 2 A detailed schematic diagram showing interpolation operation is performed on at least two first style feature vectors according to a preset operation rule to derive at least one second style feature vector according to an embodiment of the present application;
[0014] Figure 3 A schematic diagram showing the results of performing multiple interpolation operations according to an embodiment of the present application is shown;
[0015] Figure 4 A schematic diagram showing a first sample handwritten text and a second sample handwritten text according to an embodiment of the present application is shown;
[0016] Figure 5 A schematic diagram of a process of obtaining each sample handwritten text and an imitation handwritten text corresponding to each sample handwritten text according to an embodiment of the present application is shown;
[0017] Figure 6 A schematic diagram of the structure of a first neural network model according to an embodiment of the present application is shown;
[0018] Figure 7 A schematic diagram showing the result of constructing a plurality of handwritten text unit sets according to an embodiment of the present application is shown;
[0019] Figure 8 An example schematic diagram of obtaining a target handwritten text corresponding to the printed text to be converted according to an embodiment of the present application is shown;
[0020] Fig. 9 A block diagram of a text data processing device according to an embodiment of the present application is shown;
[0021] Fig.10 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION
[0022] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.
[0023] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.
[0024] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0025] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.
[0026] It should be noted that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.
[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the objects used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those shown or described.
[0028] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0029] Some embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments may be combined with each other.
[0030] See also Figure 1 , shows a flow chart of a method for processing text data according to an embodiment of the present application, specifically including S110 to S140.
[0031] S110: Obtain a first sample handwritten text.
[0032] The number of styles of the first sample handwritten text is not limited here, and can be one or more. The text content included in the first sample handwritten text is also not limited here, including but not limited to single Chinese characters, Chinese character words, Chinese character sentences, punctuation marks, numbers, English words, English letters, English sentences, etc.
[0033] In some implementations, the first sample handwritten texts may be collected from different users. For example, user A, user B, and user C are asked to write "water and food?" and "The weather is great today", thereby forming first sample handwritten texts with three styles, and the formed first sample handwritten texts include the text contents "water and food?" and "The weather is great today".
[0034] In some implementations, the first sample handwritten text may be obtained from public data, such as by manually collecting the first sample handwritten text from different devices.
[0035] Continue to see Figure 1 , S120, determining a style feature vector based on the first sample handwritten text as a first style feature vector, wherein the style feature vector is used to characterize the style of the handwritten text.
[0036] In some implementations, the first style feature vector is a latent vector, or may be other feature vectors, which is not specifically limited in this application.
[0037] In some implementations, a vector with a set dimension may be randomly generated as the style feature vector of the first sample handwritten text.
[0038] In some implementations, the style feature vector of the first sample handwritten text may be determined by relying on a vector generation model.
[0039] It should be noted that if the first sample style feature vector has multiple styles, multiple corresponding first style feature vectors can be determined, and the dimensions of the multiple first style feature vectors are the same, that is, each first style feature vector has the same number of elements.
[0040] Continue to see Figure 1 , S130, operating the first style feature vector according to a preset operation rule to derive at least one second style feature vector.
[0041] In this embodiment, there are at least the following four implementation methods.
[0042] The first implementation method:
[0043] The first style feature vector includes a first style feature vector corresponding to a sample handwritten text of at least one style, that is, the number of the first style feature vector is at least one, and then a first style feature vector is arbitrarily determined from the at least one first style feature vector; then a set derivative operation coefficient is obtained, and based on the element value of each vector element in the first style feature vector and the obtained derivative operation coefficient, a second style feature vector is calculated according to a set operation method. Then, more second style feature vectors are obtained from the obtained second style feature vector in the above manner.
[0044] Exemplarily, assuming that the first style feature vector is: [X 1,0 ,X 1,1 ,X 1,2 ...X 1,126 ,X 1,127 ], set the derivative operation coefficient to 0.8, and set the operation method to multiply the set derivative operation coefficient with each vector element of the first style feature vector to obtain the second style feature vector. Therefore, the second style feature vector is: [0.8X 1,0 ,0.8X 1,1 ,0.8X 1,2 ...0.8X 1,126 ,0.8X 1,127 ]. At the same time, the obtained second style feature vector is used as the new first style feature vector, so that more second style feature vectors can be obtained, such as: [0.64X 1,0 ,0.64X 1,1 ,0.64X 1, 2...0.64X 1,126 ,0.64X 1,127 ].
[0045] It should be noted that, in this embodiment, the setting of the derivative operation coefficient is not limited, and the setting operation method can be the element value of each vector element in the first style feature vector and the setting derivative operation coefficient. Specifically, this application does not limit the setting operation method.
[0046] The second implementation method of S130:
[0047] The first sample handwritten text includes sample handwritten texts of at least two styles, and the first style feature vector includes first style feature vectors corresponding to the sample handwritten texts of the at least two styles, that is, the number of the first style feature vectors is at least two.
[0048] At least two first style feature vectors are operated according to a preset operation rule to derive at least one second style feature vector.
[0049] In one implementation, at least two first style feature vectors may be interpolated according to a preset operation rule to derive at least one second style feature vector. Specifically, the interpolation operation may be performed according to the following steps: Figure 2 The steps shown are executed, including S131 to S133.
[0050] S131 : arbitrarily determine two first style feature vectors from the at least two first style feature vectors.
[0051] It can be understood that one first style feature vector represents one style, and then any two first style feature vectors determined represent two different styles.
[0052] S132: Determine an element value interval corresponding to each vector element position based on the element value of each vector element position in the two first style feature vectors.
[0053] It should be noted that the number of elements in each first style feature vector is the same, and the element value interval is composed of the values of two vector elements corresponding to each vector element position. Therefore, a first style feature vector has as many element values as it has corresponding element value intervals.
[0054] Exemplarily, the two first style feature vectors obtained are X1 and X2, X1 = [X 1,0 ,X 1,1 ,X 1, 2...X 1,126 ,X 1,127 ]; X2 = [X 2,0 ,X 2,1 ,X 2,2 ...X 2,126 ,X 2,127 ].
[0055] It can be seen that the dimensions of X1 and X2 are both 128. At the first vector element position, the element value of X1 is [X 1,0 ], the element value of X2 is [X 2,0 ], therefore, the element value interval corresponding to the first vector element position is [X 1,0 , X 2,0 According to this logic, we can determine that the element value interval corresponding to the second vector element position is [X 1,1 , X 2,1 ], the element value interval corresponding to the third vector element position is [X 1,3 , X 2,3 ]...
[0056] Continue to see Figure 2 , S133, interpolating in each element value interval respectively to obtain at least one second style feature vector, wherein the number of elements of the second style feature vector is the same as the number of elements of the first style feature vector.
[0057] It should be noted that the principle of interpolation is to insert a new element value in the element value interval, and the new element value is within the element value interval, so as to avoid the collapse of the expansion style pattern, so that the second sample handwritten text determined by the formed second style feature vector can be more realistic.
[0058] It should also be noted that the present application does not limit the specific implementation methods of interpolating in each element value interval, for example:
[0059] In some implementations, a number may be randomly selected from the element value interval as a new element value.
[0060] For example, if an element value interval is [10,200], then a value between 10-200 can be randomly selected as the new element value, such as selecting 100 to interpolate in the element value interval.
[0061] In some implementations, the average of the two element values corresponding to the element value interval may be used as the new element value. For example, if an element value interval is [10, 200], then 105 may be selected between 10-200 to perform interpolation in the element value interval.
[0062] In some implementations, a geometric interpolation method may be used to insert a new element value into an element value interval.
[0063] It should also be noted that, because a new element value is inserted into each vector element position, the new element value corresponding to each vector element position constitutes a second style feature vector.
[0064] Exemplarily, the two first style feature vectors are X1 and X2. 1,0 ,X 1,1 ,X 1,2 ...X 1,126 ,X 1,127 ]; X2 = [X 2,0 ,X 2,1 ,X 2,2 ...X 2,126 ,X 2,127 ], after interpolation operation, the second style feature vector X3 = [X 3,0 ,X 3,1 ,X 3,2 ...X 3,126 ,X 3,127 ], where the value range of each element in X3 is [min(X 1,i ,X 2,i ), max(X 1,i ,X 2,i )].
[0065] The third implementation method of S130:
[0066] On the basis of the second implementation, multiple interpolation operations are performed on the first style feature vector or the second style feature vector according to the preset operation rule to derive a style feature vector for characterizing the style of the handwritten text.
[0067] It can be understood that in the third embodiment, after performing an interpolation operation, two more first style feature vectors can be arbitrarily selected from at least two first style feature vectors, or two second style feature vectors can be arbitrarily selected from the obtained second style feature vectors, so that by continuing to perform interpolation operations based on the two selected style feature vectors, more second style feature vectors can be obtained.
[0068] Exemplarily, X1 and X2 are selected from the first style feature vectors X1, X2, X3, and X4 to perform a first interpolation operation to obtain a second style feature vector X5, and X1 and X3 are selected to perform a first interpolation operation to obtain a second style feature vector X6. Afterwards, X2 and X3 can be selected from the first style feature vectors X1, X2, X3, and X4 to continue the interpolation operation, or the second style feature vectors X5 and X6 can be selected to continue the interpolation operation.
[0069] It can be understood that in the third embodiment, an infinite number of interpolations can be implemented, thereby expanding the number of styles.
[0070] The fourth implementation method of S130:
[0071] The first style feature vector and the second style feature vector are interpolated multiple times according to the preset operation rule to derive a style feature vector for representing the style of the handwritten text.
[0072] It can be understood that in the fourth embodiment, after obtaining the second style feature vector, a style feature vector can be randomly selected from the obtained second style feature vectors, and a style feature vector can be randomly selected from the first style feature vector, so that the second style feature vector can be obtained by performing interpolation operation again based on the two randomly selected style feature vectors.
[0073] Exemplarily, the two first style feature vectors are X1 and X2. 1,0 ,X 1,1 ,X 1,2 ...X 1,126 ,X 1,127 ]; X2 = [X 2,0 ,X 2,1 ,X 2,2 ...X 2,126 ,X 2,127 ], after interpolation operation, the second style feature vector X3 = [X 3,0 ,X 3,1 ,X 3,2 ...X 3,126 ,X 3,127 ]; then select X1 and X3 for interpolation operation, so as to obtain the second style feature vector X4.
[0074] Based on the above examples, it can be understood that in the fourth implementation, an infinite number of interpolations can be achieved, thereby expanding the number of styles.
[0075] In order to enable those skilled in the art to better understand the third and fourth embodiments, Figure 3 Provide explanation.
[0076] See also Figure 3 , shows a schematic diagram of the results of multiple interpolation operations according to an embodiment of the present application.
[0077] Figure 3 Each origin in represents a handwritten text style, where Figure 3 (a) represents the style of the first sample handwritten text, which contains 40 styles in total. After the first interpolation operation, 40 new styles are obtained, that is, 40 second style feature vectors are obtained. Figure 3 (b) shows the style of the first sample handwritten text and the style obtained after the first interpolation operation, totaling 80 styles. Figure 3 (b) is used as the basis for the second interpolation operation, and 80 new styles are obtained, such as Figure 3 (c) includes Figure 3 The styles in (b) and the 80 new styles formed, thus showing a total of 160 styles.
[0078] from Figure 3 It can be seen that, through the third implementation and / or the fourth implementation, multiple interpolation operations are performed, so that the style of the handwritten text increases exponentially, thereby expanding the diversity of styles.
[0079] In this embodiment, since countless values can be selected within the element value interval as new element values for interpolation, and the selection range for setting the derivative operation coefficients is wide and the operation methods are set in a variety of ways, the technical solution of this embodiment can expand countless styles and meet the requirements of a data training set with diverse styles required for training a handwritten text recognition model.
[0080] Continue to see Figure 1 , S140, determining a second sample handwritten text through the second style feature vector to increase the number of styles of the handwritten text style.
[0081] In some embodiments, the obtained second style feature vector can be passed through a preset model to obtain a handwritten text corresponding to the second style feature vector as a second sample handwritten text. The text content in the second sample handwritten text can be the same as the text content in the first sample text, or can be partially the same, or can be completely different. Specifically, this application does not limit this.
[0082] In order to make those skilled in the art better understand this embodiment, Figure 4 Give an example.
[0083] Assume that the first sample handwritten text is a “Vector” written by four different users, namely Figure 4 The four "Vectors" shown in the first column obviously correspond to four styles. By performing multiple interpolation operations in the third implementation and / or the fourth implementation in S130 of the present application, 16 second style feature vectors are obtained, and then the handwritten texts of "Vector" corresponding to the 16 second style feature vectors are determined, and the display result is Figure 4 The "Vector" shown in the second to fifth columns.
[0084] from Figure 4 It can be clearly seen that the second sample handwritten text effect is produced after multiple interpolation operations.
[0085] In some embodiments of the present application, the text data processing method further includes: Figure 5 S100 to S300 shown.
[0086] S100, obtaining sample handwritten texts, and obtaining printed texts having text contents different from those in the sample handwritten texts, wherein the sample handwritten texts include the first sample handwritten texts and / or the second sample handwritten texts.
[0087] It can be understood that in this embodiment, the sample handwritten text may include a first sample handwritten text, the sample handwritten text may also include a second sample handwritten text, and the sample handwritten text may also include a first sample handwritten text and a second sample handwritten text. Of course, the sample handwritten text may also include handwritten text collected by other means, such as handwritten text in a public data set, or handwritten text collected manually through different devices. Specifically, in this application, there is no limitation on which styles and which handwritten texts are included in the sample handwritten text.
[0088] It should be noted that the text content of the printed text in this embodiment depends on the text content of the acquired sample handwritten text.
[0089] For example, suppose the sample handwritten text includes "water" and "food" in three styles; that is, the sample handwritten text includes text content including "water" and "food". Then the printed text can select all other text content except "water" and "food", including but not limited to single Chinese characters, Chinese words, Chinese sentences, punctuation marks, numbers, English words, English letters, English sentences, etc.
[0090] It will be appreciated that the printed text selected in this application is intended to be exhaustive of the existing text content.
[0091] Continue to see Figure 5 , S200, for each sample handwritten text, train a first neural network model through each sample handwritten text, so that the first neural network model learns the style of each sample handwritten text.
[0092] In some embodiments, the first neurological disease network model may be a GAN network model.
[0093] In some embodiments, the configuration structure of the first neural network model can refer to Figure 6 The specific structure shown is designed. Of course, other structures can also be designed to learn the style of each sample handwritten text, and this application does not limit this.
[0094] It should be noted that, because each of the obtained sample handwritten texts corresponds to a style, the first neural network model can learn to imitate the style of each sample handwritten text after being trained using the sample handwritten texts.
[0095] For example, if the sample handwritten text corresponds to three handwritten text styles, then the first neural network model will learn to imitate these three handwritten text styles after training.
[0096] Continue to see Figure 5 , S300, converting the printed text into an imitation handwritten text consistent with the style of each sample handwritten text through the first neural network model.
[0097] For example, if the printed text includes "floor" and the sample handwritten text includes three handwriting styles, then the trained first neural network model can convert the "floor" in the printed text into "floor" in the three handwriting styles.
[0098] In this embodiment, the specific implementation is related to the structure of the designed first neural network model, and the specific implementation can be determined based on actual conditions, and this application does not limit it here.
[0099] For example, if the first neural network model is as follows Figure 6 The structure shown in the figure is designed, then, the specific implementation method of converting the printed text into the imitation handwritten text consistent with the style of each sample handwritten text through the first neural network model can be performed according to the following S310 to S330, wherein the specific parameters and the number of layers in the model structure can be adjusted according to actual conditions, and this application does not limit it here.
[0100] See also Figure 6 ,The structure of the first neural network model includes an encoder and a decoder. The black part is the encoder and the white part is the decoder.
[0101] S310, inputting the printed text into the encoder of the first neural network model so that the encoder encodes the printed text to obtain a text feature vector of the printed text.
[0102] See also Figure 6 , wherein a printed text with a dimension of 256*256*1 is input into the input layer, and then encoded by the encoder, and the input printed text encoding is represented by a low-dimensional vector to obtain a 512-dimensional text feature vector.
[0103] S320: Concatenate the text feature vector of the printed text with the style feature vector used to characterize each sample handwritten text to obtain an intermediate feature vector.
[0104] It can be understood that each sample handwritten text corresponds to a style, and each style has a corresponding style feature vector. One style feature vector is used in one concatenation.
[0105] like Figure 6 As shown, the style feature vector is 128-dimensional. After horizontal connection, that is, after splicing, the intermediate feature vector obtained is 1*1*(512+128) dimensional.
[0106] S330, the decoder decodes the intermediate feature vector to obtain an imitation handwritten text that is consistent with the style of each sample handwritten text.
[0107] like Figure 6 As shown, after the concatenated intermediate feature vector is decoded by the decoder, the imitation handwriting text in the style corresponding to the style feature vector used for concatenation and consistent with the text content of the input printed text can be obtained.
[0108] For example, if the printed text input into the input layer is "floor", the style feature vector of style 1 is concatenated with the text feature vector of "floor" obtained by the encoder to obtain an intermediate feature vector, and finally, after decoding through the decoder, the "floor" of style 1 can be output as an imitation of handwritten text.
[0109] In this embodiment, after the conversion of the first neural network model, a large amount of handwritten texts in various handwritten text styles can be obtained. Thus, a diverse and comprehensive handwritten text data training set can be obtained, and the handwritten text recognition model can be trained based on the data training set to improve the accuracy of the handwritten text recognition model in recognizing handwritten texts.
[0110] In some implementations of the present application, the text data processing method further includes the following S10 to S20.
[0111] S10, training a second neural network model through each sample handwritten text and the imitation handwritten text corresponding to each sample handwritten text, so that the second neural network model learns the text style and text content of each handwritten text.
[0112] This is equivalent to obtaining a handwritten text data training set after the above-mentioned technical solution of the present application, and the data training set includes each sample handwritten text and the imitation handwritten text corresponding to each sample handwritten text, that is, includes diverse and rich handwritten texts.
[0113] In some embodiments, the Figure 6 The structure shown is used to construct the second neural network model. Of course, other structures can also be designed according to actual conditions, and this application does not limit this.
[0114] S20, recognizing the handwritten text to be recognized by using the second neural network model to convert the handwritten text to be recognized into a target printed text.
[0115] This is equivalent to the second neural network model, after being trained and learned using the obtained handwritten text data training set, becoming a handwritten text recognition model with handwritten text recognition function, so that based on the second neural network model, various handwritten texts to be recognized can be recognized and converted into corresponding target printed texts.
[0116] It can be understood that this embodiment is a practical application of the above-mentioned technical solution of the present application. After the above-mentioned technical solution is used, a diverse and rich handwritten text data training set can be formed, so that the second neural network model trained based on the data training set can accurately recognize various handwritten texts to be recognized.
[0117] In some implementations of the present application, the text data processing method further includes the following S10A to S20A.
[0118] S10A: Determine handwritten text units with the same text content from each sample handwritten text and the imitation handwritten text corresponding to each sample handwritten text.
[0119] It can be understood that after the implementation of the above technical solution of the present application, each sample handwritten text formed corresponds to a style and contains a large number of handwritten texts in the style. Because each style contains a large number of handwritten texts, there will be units with the same text content but different styles, that is, handwritten text units with the same text content.
[0120] It should be noted that the content in the handwritten text unit can be a single Chinese character, Chinese word, Chinese sentence, punctuation, number, English word, English letter, English sentence, etc., and this application does not limit it specifically.
[0121] S20A: constructing a plurality of handwritten text unit sets based on the handwritten text units with the same text content, wherein each handwritten text unit set includes handwritten texts with the same text content but different styles.
[0122] For example, each sample handwritten text and the imitation handwritten text corresponding to each sample handwritten text contain 100 styles, and the handwritten text units with the same text content in these 100 styles are determined to be ",","?","ancient,""and,"believe,""Britons,""food,""I,""The,""to,""used,""water". Finally, after construction, 12 handwritten text unit sets are obtained, each of which has 100 styles.
[0123] Specifically, the constructed handwritten text unit set can be embodied in the form of a folder, such as Figure 7 shown.
[0124] It can be understood that this embodiment is a practical application of the above-mentioned technical solution of this application. After the above-mentioned technical solution is used, diverse and rich handwritten texts can be formed. This embodiment organizes the large amount of data obtained to form multiple handwritten text unit sets, which can be beneficial to the subsequent application of the data.
[0125] In some implementations of the present application, the text data processing method further includes the following S10B to S30B.
[0126] S10B, obtaining a printed text to be converted, and splitting a printed text in the printed text to be converted into at least one printed text unit.
[0127] Exemplarily, the obtained printed text to be converted is “water and food? The ancient Britons, I believe, used to”.
[0128] The text is split into multiple printed text units by space, thereby obtaining 13 printed text units, namely, "water", "and", "food", "?", "The", "ancient", "Britons", "," "I", "believe", "," "used", and "to".
[0129] S20B: for each printed text unit, extract a handwritten text unit that meets the target style from the corresponding handwritten text unit set according to the text content of each printed text unit as a target handwritten text unit.
[0130] Exemplarily, the printed text to be converted obtained in the above example can be obtained from Figure 7 Extract handwritten text units that match the target style from the various folders shown.
[0131] S30B, concatenating the target handwritten text units to obtain a target handwritten text corresponding to the printed text to be converted.
[0132] Exemplarily, in the process of splitting the acquired printed text to be converted into at least one printed text unit, a position mark may be made for each printed text unit.
[0133] Specific examples include: “0”: “water”, “1”: “and”, “2”: “food”, “3”: “?”, “4”: “The”, “5”: “ancient”, “6”: “Britons”, “7”: “,”, “8”: “I”, “9”: “believe”, “10”: “,”, “11”: “used”, “12”: “to”.
[0134] After extracting handwritten text units that match the target style from the handwritten text unit set, 13 target handwritten text units of the target style are obtained. Then, normalized splicing is performed according to the position marks recorded during the splitting process, so as to obtain the target handwritten text corresponding to the printed text to be converted.
[0135] Exemplarily, after processing in this embodiment, the following is obtained: Figure 8 The display effect is shown. In the process of normalization and splicing, the spacing between each target handwritten text can also be set and adjusted.
[0136] It can be understood that this embodiment is a practical application of the above technical solution of this application, and can convert printed text into handwritten text in a target style through extraction and other methods.
[0137] In the technical solutions provided in some embodiments of the present application, the technical solution of the present application first obtains a first sample handwritten text, then determines a style feature vector for characterizing the style of the handwritten text based on the first sample handwritten text as a first style feature vector; then operates the first style feature vector according to a preset operation rule to derive at least one second style feature vector; finally determines the second sample handwritten text through the second style feature vector, thereby increasing the number of styles of the handwritten text style. The technical solution of the present application can at least achieve the following beneficial effects:
[0138] First, since the number of handwritten text styles is increased based on the style feature vector of the first sample handwritten text, other styles different from the style in the first sample handwritten text can be derived. In the entire technical solution, there is no need to rely on manual collection of large amounts of data, thereby saving manpower and material resources and improving the efficiency of generating handwritten text data training sets.
[0139] Second aspect: compared with the number of handwritten text styles collected manually, the number of handwritten text styles that can be obtained through the technical solution of the present application is not limited, so the handwritten text styles formed are more diverse.
[0140] The third aspect: Based on the diverse handwritten text styles formed by the technical solution of the present application, various styles in the diverse handwritten text styles are learned based on the constructed first neural network model, so as to expand the handwritten texts of the diverse handwritten text styles, thereby obtaining a handwritten text data training set with diverse styles and rich handwritten texts.
[0141] Fourth aspect: Using the handwritten text data training set with diverse styles and rich handwritten text obtained by the technical solution of the present application to train the model can obtain a handwritten text recognition model with higher recognition accuracy.
[0142] Fifth aspect: Through the multiple handwritten text unit sets constructed by this application, the printed text to be converted can be converted into the target handwritten text in the target style, thereby enhancing the application interest of the multiple handwritten text unit sets.
[0143] The following describes an apparatus embodiment of the present application, which can be used to execute the text data processing method in the above-mentioned embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the embodiment of the text data processing method in the above-mentioned embodiment of the present application.
[0144] Fig. 9 A block diagram of a device for processing text data according to an embodiment of the present application is shown.
[0145] Reference Fig. 9 As shown, a text data processing device 900 according to an embodiment of the present application includes: an acquisition unit 901 , a first determination unit 902 , a derivation unit 903 , and a second determination unit 904 .
[0146] Among them, the acquisition unit 901 is used to acquire a first sample handwritten text; the first determination unit 902 is used to determine a style feature vector based on the first sample handwritten text as a first style feature vector, and the style feature vector is used to characterize the style of the handwritten text; the derivation unit 903 is used to operate the first style feature vector according to a preset operation rule to derive at least one second style feature vector; the second determination unit 904 is used to determine the second sample handwritten text through the second style feature vector to increase the number of styles of the handwritten text style.
[0147] In some embodiments of the present application, based on the aforementioned scheme, the first sample handwritten text includes sample handwritten texts of at least two styles, the first style feature vector includes the first style feature vectors corresponding to the at least two styles of sample handwritten texts, and the derivation unit 903 is also used to: operate at least two first style feature vectors according to preset operation rules to derive at least one second style feature vector.
[0148] In some embodiments of the present application, based on the above solution, the derivation unit 903 is further used to: perform an interpolation operation on at least two first style feature vectors according to a preset operation rule to derive at least one second style feature vector.
[0149] In some embodiments of the present application, based on the aforementioned scheme, the derivation unit 903 is further used to: arbitrarily determine two first style feature vectors from the at least two first style feature vectors; determine the element value interval corresponding to each vector element position based on the element value of each vector element position in the two first style feature vectors; and interpolate within each element value interval to obtain at least one second style feature vector, wherein the number of elements of the second style feature vector is the same as the number of elements of the first style feature vector.
[0150] In some embodiments of the present application, based on the aforementioned scheme, the processing device of the present application also includes a multiple interpolation unit, and the multiple interpolation unit is used to: perform multiple interpolation operations on the first style feature vector or the second style feature vector according to the preset operation rules to derive a style feature vector for characterizing the style of the handwritten text.
[0151] In some embodiments of the present application, based on the aforementioned scheme, the multiple interpolation unit is further used to: perform multiple interpolation operations on the first style feature vector and the second style feature vector according to the preset operation rules to derive a style feature vector for characterizing the style of the handwritten text.
[0152] In some embodiments of the present application, based on the aforementioned scheme, the processing device of the present application also includes an imitation unit, which is used to: obtain sample handwritten text, and obtain printed text that is different from the text content in the sample handwritten text, the sample handwritten text includes the first sample handwritten text and / or the second sample handwritten text; for each sample handwritten text, train a first neural network model through each sample handwritten text so that the first neural network model learns the style of each sample handwritten text; through the first neural network model, convert the printed text into an imitation handwritten text consistent with the style of each sample handwritten text.
[0153] In some embodiments of the present application, based on the aforementioned scheme, the first neural network model includes an encoder and a decoder, and the imitation unit is further used to: input the printed text into the encoder of the first neural network model, so that the encoder encodes the printed text to obtain a text feature vector of the printed text; concatenate the text feature vector of the printed text with the style feature vector used to characterize each sample handwritten text to obtain an intermediate feature vector; and decode the intermediate feature vector by the decoder to obtain an imitation handwritten text consistent with the style of each sample handwritten text.
[0154] In some embodiments of the present application, based on the aforementioned scheme, the processing device of the present application also includes a recognition unit, which is used to: train a second neural network model through each sample handwritten text and an imitation handwritten text corresponding to each sample handwritten text, so that the second neural network model learns the text style and text content of each handwritten text; recognize the handwritten text to be recognized through the second neural network model, so as to convert the handwritten text to be recognized into a target printed text.
[0155] In some embodiments of the present application, based on the aforementioned scheme, the processing device of the present application also includes a construction unit, which is used to: determine handwritten text units with the same text content from each sample handwritten text and the imitation handwritten text corresponding to each sample handwritten text; based on the handwritten text units with the same text content, construct multiple handwritten text unit sets, wherein each handwritten text unit set includes handwritten texts with the same text content but different styles.
[0156] In some embodiments of the present application, based on the aforementioned scheme, the processing device of the present application also includes a splicing unit, which is used to: obtain the printed text to be converted, and split the printed text in the printed text to be converted into at least one printed text unit; for each printed text unit, extract a handwritten text unit that conforms to the target style from the corresponding handwritten text unit set according to the text content of each printed text unit, as a target handwritten text unit; splice each of the target handwritten text units to obtain a target handwritten text corresponding to the printed text to be converted.
[0157] Fig.10 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.
[0158] It should be noted that Fig.10 The computer system 1000 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0159] like Fig.10As shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage part 1008 to the random access memory (RAM) 1003, such as executing the method described in the above embodiment. In the RAM 1003, various programs and data required for system operation are also stored. The CPU 1001, the ROM 1002 and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.
[0160] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.
[0161] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part 1009, and / or installed from a removable medium 1011. When the computer program is executed by a central processing unit (CPU) 1001, various functions defined in the system of the present application are executed.
[0162] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0163] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0164] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.
[0165] As another aspect, the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the text data processing method described in the above embodiment.
[0166] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the text data processing method described in the above embodiment.
[0167] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.
[0168] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.
[0169] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary techniques in the art that are not disclosed in the present application. It should be understood that the present application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A method for processing text data, It is characterized in that The method comprises: Obtaining a first sample handwritten text; Determine a style feature vector based on the first sample handwritten text as a first style feature vector, wherein the style feature vector is used to characterize the style of the handwritten text; Calculate the first style feature vector according to a preset calculation rule to derive at least one second style feature vector; The second sample handwritten text is determined by using the second style feature vector to increase the number of styles of handwritten text.
2. The method according to claim 1, It is characterized in that The first sample handwritten text includes sample handwritten texts of at least two styles, the first style feature vector includes a first style feature vector corresponding to the sample handwritten texts of the at least two styles, and the first style feature vector is operated according to a preset operation rule to derive at least one second style feature vector, including: At least two first style feature vectors are operated according to a preset operation rule to derive at least one second style feature vector.
3. The method according to claim 2, It is characterized in that The step of operating at least two first style feature vectors according to a preset operation rule to derive at least one second style feature vector includes: An interpolation operation is performed on at least two first style feature vectors according to a preset operation rule to derive at least one second style feature vector.
4. The method according to claim 3, It is characterized in that The interpolation operation is performed on at least two first style feature vectors according to a preset operation rule to derive at least one second style feature vector, including: arbitrarily determining two first style feature vectors from the at least two first style feature vectors; Determining, based on the element values of the respective vector element positions in the two first style feature vectors, the element value intervals corresponding to the respective vector element positions; Interpolation is performed in each element value interval to obtain at least one second style feature vector, wherein the number of elements in the second style feature vector is the same as the number of elements in the first style feature vector.
5. The method according to claim 4, It is characterized in that The method further comprises: Perform multiple interpolation operations on the first style feature vector or the second style feature vector according to the preset operation rule to derive a style feature vector for representing the style of the handwritten text.
6. The method according to claim 4, It is characterized in that The method further comprises: The first style feature vector and the second style feature vector are interpolated multiple times according to the preset operation rule to derive a style feature vector for representing the style of the handwritten text.
7. The method according to claim 1, It is characterized in that The method further comprises: Acquire sample handwritten text, and acquire printed text different from the text content in the sample handwritten text, wherein the sample handwritten text includes the first sample handwritten text and / or the second sample handwritten text; For each sample handwritten text, training a first neural network model by using each sample handwritten text so that the first neural network model learns the style of each sample handwritten text; The printed text is converted into imitation handwritten text consistent with the style of each sample handwritten text through the first neural network model.
8. The method according to claim 7, It is characterized in that The first neural network model includes an encoder and a decoder, and the first neural network model is used to convert the printed text into an imitation handwritten text consistent with the style of each sample handwritten text, including: Inputting the printed text into the encoder of the first neural network model so that the encoder encodes the printed text to obtain a text feature vector of the printed text; Concatenating the text feature vector of the printed text with the style feature vector used to characterize each of the sample handwritten texts to obtain an intermediate feature vector; The decoder decodes the intermediate feature vector to obtain simulated handwritten text that is consistent with the style of each sample handwritten text.
9. The method according to any one of claims 7 to 8, It is characterized in that The method further comprises: Training a second neural network model through each sample handwritten text and the imitation handwritten text corresponding to each sample handwritten text, so that the second neural network model learns the text style and text content of each handwritten text; The handwritten text to be recognized is recognized by the second neural network model to convert the handwritten text to be recognized into a target printed text.
10. The method according to any one of claims 7 to 8, It is characterized in that The method further comprises: Determine handwritten text units with the same text content from each sample handwritten text and the imitation handwritten text corresponding to each sample handwritten text; Based on the handwritten text units with the same text content, a plurality of handwritten text unit sets are constructed, wherein each handwritten text unit set includes handwritten texts with the same text content but different styles.
11. The method according to claim 10, It is characterized in that The method further comprises: Acquire a printed text to be converted, and split the printed text in the printed text to be converted into at least one printed text unit; For each printed text unit, extracting a handwritten text unit that meets the target style from the corresponding handwritten text unit set according to the text content of each printed text unit as a target handwritten text unit; The target handwritten text units are spliced together to obtain a target handwritten text corresponding to the printed text to be converted.
12. A text data processing device, It is characterized in that The device comprises: An acquisition unit, used for acquiring a first sample handwritten text; A first determining unit, configured to determine a style feature vector based on the first sample handwritten text as a first style feature vector, wherein the style feature vector is used to characterize the style of the handwritten text; a derivation unit, configured to operate the first style feature vector according to a preset operation rule to derive at least one second style feature vector; The second determining unit is used to determine a second sample handwritten text by using the second style feature vector to increase the number of styles of the handwritten text.
13. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores at least one program code, and the at least one program code is loaded and executed by a processor to implement the operations performed by the method according to any one of claims 1 to 11.
14. An electronic device, It is characterized in that The method comprises one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the method according to any one of claims 1 to 11.
Citation Information
Patent Citations
Handwritten character recognition method
CN112308058A
Handwritten image generation method, model training method, device and equipment
CN113516136A
Chinese font generation method and system based on nested coding-decoding network
CN114330237A
Method and device for generating whole-line handwritten sample
CN115331236A
Handwritten data generation method and device, storage medium and electronic equipment
CN115565185A
