Text processing method and device, electronic equipment and storage medium

Abstract

This disclosure provides a text processing method, apparatus, electronic device, and storage medium. The method includes: acquiring a first image including text to be processed; training a target stroke order determination model by combining spatial attention and channel attention mechanisms; and inputting the first image into the pre-trained target stroke order determination model to obtain a target stroke order corresponding to the text to be processed. The technical solution of this disclosure can accurately obtain the position and order of each stroke of the text, greatly reducing the occurrence of stroke breaks, uneven stroke edges, missing or redundant strokes in the generated text, and improving the accuracy of the generated text.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a text processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, research on font generation using artificial intelligence (AI) has been gradually unfolding. This approach not only meets users' needs for a variety of fonts but also improves designers' productivity.

[0003] When generating text using relevant models in practice, existing style transfer or image translation techniques excel at modifying image texture but are less adept at modifying structural information. However, in the field of text generation, the character structure is precisely the key differentiator between fonts. Therefore, fonts generated using existing technologies often suffer from numerous problems, such as broken strokes, uneven stroke edges, missing or redundant strokes. This not only causes discrepancies between automatically generated text and the user's expectations but also results in a high error rate. Summary of the Invention

[0004] This disclosure provides a text processing method, apparatus, electronic device, and storage medium that can accurately obtain the position and order of each stroke of a character, greatly reducing the occurrence of broken strokes, uneven stroke edges, missing or redundant strokes in the generated text, and improving the accuracy of the generated text.

[0005] In a first aspect, embodiments of this disclosure provide a text processing method, including:

[0006] Acquire the first image containing the text to be processed;

[0007] The model is determined by training the target stroke order by combining spatial attention and channel attention mechanisms.

[0008] The first image is input into a pre-trained target stroke order determination model to obtain the target stroke order corresponding to the text to be processed.

[0009] Secondly, embodiments of this disclosure also provide a word processing device, including:

[0010] The first image acquisition module is used to acquire a first image including the text to be processed;

[0011] The stroke order determination model training module is used to train the target stroke order determination model by combining spatial attention mechanism and channel attention mechanism.

[0012] The target stroke order determination module is used to input the first image into a pre-trained target stroke order determination model to obtain the target stroke order corresponding to the text to be processed.

[0013] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0014] One or more processors;

[0015] Storage device for storing one or more programs.

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the text processing method as described in any of the embodiments of this disclosure.

[0017] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the text processing method as described in any of the embodiments of this disclosure.

[0018] The technical solution of this disclosure first acquires a first image including the text to be processed, and then inputs the first image into a pre-trained target stroke order determination model that includes spatial attention mechanism and channel attention mechanism, thereby obtaining the target stroke order corresponding to the text to be processed. By introducing the above two mechanisms into the stroke order determination model, the position and order of each stroke of the text can be accurately obtained, thereby greatly reducing the occurrence of stroke breakage, uneven stroke edges, missing or redundant strokes in the generated text, and improving the accuracy of the generated text. Attached Figure Description

[0019] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0020] Figure 1 This is a schematic flowchart of a text processing method provided in an embodiment of the present disclosure;

[0021] Figure 2 This is a schematic diagram of the stroke order determination model provided in the embodiments of this disclosure;

[0022] Figure 3 This is a schematic flowchart of a text processing method provided in an embodiment of the present disclosure;

[0023] Figure 4 A schematic diagram of the style feature fusion model provided in the embodiments of this disclosure;

[0024] Figure 5 A schematic diagram of the target text style provided in the embodiments of this disclosure;

[0025] Figure 6 This is a schematic diagram of the structure of a text processing device provided in an embodiment of the present disclosure;

[0026] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0027] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0028] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0029] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0030] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules, or units, and are not used to limit the order of functions performed by these devices, modules, or units or their interdependencies. It should also be noted that the modifications of "a" and "a plurality of" mentioned in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0031] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0032] Before introducing this technical solution, an exemplary application scenario can be provided. This technical solution can be applied to scenarios where the stroke order of characters needs to be determined with high accuracy based on neural networks. For example, when generating characters of a certain font using artificial intelligence algorithms, problems such as broken strokes, uneven stroke edges, missing or redundant strokes may occur in the generated characters. In this case, the solution based on this embodiment can accurately determine the stroke order and position of each stroke, thereby avoiding the occurrence of the above problems.

[0033] Figure 1 This is a schematic flowchart of a text processing method provided in an embodiment of the present disclosure. The embodiments of the present disclosure are applicable to situations where the stroke order of characters can be determined with high accuracy. The method can be executed by a text processing device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, such as a mobile terminal, a PC, or a server.

[0034] like Figure 1 As shown, the method includes:

[0035] S110. Obtain the first image containing the text to be processed.

[0036] The first image can be an image received by the server or client and captured in real time by the user through a camera device, or it can be a stored image retrieved by the server or client from a relevant database. At the same time, the image includes at least one or more characters. It can be understood that the characters in the image are the characters to be processed. Based on the neural network model of this embodiment, it is necessary to determine at least the stroke order of the characters to be processed.

[0037] For example, when a user photographs a calligraphy work containing a single Chinese character and uploads the image to a server or client, this image becomes the first image. Simultaneously, the server or client can use relevant algorithms to identify the Chinese character in the image as the text to be processed. Of course, in practical applications, the text in the first image can also be other types of characters besides Chinese, such as English or Latin. Furthermore, the number of characters to be processed in the first image can be one or more; this embodiment does not impose specific limitations.

[0038] S120. Input the first image into the pre-trained target stroke order determination model to obtain the target stroke order corresponding to the text to be processed.

[0039] In this embodiment, after the server or the client determines the first image, the image can be input into a pre-trained target stroke order model. The target stroke order model can be a long short-term memory artificial neural network model (Long Short-Term Memory, LSTM) with a spatial attention mechanism and a channel attention mechanism. That is to say, this model is trained by combining the spatial attention mechanism and the channel attention mechanism.

[0040] In this embodiment, the target stroke order determination model incorporates a spatial attention mechanism and a channel attention mechanism. Specifically, based on the spatial attention mechanism, the model can use a spatial transformer to transform the spatial information in the original image to another space and extract and retain its key information during this process. Based on the channel attention mechanism, the model can add a weight to the signals on each channel during the convolution process to represent the relevance of the key information in the channel domain image. It can be understood that the greater this weight, the greater the relevance of the channel to the key information.

[0041] In this embodiment, after the first image is input into the target stroke order determination model for processing, the model can output the target stroke order corresponding to the text to be processed. The target stroke order is the information reflecting the structure of the text to be processed and the position and order of each stroke constituting the text. For example, when the text to be processed in the input first image is "cang", the model can output the positions and stroke order of the 4 strokes of this text, and at the same time determine the structure of this text.

[0042] It should be noted that before applying the target stroke order determination model of the embodiments of the present disclosure, the stroke order determination model to be trained needs to be trained first. Optionally, obtain at least one first training sample; for each first training sample, input the sample text image in the current first training sample into the stroke order determination model to be trained to obtain the predicted stroke order; based on the predicted stroke order and the theoretical text stroke order in the current first training sample, determine the loss value, and based on the loss value, correct the model parameters of the stroke order determination model to be trained; use the convergence of the loss function in the stroke order determination model to be trained as the training target to obtain the target stroke order determination model.

[0043] The first training sample includes a sample text image and the theoretical text stroke order corresponding to the sample text image. For example, the sample text image can be the image corresponding to the Chinese character "cang", and the theoretical text stroke order is the information accurately representing the position and order of each stroke of "cang". Based on this information, the server or the client can accurately determine a standard "cang" character.

[0044] In this embodiment, after obtaining the first training samples, each of the samples can be input into the to-be-trained stroke order determination model, so as to obtain the predicted stroke order. Continuing with the above example, after the to-be-trained stroke order determination model processes the image corresponding to the Chinese character "cang", it can output the information representing the position and order of each stroke of "cang". However, when the model is not yet trained, the server and the client cannot accurately construct the Chinese character according to the predicted stroke order corresponding to the character "cang", and the generated "cang" may have problems with some incorrect strokes. For example, based on the predicted stroke order, a character "he" may be generated.

[0045] Therefore, after obtaining the predicted stroke orders of each training sample, it is also necessary to determine the loss value of the model based on the predicted stroke order and the theoretical character stroke order in the training sample, and then correct the model parameters. Specifically, when using the loss value to correct the model parameters in the to-be-trained stroke order determination model, the convergence of the loss function can be used as the training goal. For example, whether the training error is less than the preset error, or whether the error change tends to be stable, or whether the current iteration number is equal to the preset number. If it is detected that the convergence condition is reached, such as the training error of the loss function is less than the preset error, or the error change trend tends to be stable, it indicates that the to-be-trained stroke order determination model is trained, and at this time, the iterative training can be stopped. If it is detected that the current condition does not reach the convergence condition, other training samples can be further obtained to continue training the to-be-trained stroke order determination model until the training error of the loss function is within the preset range. When the training error of the loss function reaches convergence, the trained to-be-trained stroke order determination model can be used as the target stroke order determination model. That is, when the text image is input into this target stroke order determination model, the stroke order of the text in the image can be accurately obtained.

[0046] It should be noted that both the target stroke order determination model and the to-be-trained stroke order determination model can process the text image in the following order. Optionally, the sample text image is input into the convolutional layer to obtain the first to-be-processed feature; the first to-be-processed feature is subjected to feature extraction through the channel attention mechanism and the spatial attention mechanism to obtain the second to-be-processed feature; the second to-be-processed feature is respectively input into the recurrent neural network unit to obtain the feature sequence corresponding to each stroke position; and the predicted stroke order is obtained based on the classifier processing each feature sequence. The following combines Figure 2 to illustrate this processing process.

[0047] Those skilled in the art should understand that the convolutional layer consists of several convolutional units, and the parameters of each convolutional unit can be optimized through the backpropagation algorithm. See Figure 2Taking the stroke order determination model as an example, after inputting the sample text image into the model, the image can be processed based on the Residual Network (ResNet) model to extract multiple features corresponding to the text image. The Residual Network model can be understood as a sub-network. Furthermore, when the convolutional layer is the first layer of the neural network, the extracted features can be low-level features, i.e., the first features to be processed.

[0048] See also Figure 2 Based on channel attention and spatial attention mechanisms, feature extraction can be performed on the first feature to be processed, thereby obtaining a more advanced and abstract second feature to be processed. Furthermore, since the stroke order determination model contains multiple recurrent neural network units, after obtaining multiple second features to be processed, these features need to be input into the corresponding recurrent neural network units to obtain the feature sequence corresponding to each stroke order position. It can be understood that this feature sequence is the output of each recurrent neural network unit.

[0049] See also Figure 2 After obtaining the feature sequences, a classifier can be used to process each feature sequence to predict the stroke order of the text. In this embodiment, the classifier is a classification function learned or a classification model constructed based on existing data. This function or model can map data to a specific item in a given category, thereby predicting the stroke order of the text. It can be understood that the feature vectors extracted from each RNN unit, when combined, constitute the stroke order feature (SOF). For any network generating text, the model can extract an SOF (SOF fake) from the generated fake image. Furthermore, another SOF (SOF ground truth) can be extracted from the corresponding ground truth data of the fake image. Based on this data, an additional loss (such as focal loss) can be generated for the stroke order model to be trained, thereby improving the quality of the generated font.

[0050] For example, Figure 2Module A contains multiple recurrent neural network units, which can be represented by A1, A2, ..., AN. Their number can correspond to the maximum number of strokes in a given character. For example, when the character is Chinese, the number of recurrent neural networks can be 36. After inputting the second feature to be processed into A1, the corresponding output H1 is obtained, representing the position and order of the first stroke of the Chinese character. Simultaneously, A1 can output a parameter different from H1, which is then input into A2. A2 then outputs H2, representing the position and order of the second stroke of the Chinese character. A2 also outputs a parameter different from H2, which is then input into A3, and so on, gradually obtaining the position and order information of each stroke of the Chinese character. It can be understood that in the above example, when the number of strokes in the Chinese character has not reached the number of recurrent neural network units, after obtaining the position and order information of each stroke, the output of one or more subsequent recurrent neural networks is zero. This embodiment of the present disclosure will not elaborate further on this.

[0051] It should be noted that the solution of this disclosure embodiment can be applied to office software installed on the server or client. That is, the above-mentioned target stroke order determination model is integrated into the office software. Based on this, when the text image input by the user is received, the office software deployed on the server or client can accurately determine the position and order information of the text strokes in the image based on the target stroke order determination model, and then perform subsequent processing according to actual needs based on this information.

[0052] The technical solution of this disclosure first acquires a first image including the text to be processed, and then inputs the first image into a pre-trained target stroke order determination model that includes spatial attention mechanism and channel attention mechanism, thereby obtaining the target stroke order corresponding to the text to be processed. By introducing the above two mechanisms into the stroke order determination model, the position and order of each stroke of the text can be accurately obtained, thereby greatly reducing the occurrence of stroke breakage, uneven stroke edges, missing or redundant strokes in the generated text, and improving the accuracy of the generated text.

[0053] Figure 3This is a flowchart illustrating a text generation method provided in this embodiment. Based on the aforementioned embodiments, the target stroke order model is used as the loss function of the style feature fusion model to be trained, thereby training the target style feature fusion model. This model allows the user to fuse the font style of the text to be processed and the font style of a reference text, obtaining any font style between the font styles of the text to be processed and the reference text, thus solving the problem of not being able to generate text with a font style between two font styles. Simultaneously, the style feature fusion model constructed based on multiple sub-models solves the problem of the target text's font style not matching the user's expected text style. Specific implementation details can be found in the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0054] like Figure 3 As shown, the method specifically includes the following steps:

[0055] S210. Obtain the first image containing the text to be processed.

[0056] S220. Input the first image into the pre-trained target stroke order determination model to obtain the target stroke order corresponding to the text to be processed.

[0057] S230. Use the target stroke order determination model as the loss model for the style feature fusion model to be trained, so as to obtain the target style feature fusion model.

[0058] The target style feature fusion model is used to fuse at least two font styles. It can be understood as a model that fuses different font styles. The target style feature fusion model can be a pre-trained neural network model. The input data of this model is in image format, and correspondingly, the output data is also in image format. The target style feature can be understood as fusing the font styles of the text to be processed and the reference text to obtain any font style between the two styles. It should be noted that the fused style features can include multiple types, and any font style can be used as the target style feature. Accordingly, the target text in the output image can be understood as text possessing the target style feature.

[0059] In this embodiment, the input of the target style feature fusion model can be the text image to be processed and the reference text image, and the output image is the image corresponding to the target style feature text. Specifically, the text to be processed can be understood as the text for which the user expects to perform font style conversion. The text in the text image to be processed can be the text selected by the user from the font library or the text written by the user. For example, after the user writes the text, image recognition can be performed on the written text, and the recognized text can be used as the text to be processed. The text in the reference text image can be understood as the text whose font style needs to be fused with the text style of the text to be processed. For example, the style of the reference text can include regular script style, clerical script style, running script style, cursive script style, or the user's handwritten font style, etc.

[0060] Specifically, after obtaining the text image to be processed and the reference text image, the text image to be processed and the reference text image can be input into the style feature fusion model to be trained. Refer to Figure 4 , the text to be processed in the text image to be processed is "Cang", and the reference text in the reference text image is "Jie". The font styles of the two texts are different. After inputting the text to be processed and the reference text, the two texts are converted into the corresponding two images to be processed through image conversion, and then the two images to be processed obtained are input into the style feature fusion model to be trained. After processing the two images based on the style feature fusion model to be trained, a font style between the font style of the text to be processed and the font style of the reference text can be obtained. Any one of the font styles can be used as the target font style, and the target text corresponding to the target font style can be obtained.

[0061] It should be noted that if the text of the obtained target style feature does not match the style feature required by the user, the user can use the text with the target font style as the text to be processed and continue to fuse the style features of the font until the style feature satisfactory to the user is obtained.

[0062] Exemplarily, taking the font style processing of "Ji" as an example, refer to Figure 5, input the text image to be processed numbered 1 and the text image to be processed corresponding to number 10 into the target style feature fusion model, and any font style between numbers 2 and 9 can be obtained, and any font style can be used as the target style feature. For example, if the obtained target font style feature is the font style numbered 5, and the font style actually required by the user is the font style numbered 8, that is, when the obtained target font style feature is different from the font style expected by the user, the font style can be further fused based on the target style feature fusion model. Optionally, input numbers 5 and 10 as the text images to be processed into the target style feature fusion model for processing until the target font style consistent with the font style expected by the user is obtained.

[0063] During the process of training the target style feature fusion model, optionally, determine at least one training sample; for each training sample, input the text image to be processed and the reference text image in the current training sample into the style feature fusion model to be trained, and obtain the actual output text image corresponding to the text image to be processed; based on the target stroke order determination model, perform stroke loss processing on the actual output text image and the text image to be processed to obtain the first loss value; based on the reconstruction loss function, determine the actual output text image and the text image to be trained, and determine the reconstruction loss; based on the style encoding loss function, determine the style loss value between the actual output text image and the fused text image.

[0064] It can be understood that the training sample includes the text image to be trained and the reference text image; the fused text image is determined based on the font styles of the text image to be trained and the reference text image. Exemplarily, when the image of the text "仓" to be processed and the image of the reference text "颉" in the training sample are obtained, the images of these two characters can be input into the style feature fusion model to be trained, so as to obtain an image of the character "仓" with a font style similar to that of the character "颉". This image is the actual output text image. At the same time, when the model has not been trained yet, this image may not accurately reflect the structure of each stroke of the character "仓". For example, the stroke positions of the generated character "仓" are inaccurate, or even the character "合" is generated, or the generated character "仓" does not accurately reflect the font style of the character "颉". Therefore, it is still necessary to perform stroke loss processing on the image of the actual output text "仓" and the image of the text "仓" to be processed based on the target stroke order determination model (Stroke Order Loss) integrated into the model that has been trained, and obtain the first loss value. It can be understood that the number of nodes of the RNN in the target stroke order determination model is the maximum number of strokes of Chinese characters, and the features predicted by each node are combined through a connection function to form a stroke order feature matrix. The specific processing process can be implemented in the manner described in detail in the above embodiments, and will not be elaborated in this embodiment of the present disclosure.

[0065] In this embodiment, when the actual output text image is the character image of "合", the actual output character image of "合" and the text image to be trained (i.e., the standard character image of "仓") can also be determined based on the reconstruction loss function (Rec Loss), and then the reconstruction loss can be determined. It can be understood that the reconstruction loss function is used to intuitively constrain whether the network output conforms to the reconstruction loss, which is used to correct the model parameters in the subsequent process, so that the model with corrected parameters can output the stroke positions and order of the text exactly the same as those of the character "仓".

[0066] In this embodiment, if the actual output text image is the character image of "仓", but there are significant differences in the font style from that of the character "颉", the actual output character image of "仓" and the fused text image (i.e., the image with the same font style as the character "颉") can be determined based on the style encoding loss function (Triplet loss), and then the style loss value can be determined. It can be understood that the style encoding loss function is used to constrain the L2 norm of the font style encodings generated by different fonts to be as close to 0 as possible. That is to say, the style encoding loss function can obtain the L2 norm between two different font styles. According to the value of the L2 norm, it can be determined which font style the obtained font style is more biased towards. In order to make the fusion of different font styles continuous and keep the value of the L2 norm as close to 0 as possible, the fused font style is between the two font styles and does not bias towards any one of them. The style loss value is also used to correct the model parameters in the subsequent process, so that the corrected model can output the text font style exactly the same as that of the character "颉".

[0067] In this embodiment, after obtaining the first loss value, the reconstruction loss, and the style loss, the model parameters in the style fusion model to be trained can be corrected based on the first loss value, the reconstruction loss, and the style loss; taking the convergence of the loss function in the style feature fusion model to be trained as the training goal, the target style feature fusion model is trained. It can be understood that there are differences in the training objects and corresponding loss functions between the style fusion model to be trained and the above-mentioned stroke order determination model to be trained, and their specific training steps are similar to those of the stroke order determination model to be trained, which will not be elaborated in this embodiment of the present disclosure.

[0068] It should also be noted that the target style feature fusion model includes a style feature extraction sub-model, a stroke feature extraction sub-model, a content extraction sub-model, and an encoding sub-model. The following will be combined with Figure 4 to illustrate the above sub-models.

[0069] See Figure 2, in the figure, the box 1 is a style feature extraction sub-model for extracting the reference style of the reference text image, and the box 2 is a stroke feature extraction sub-model for extracting the stroke features of the text to be processed. In the content extraction sub-model (i.e., the font style extractor), the reference character "Xie" and the font style label corresponding to the character "Xie" can be input. Therefore, it can be understood that the content extraction sub-model is used to extract the content features of the text to be trained, where the content features include the text content and the style of the text to be processed. The encoding sub-model is used to encode the reference font style, stroke features, and image features to obtain the actual output text image. Specifically, after extracting the font style of the reference text, based on the encoding sub-model, the extraction result can be encoded, and then the encoding result of the font style of the reference text and the extraction result of the stroke order features of the text to be processed are jointly input into the compiler (Decoder) to obtain a text with a font style between the font styles of the text to be processed and the reference text through the compiler. In addition, after the encoding sub-model, a stroke order prediction sub-model is also connected, which is used to predict the stroke order of the input text. Exemplarily, as Figure 2 shown, the stroke features corresponding to the character "Cang" are "left-falling stroke", "right-falling stroke", "horizontal fold hook", and "vertical hook with a turn". After inputting the character "Cang" into the model, the corresponding stroke features of the character "Cang" can be stored in the ht vector respectively, and the vector ht = {h1, h2, h3, and h4} can be obtained according to the stroke order. Then, the obtained stroke order vector is input into the stroke order prediction model, and the stroke features are trained and analyzed based on a neural network (such as a convolutional neural network) so that after the training of the style feature fusion model to be trained is completed, the stroke features of each character can be predicted to avoid the situation of missing or incorrect stroke order in the output text result.

[0070] It should be noted that before using the target style feature extraction sub-model to process the text image, it also includes training the stroke feature extraction sub-model in the target style feature fusion model. Specifically, in the training process of the stroke feature extraction sub-model, a first training sample set can be obtained; where the first training sample set includes multiple first training samples, and each first training sample includes a first image corresponding to the first training text and a first stroke vector; for each first training sample, the first image of the current first training sample is used as the input parameter of the stroke feature extraction sub-model to be trained, and the corresponding first stroke vector is used as the output parameter of the stroke feature extraction sub-model to be trained to train the stroke feature extraction sub-model to obtain the stroke feature extraction sub-model.

[0071] S240. Generate a text package that combines at least two font styles based on the target style feature fusion model.

[0072] In this embodiment, after obtaining the target style feature fusion model, the model can be used to generate a text package that fuses at least two font styles. The text package includes multiple texts to be used, and the texts to be used are generated based on the target style feature fusion model. Specifically, texts with two different font styles can be obtained, and the images corresponding to the two texts are respectively processed based on the target style feature fusion model to obtain any font style between the two font styles. If the obtained font style is consistent with the user's expectation at this time, the texts with the above two font styles can be processed based on the target font style fusion model to obtain the texts to be used in the corresponding styles for each text. The set of all texts to be used can be the text package.

[0073] It should be noted that after generating a text package that fuses at least two styles, the text package can be integrated into relevant application programs, for example, the generated text package can be fused into the drop-down list of the editing bar of a text processing application. The display mode of this drop-down list can be a drop-down window containing each text style or a picture display window, etc. The user can click and select the target font style based on the option information in the list. When the client or the server receives a relevant request from the user to select the target font style, the text package resource corresponding to this font style can be provided to the user, so that the user can use multiple texts to be used for text editing and processing work.

[0074] Exemplarily, when the user selects the target font style as font C fused by font A and font B in the drop-down list, when receiving the input text to be processed "可", the server or the client can determine the character "可" from the text package corresponding to the target font style C and display it as the target text. Those skilled in the art should understand that this technical solution can be applied in existing office software, that is, integrating this technical solution in office software, or directly integrating the text package in office software, or integrating the target style feature fusion model into a certain application software on the server or the client. Of course, in the actual application process, one or more of the above methods can be selected according to needs to implement the technical solution of the present disclosure. The embodiments of the present disclosure do not make specific limitations here.

[0075] In this embodiment, when receiving the target reference style text image and the target style conversion text image, at least one display text image can also be output based on the text content and conversion text style of the target style conversion text image, and the reference text style of the target reference style text image, so as to determine the target display text image based on a trigger operation. The following combines Figure 5 An exemplary description will be given to the process of determining the target display text image.

[0076] See Figure 5, when the user hopes to obtain at least ten styles of text, the target style feature fusion model for generating these ten styles of text can be integrated into the server or the client. Further, after integrating the target style feature fusion models corresponding to the ten fonts into a server or a client with sufficient computing power, when receiving the target reference style text image containing the Chinese character "济" numbered 1 and the target style conversion text image containing the Chinese character "济" numbered 10, the server or the client integrated with the target style feature fusion model can be used to process the above two images, determine the content of the text and the corresponding text styles in the two images, and then output the display text images respectively containing the Chinese character "济" numbered 2-9. Through Figure 5 It can be seen that the font styles of each "济" character finally obtained are between the font style of the "济" character numbered 1 and the font style of the "济" character numbered 10. It can be understood that the font styles of the text in these images are obtained by fusing two font styles. Further, if the Chinese character "济" numbered 5 and its font style both meet the user's expectations, the user can perform a trigger operation on the display text image (such as clicking on the image containing the Chinese character "济" numbered 5 on the touch screen), or issue a confirmation instruction for the image of the Chinese character "济" numbered 5 to the server or the client in various ways. When the server or the client detects the trigger operation or receives the confirmation instruction, the image containing the Chinese character "济" numbered 5 can be determined as the target display text image, and then a text package consistent with the font style of this text can be constructed in the manner of the embodiments of the present disclosure. The embodiments of the present disclosure will not be elaborated herein.

[0077] It can be understood that: Different models with different fusion ratios can be pre-trained and deployed on the mobile terminal or the server, so that when the initial input text image is detected, based on each model, the text styles of the two text images can be fused to obtain text images with different fusion ratios and displayed. The user can trigger any text image and use the text image corresponding to the click confirmation as the target display text image. At the same time, the target model corresponding to the target display text image can be recorded, and a corresponding text package can be generated based on the target model, or text editing can be performed in real time. It can also be that a text package is generated based on the model for use during subsequent text editing.

[0078] Optionally, based on the target style feature fusion model corresponding to the target display text image, text editing is performed in real time, or a text package corresponding to the target display text image is generated. Specifically, when the server or the client determines the target style feature fusion model corresponding to the target display text image based on the user's selection, this model can be used as the model used by the server or the client at the current node. On this basis, when the server or the client receives the text information input by the user, this model can be used to convert all one or more characters in the text information into the font style of the characters in the target display text image, and the converted characters are displayed on the corresponding display interface, so as to implement the real-time processing function of the text font style. For example, when the user uses the image of the character "济" numbered 5 as the target display text image, the application can determine that the model generating this image is the target style feature fusion model used in the current stage. Based on this, when the user inputs any Chinese character in real time, the server or the client can use the target style feature fusion model to generate a Chinese character with the same font style and proportion as the character "济" numbered 5.

[0079] Alternatively, when the server or the client determines the target style feature fusion model corresponding to the target display text image based on the user's selection, it can directly use this model to convert the fonts of all the characters in the existing font library. After obtaining multiple characters with the same font style as the characters in the target display text image, a new text package can be constructed based on these characters, and then the text package can be integrated into the system or the corresponding application software for the user to use. Of course, in the actual application process, after determining the target style feature fusion model, the above two processing methods can be selected according to actual needs, and the embodiments of the present disclosure do not make specific limitations in this regard.

[0080] It should be noted that if the text style of the target display text is inconsistent with the expected text style, the target reference style text image and / or the target style conversion text image are updated according to the expected text style. The following continues with Figure 5 as an example for illustration.

[0081] Continue to refer to Figure 5 , if the character "济" numbered 5 and its font style do not meet the user's expectations, and the character "济" numbered 4 and its font style are the characters that the user finally wants to obtain. At this time, the server or the client can use the image of the character "济" numbered 1 as the target reference style text image and the image of the character "济" numbered 5 as the target style conversion text image in the above manner, and then continue to use the target style feature fusion model to process the above two images, so as to obtain the image of the character "济" numbered 3, and continue to determine whether the font style of the characters in this image meets the user's expectations according to the user's trigger operation.

[0082] The technical solution of this embodiment uses the target stroke order model as the loss function of the style feature fusion model to be trained, thereby training the target style feature fusion model. This model allows users to fuse the font style of the text to be processed and the font style of the reference text to obtain any font style between the font styles of the text to be processed and the reference text, thus solving the problem of not being able to generate text with a font style between two font styles. At the same time, the style feature fusion model built based on multiple sub-models solves the problem of the font style of the target text not matching the font style expected by the user.

[0083] Figure 6 This is a schematic diagram of the structure of a word processing device provided in an embodiment of the present disclosure, as shown below. Figure 6 As shown, the device includes: a first image acquisition module 310, a stroke order determination model training module 320, and a target stroke order determination module 330.

[0084] The first image acquisition module 310 is used to acquire a first image including the text to be processed.

[0085] The stroke order determination model training module 320 is used to train the target stroke order determination model by combining spatial attention mechanism and channel attention mechanism.

[0086] The target stroke order determination module 330 is used to input the first image into a pre-trained target stroke order determination model to obtain the target stroke order corresponding to the text to be processed.

[0087] Based on the above technical solutions, the text processing device also includes a first training sample acquisition module, a predicted stroke order determination module, a correction module, and a target stroke order determination model determination module.

[0088] The first training sample acquisition module is used to acquire at least one first training sample; wherein the first training sample includes a sample text image and the theoretical stroke order of the text corresponding to the sample text image.

[0089] The stroke order prediction module is used to input the sample text image in the current first training sample into the stroke order determination model to be trained, and obtain the predicted stroke order for each first training sample.

[0090] The correction module is used to determine a loss value based on the predicted stroke order and the theoretical stroke order in the current first training sample, and to correct the model parameters of the stroke order to be trained based on the loss value.

[0091] The target stroke order determination model determination module is used to take the convergence of the loss function in the stroke order determination model to be trained as the training objective, so as to obtain the target stroke order determination model.

[0092] Optionally, the stroke order prediction module is further configured to input the sample text image into a convolutional layer to obtain a first feature to be processed; extract features from the first feature to be processed through the channel attention mechanism and the spatial attention mechanism to obtain a second feature to be processed; input the second feature to be processed into a recurrent neural network unit to obtain a feature sequence corresponding to each stroke order position; and process each feature sequence based on a classifier to obtain the predicted stroke order.

[0093] Based on the above technical solutions, the text processing device also includes a loss model determination module.

[0094] The loss model determination module is used to use the target stroke order determination model as the loss model of the style feature fusion model to be trained, so as to train the target style feature fusion model; wherein, the target style feature fusion model is used to fuse at least two font styles.

[0095] Optionally, the loss model determination module is further configured to determine at least one training sample; wherein the training sample includes a text image to be trained and a reference text image; for each training sample, the text image to be processed and the reference text image in the current training sample are input into the style feature fusion model to be trained to obtain an actual output text image corresponding to the text image to be processed; the actual output text image and the text image to be processed are processed by stroke loss based on the target stroke order determination model to obtain a first loss value; the actual output text image and the text image to be trained are determined based on the reconstruction loss function to determine the reconstruction loss; the style loss value of the actual output text image and the fused text image is determined based on the style encoding loss function; wherein the fused text image is determined based on the font style of the text image to be trained and the reference text image; the model parameters in the style fusion model to be trained are corrected based on the first loss value, the reconstruction loss, and the style loss; the convergence of the loss function in the style feature fusion model to be trained is used as the training objective to train the target style feature fusion model.

[0096] Based on the above technical solutions, the target style feature fusion model includes a style feature extraction sub-model, a stroke feature extraction sub-model, a content extraction sub-model, and an encoding sub-model. The style feature extraction sub-model is used to extract the reference style of the reference text image. The stroke feature extraction sub-model is used to extract the stroke features of the text to be processed. The content extraction sub-model is used to extract the content features of the text to be trained. The content features include text content and the style of the text to be processed. The encoding sub-model is used to encode the reference font style, stroke features, and image features to obtain the actual output text image.

[0097] Based on the above technical solutions, the text processing device also includes a text package generation module.

[0098] The text package generation module is used to generate a text package that integrates at least two font styles based on the target style feature fusion model.

[0099] Based on the above technical solutions, the text processing device also includes an image receiving module and a text image determination module.

[0100] The image receiving module is used to receive the target reference style text image and the target style converted text image.

[0101] The display text image determination module is used to output at least one display text image based on the text content and text style of the target style conversion text image and the reference text style of the target reference style text image, so as to determine the target display text image based on the trigger operation.

[0102] Based on the above technical solutions, the text processing device also includes a text processing module.

[0103] The text processing module is used to perform real-time text editing or generate a text package corresponding to the target displayed text image based on the target style feature fusion model corresponding to the target displayed text image.

[0104] Based on the above technical solutions, the text processing device also includes an image update module.

[0105] An image update module is used to update the target reference style text image and / or the target style converted text image according to the expected text style if the text style of the target displayed text is inconsistent with the expected text style.

[0106] The technical solution provided in this embodiment first acquires a first image including the text to be processed, and then inputs the first image into a pre-trained target stroke order determination model that includes spatial attention mechanism and channel attention mechanism, thereby obtaining the target stroke order corresponding to the text to be processed. By introducing the above two mechanisms into the stroke order determination model, the position and order of each stroke of the text can be accurately obtained, thereby greatly reducing the occurrence of stroke breakage, uneven stroke edges, missing or redundant strokes in the generated text, and improving the accuracy of the generated text.

[0107] The text processing apparatus provided in this disclosure can execute the text processing method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of executing the method.

[0108] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0109] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Reference is made below. Figure 7 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 7 The diagram below shows the structure of the terminal device or server 400. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0110] like Figure 7 As shown, electronic device 400 may include a processing device (e.g., a central processing unit, pattern processor, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from storage device 406 into random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of electronic device 400. The processing device 401, ROM 402, and RAM 403 are interconnected via bus 404. An edit / output (I / O) interface 405 is also connected to bus 404.

[0111] Typically, the following devices can be connected to I / O interface 405: editing devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 400 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0112] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 409, or installed from storage device 406, or installed from ROM 402. When the computer program is executed by processing device 401, it performs the functions defined in the methods of embodiments of this disclosure.

[0113] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0114] The electronic device provided in this embodiment and the text processing method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0115] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the text processing method provided in the above embodiments.

[0116] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0117] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0118] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0119] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to:

[0120] Acquire the first image containing the text to be processed;

[0121] The model is determined by training the target stroke order by combining spatial attention and channel attention mechanisms.

[0122] The first image is input into a pre-trained target stroke order determination model to obtain the target stroke order corresponding to the text to be processed.

[0123] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0124] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0125] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0126] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0127] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0128] According to one or more embodiments of this disclosure, [Example 1] provides a text processing method, the method comprising:

[0129] Acquire the first image containing the text to be processed;

[0130] The model is determined by training the target stroke order by combining spatial attention and channel attention mechanisms.

[0131] The first image is input into a pre-trained target stroke order determination model to obtain the target stroke order corresponding to the text to be processed.

[0132] According to one or more embodiments of this disclosure, [Example 2] provides a text processing method, which further includes:

[0133] Optionally, at least one first training sample is obtained; wherein the first training sample includes a sample text image and the theoretical stroke order of the text corresponding to the sample text image;

[0134] For each first training sample, the sample text image in the current first training sample is input into the stroke order determination model to be trained, and the predicted stroke order is obtained;

[0135] Based on the predicted stroke order and the theoretical stroke order in the current first training sample, a loss value is determined, and the model parameters of the stroke order to be trained are corrected based on the loss value.

[0136] The convergence of the loss function in the stroke order determination model to be trained is taken as the training objective to obtain the target stroke order determination model.

[0137] According to one or more embodiments of this disclosure, [Example 3] provides a text processing method, which further includes:

[0138] Optionally, the sample text image is input into a convolutional layer to obtain the first feature to be processed;

[0139] The first feature to be processed is extracted using the channel attention mechanism and the spatial attention mechanism to obtain the second feature to be processed.

[0140] The second feature to be processed is fed into a recurrent neural network unit to obtain a feature sequence corresponding to each stroke order position;

[0141] The predicted stroke order is obtained by processing each feature sequence using a classifier.

[0142] According to one or more embodiments of this disclosure, [Example 4] provides a text processing method, which further includes:

[0143] Optionally, the target stroke order determination model can be used as the loss model of the style feature fusion model to be trained, so as to train the target style feature fusion model.

[0144] The target style feature fusion model is used to fuse at least two font styles.

[0145] According to one or more embodiments of this disclosure, [Example 5] provides a text processing method, which further includes:

[0146] Optionally, at least one training sample is determined; wherein the training sample includes a text image to be trained and a reference text image;

[0147] For each training sample, the text image to be processed and the reference text image in the current training sample are input into the style feature fusion model to be trained to obtain the actual output text image corresponding to the text image to be processed.

[0148] Based on the target stroke order determination model, stroke loss processing is performed on the actual output text image and the text image to be processed to obtain a first loss value;

[0149] The actual output text image and the text image to be trained are determined based on the reconstruction loss function, and the reconstruction loss is determined accordingly.

[0150] The style loss value between the actual output text image and the fused text image is determined based on the style coding loss function; wherein, the fused text image is determined based on the font style of the text image to be trained and the reference text image;

[0151] The model parameters in the style fusion model to be trained are corrected based on the first loss value, reconstruction loss, and style loss;

[0152] The convergence of the loss function in the style feature fusion model to be trained is taken as the training objective, and the target style feature fusion model is trained.

[0153] According to one or more embodiments of this disclosure, [Example Six] provides a text processing method, which further includes:

[0154] Optionally, the target style feature fusion model includes a style feature extraction sub-model, a stroke feature extraction sub-model, a content extraction sub-model, and an encoding sub-model;

[0155] The style feature extraction sub-model is used to extract the reference style of the reference text image.

[0156] The stroke feature extraction sub-model is used to extract the stroke features of the text to be processed;

[0157] The content extraction sub-model is used to extract the content features of the text to be trained; wherein, the content features include text content and the style of the text to be processed;

[0158] The encoding sub-model is used to encode the reference font style, stroke features, and image features to obtain the actual output text image.

[0159] According to one or more embodiments of this disclosure, [Example Seven] provides a text processing method, which further includes:

[0160] Optionally, a target reference style text image and a target style-transformed text image are received;

[0161] Based on the text content and converted text style of the target style-transformed text image, and the reference text style of the target reference style text image, at least one display text image is output to determine the target display text image based on a trigger operation.

[0162] According to one or more embodiments of this disclosure, [Example Eight] provides a text processing method, which further includes:

[0163] Optionally, based on the target style feature fusion model corresponding to the target displayed text image, text editing can be performed in real time, or a text package corresponding to the target displayed text image can be generated.

[0164] According to one or more embodiments of this disclosure, [Example Nine] provides a word processing apparatus, the apparatus comprising:

[0165] The first image acquisition module is used to acquire a first image including the text to be processed;

[0166] The stroke order determination model training module is used to train the target stroke order determination model by combining spatial attention mechanism and channel attention mechanism.

[0167] The target stroke order determination module is used to input the first image into a pre-trained target stroke order determination model to obtain the target stroke order corresponding to the text to be processed.

[0168] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0169] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0170] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A text processing method, characterized in that, include: Acquire the first image containing the text to be processed; The model is determined by training the target stroke order by combining spatial attention and channel attention mechanisms. The first image is input into a pre-trained target stroke order determination model to obtain the target stroke order corresponding to the text to be processed. The method further includes: using the target stroke order determination model as the loss model of the style feature fusion model to be trained, so as to train the target style feature fusion model; The target style feature fusion model is used to fuse at least two font styles. The target style feature fusion model is determined based on the first loss value obtained by the target stroke order determination model through stroke loss processing, the reconstruction loss determined by the reconstruction loss function, and the style loss value determined by the style encoding loss function.

2. The method according to claim 1, characterized in that, Also includes: Obtain at least one first training sample; wherein the first training sample includes a sample text image and the theoretical stroke order of the text corresponding to the sample text image; For each first training sample, the sample text image in the current first training sample is input into the stroke order determination model to be trained, and the predicted stroke order is obtained; Based on the predicted stroke order and the theoretical stroke order in the current first training sample, a loss value is determined, and the model parameters of the stroke order to be trained are corrected based on the loss value. The convergence of the loss function in the stroke order determination model to be trained is taken as the training objective to obtain the target stroke order determination model.

3. The method according to claim 2, characterized in that, The step of inputting the sample text image from the current first training sample into the stroke order determination model to obtain the predicted stroke order includes: The sample text image is input into a convolutional layer to obtain the first feature to be processed; The first feature to be processed is extracted using the channel attention mechanism and the spatial attention mechanism to obtain the second feature to be processed. The second feature to be processed is fed into a recurrent neural network unit to obtain a feature sequence corresponding to each stroke order position; The predicted stroke order is obtained by processing each feature sequence using a classifier.

4. The method according to claim 1, characterized in that, The target style feature fusion model obtained through training includes: Determine at least one training sample; wherein the training sample includes a text image to be trained and a reference text image; For each training sample, the text image to be processed and the reference text image in the current training sample are input into the style feature fusion model to be trained to obtain the actual output text image corresponding to the text image to be processed. Based on the target stroke order determination model, stroke loss processing is performed on the actual output text image and the text image to be processed to obtain a first loss value; The actual output text image and the text image to be trained are determined based on the reconstruction loss function, and the reconstruction loss is determined accordingly. The style loss value between the actual output text image and the fused text image is determined based on the style coding loss function; wherein, the fused text image is determined based on the font style of the text image to be trained and the reference text image; The model parameters in the style feature fusion model to be trained are corrected based on the first loss value, reconstruction loss, and style loss. The convergence of the loss function in the style feature fusion model to be trained is taken as the training objective, and the target style feature fusion model is trained.

5. The method according to claim 4, characterized in that, The target style feature fusion model includes a style feature extraction sub-model, a stroke feature extraction sub-model, a content extraction sub-model, and an encoding sub-model. The style feature extraction sub-model is used to extract the reference style of the reference text image. The stroke feature extraction sub-model is used to extract the stroke features of the text to be processed; The content extraction sub-model is used to extract the content features of the text to be trained; wherein, the content features include text content and the style of the text to be processed; The encoding sub-model is used to encode the reference style, stroke features, and content features to obtain the actual output text image.

6. The method according to claim 1, characterized in that, Also includes: Receive the target reference style text image and the target style-transformed text image; Based on the text content and converted text style of the target style-transformed text image, and the reference text style of the target reference style text image, at least one display text image is output to determine the target display text image based on a trigger operation.

7. The method according to claim 6, characterized in that, Also includes: Based on the target style feature fusion model corresponding to the target displayed text image, text editing is performed in real time, or a text package corresponding to the target displayed text image is generated.

8. A word processing device, characterized in that, include: The first image acquisition module is used to acquire a first image including the text to be processed; The stroke order determination model training module is used to train the target stroke order determination model by combining spatial attention mechanism and channel attention mechanism. The target stroke order determination module is used to input the first image into a pre-trained target stroke order determination model to obtain the target stroke order corresponding to the text to be processed. The device further includes: using the target stroke order determination model as a loss model for the style feature fusion model to be trained, so as to train the target style feature fusion model; The target style feature fusion model is used to fuse at least two font styles. The target style feature fusion model is determined based on the first loss value obtained by the target stroke order determination model through stroke loss processing, the reconstruction loss determined by the reconstruction loss function, and the style loss value determined by the style encoding loss function.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the text processing method as described in any one of claims 1-7.

10. A storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to perform the text processing method as described in any one of claims 1-7.

Images