A text processing method, device, equipment and storage medium

Abstract

The present disclosure relates to a text processing method and device, equipment and storage medium, and belongs to the technical field of computer, which can improve the accuracy of text processing. The text processing method comprises: obtaining a to-be-modified text; determining a target text according to the to-be-modified text, a preset number of words and a pre-trained text modification model; the number of words of the target text is the preset number of words; the text modification model is obtained according to a sample text pair and a sample number; the sample text pair comprises a to-be-modified sample text and a modified sample text; and the sample number is used to represent the remaining number of words after the byte corresponding to the sample number in the modified sample text.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a text processing method, apparatus, device, and storage medium. Background Technology

[0002] With the rapid development of internet technology, people are increasingly using the internet to find the information they need. Correspondingly, content community applications (APPs) also recommend relevant keywords to users in the display location of recommended keywords (such as the recommended keyword display box on the search page).

[0003] However, due to the limited display space for recommended words, only a fixed number of words can be displayed. When the text length of a recommended word is long, the display space cannot fully display the word, thus degrading the user experience. Summary of the Invention

[0004] This disclosure provides a text processing method, apparatus, device, and storage medium that can improve the accuracy of text processing.

[0005] The technical solution of this disclosure is as follows:

[0006] According to a first aspect of the present disclosure, a text processing method is provided, which can be applied to an electronic device. The method may include:

[0007] Get the text to be modified;

[0008] The target text is determined based on the text to be modified, the preset number of characters, and the pre-trained text modification model. The number of characters in the target text is the preset number of characters. The text modification model is trained based on sample text pairs and sample numbers. The sample text pairs include: the sample text to be modified and the modified sample text. The sample numbers are used to represent the number of characters remaining after the byte corresponding to the sample number in the modified sample text.

[0009] Optionally, text processing methods also include:

[0010] Obtain sample text pairs and sample numbers;

[0011] The text modification model is obtained by training on sample text pairs and sample numbers based on a preset model; the preset model includes: Long Short-Term Memory (LSTM) network model.

[0012] Optionally, the target text is determined based on the text to be modified, the preset number of words, and a pre-trained text modification model, including:

[0013] The text to be modified and the preset number of characters are input into the encoder of the text modification model to obtain a set of text vectors of the text to be modified and a set of numerical codes of the preset number of characters; the set of text vectors includes the byte vector of each byte in the text to be modified; the set of numerical codes includes the numerical code of each number in the preset number of characters.

[0014] The text vector set and the number encoding set are input into the decoder of the text modification model to obtain the target text.

[0015] Optionally, the text to be modified and a preset number of characters are input into the encoder of the text modification model to obtain a set of text vectors for the text to be modified and a set of numerical codes for the preset number of characters, including:

[0016] The first byte of the text to be modified is input into the first encoder of the text modification model to obtain the byte vector of the first byte;

[0017] For the text to be modified, the first operation is performed on all bytes except the first byte to obtain the byte vectors of the other bytes. The first operation includes: inputting the target byte of the text to be modified and the first byte vector into the target encoder of the text modification model to obtain the byte vector of the target byte; the first byte vector is the byte vector of the byte preceding the target byte; the target encoder is the encoder corresponding to the target byte; the target byte is any one of the other bytes.

[0018] The set of text vectors for determining the text to be modified includes the byte vector of the first byte and the byte vectors of the other bytes;

[0019] The preset number of characters is input into the second encoder of the text modification model to obtain a set of numerical codes.

[0020] Optionally, the set of text vectors and the set of numeric codes are input into the decoder of the text modification model to obtain the target text, including:

[0021] The end marker byte of the text to be modified, the numeric code of the largest number in the numeric code set, and the byte vector of the first byte in the text vector set are input into the first decoder of the text modification model to obtain the first byte of the target text.

[0022] For all byte vectors in the text vector set except for the first byte, and for all numeric codes in the numeric code set except for the largest number, the second operation is performed to obtain the other bytes of the target text. The second operation includes: inputting the numeric code of the target number in the preset number of characters, the byte vector of the target byte in the text vector set, and the byte preceding the target byte into the target decoder of the text modification model to obtain the target byte. The target number is used to represent the number of remaining characters after the target byte in the target text. The target decoder is the decoder corresponding to the target byte.

[0023] The target text is determined to include the first byte and the other bytes.

[0024] Optionally, when the target number is the last number of the preset number of characters, the target byte is the end marker byte of the target text.

[0025] According to a second aspect of the present disclosure, a text processing apparatus is provided, the method of which can be applied to an electronic device. The apparatus may include: an acquisition unit and a processing unit;

[0026] The acquisition unit is used to acquire the text to be modified.

[0027] The processing unit is used to determine the target text based on the text to be modified, the preset number of characters, and the pre-trained text modification model. The number of characters in the target text is the preset number of characters. The text modification model is trained based on sample text pairs and sample numbers. The sample text pairs include: the sample text to be modified and the modified sample text. The sample numbers are used to represent the number of characters remaining after the byte corresponding to the sample number in the modified sample text.

[0028] Optionally, the acquisition unit can also be used to acquire sample text pairs and sample numbers;

[0029] The processing unit is also used to train sample text pairs and sample numbers based on a preset model to obtain a text modification model; the preset model includes: a Long Short-Term Memory (LSTM) network model.

[0030] Optional, processing unit, specifically used for:

[0031] The text to be modified and the preset number of characters are input into the encoder of the text modification model to obtain a set of text vectors of the text to be modified and a set of numerical codes of the preset number of characters; the set of text vectors includes the byte vector of each byte in the text to be modified; the set of numerical codes includes the numerical code of each number in the preset number of characters.

[0032] The text vector set and the number encoding set are input into the decoder of the text modification model to obtain the target text.

[0033] Optional, processing unit, specifically used for:

[0034] The first byte of the text to be modified is input into the first encoder of the text modification model to obtain the byte vector of the first byte;

[0035] For the text to be modified, the first operation is performed on all bytes except the first byte to obtain the byte vectors of the other bytes. The first operation includes: inputting the target byte of the text to be modified and the first byte vector into the target encoder of the text modification model to obtain the byte vector of the target byte; the first byte vector is the byte vector of the byte preceding the target byte; the target encoder is the encoder corresponding to the target byte; the target byte is any one of the other bytes.

[0036] The set of text vectors for determining the text to be modified includes the byte vector of the first byte and the byte vectors of the other bytes;

[0037] The preset number of characters is input into the second encoder of the text modification model to obtain a set of numerical codes.

[0038] Optional, processing unit, specifically used for:

[0039] The end marker byte of the text to be modified, the numeric code of the largest number in the numeric code set, and the byte vector of the first byte in the text vector set are input into the first decoder of the text modification model to obtain the first byte of the target text.

[0040] For all byte vectors in the text vector set except for the first byte, and for all numeric codes in the numeric code set except for the largest number, the second operation is performed to obtain the other bytes of the target text. The second operation includes: inputting the numeric code of the target number in the preset number of characters, the byte vector of the target byte in the text vector set, and the byte preceding the target byte into the target decoder of the text modification model to obtain the target byte. The target number is used to represent the number of remaining characters after the target byte in the target text. The target decoder is the decoder corresponding to the target byte.

[0041] The target text is determined to include the first byte and the other bytes.

[0042] Optionally, when the target number is the last number of the preset number of characters, the target byte is the end marker byte of the target text.

[0043] According to a third aspect of the present disclosure, an electronic device is provided, which may include: a processor and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement any of the optional text processing methods of the first aspect described above.

[0044] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which instructions are stored, such that when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform any of the optional text processing methods of the first aspect described above.

[0045] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product including computer instructions that, when executed on a processor of an electronic device, cause the processor of the electronic device to perform a text processing method as described in any of the optional implementations of the first aspect.

[0046] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0047] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0048] Based on any of the above aspects, in this disclosure, after acquiring the text to be modified, the electronic device can determine the target text based on the text to be modified, a preset number of characters, and a pre-trained text modification model. The target text has a preset number of characters; the text modification model is trained based on sample text pairs and sample numbers; the sample text pairs include the sample text to be modified and the modified sample text; the sample numbers represent the number of remaining characters after the byte corresponding to the sample number in the modified sample text.

[0049] In this way, electronic devices can modify the text to be modified using a text modification model, and then add a preset number of characters to the text modification model to obtain the target text with the preset number of characters. This ensures the correct number of characters in the text processing and improves the accuracy and efficiency of text processing. Subsequently, the target text with the preset number of characters can be displayed at the target text display location, ensuring the completeness of the target text display and thus enriching the user experience. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0051] Figure 1 A flowchart of a text processing system provided in an embodiment of this disclosure is shown;

[0052] Figure 2 A flowchart illustrating a text processing method provided in an embodiment of this disclosure is shown.

[0053] Figure 3 A flowchart illustrating yet another text processing method provided in an embodiment of this disclosure is shown;

[0054] Figure 4 A flowchart illustrating yet another text processing method provided in an embodiment of this disclosure is shown;

[0055] Figure 5 A flowchart illustrating yet another text processing method provided in an embodiment of this disclosure is shown;

[0056] Figure 6 A flowchart illustrating yet another text processing method provided in an embodiment of this disclosure is shown;

[0057] Figure 7 A flowchart illustrating yet another text processing method provided in an embodiment of this disclosure is shown;

[0058] Figure 8 A schematic diagram of the structure of a text processing apparatus provided in an embodiment of this disclosure is shown;

[0059] Figure 9 A schematic diagram of the structure of a terminal provided in an embodiment of this disclosure is shown;

[0060] Figure 10 A schematic diagram of the structure of a server provided in an embodiment of this disclosure is shown. Detailed Implementation

[0061] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0062] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0063] It should also be understood that the term "comprising" indicates the presence of the described feature, whole, step, operation, element and / or component, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements and / or components.

[0064] The data disclosed herein may be data authorized by the user or fully authorized by all parties.

[0065] With the rapid development of internet technology, people are increasingly using the internet to find the information they need. Correspondingly, content community applications (APPs) also recommend relevant keywords to users in the display location of recommended keywords (such as the recommended keyword display box on the search page).

[0066] However, due to the limited display space for recommended words, only a fixed number of words can be displayed. When the text length of a recommended word is long, the display space cannot fully display the word, thus degrading the user experience.

[0067] In general technology, after obtaining recommended keywords, they are usually manually rewritten to a preset character limit to ensure they are displayed fully in their designated positions. However, when the number of recommended keywords is large, manual rewriting becomes time-consuming and costly. Furthermore, when recommended keywords are trending news items, timeliness is crucial. Manual rewriting cannot guarantee this.

[0068] With the development of neural network technology, general techniques can also modify recommended words through neural network models. However, existing neural network models typically modify the semantics of recommended words, and cannot guarantee the word count of the modified recommended words.

[0069] Based on this, embodiments of this disclosure provide a text processing method. After acquiring the text to be modified, an electronic device can determine the target text based on the text to be modified, a preset number of characters, and a pre-trained text modification model. The target text has a preset number of characters; the text modification model is trained based on sample text pairs and sample numbers; the sample text pairs include the sample text to be modified and the modified sample text; the sample numbers represent the number of characters remaining after the byte corresponding to the sample number in the modified sample text.

[0070] In this way, electronic devices can modify the text to be modified using a text modification model, and then add a preset number of characters to the text modification model to obtain the target text with the preset number of characters. This ensures the correct number of characters in the text processing and improves the accuracy and efficiency of text processing. Subsequently, the target text with the preset number of characters can be displayed at the target text display location, ensuring the completeness of the target text display and thus enriching the user experience.

[0071] Figure 1 This is a schematic diagram of a text processing system provided in an embodiment of the present disclosure, such as... Figure 1As shown, the text processing system may include a server 110 and an electronic device 120. The server 110 can establish a connection with the electronic device 120 through a wired network or a wireless network.

[0072] Server 110 can be a data server for multimedia resource service platforms, used for storing and processing multimedia resources. For example, a multimedia resource service platform could be a short video application service platform, a news service platform, a live streaming service platform, a shopping service platform, a food delivery service platform, a sharing service platform, or a functional website. The multimedia resources provided by a short video application service platform can be short video works, the multimedia resources provided by a news service platform can be news information, and the multimedia resources provided by a live streaming service platform can be live broadcasts, etc., and so on. This disclosure does not limit the specific type of multimedia resource service platform.

[0073] In this disclosure, server 110 is mainly used to store text to be processed, such as trending keywords and topic-related keywords. When server 110 receives a data acquisition request from electronic device 120, it can send the corresponding text to be processed to electronic device 120.

[0074] In some embodiments, server 110 may also include or be connected to a database, and the text to be processed by the multimedia resource service platform may be stored in the database. Electronic device 120 can access the text to be processed in the database through server 110.

[0075] Electronic device 120 may be a server, a terminal, or other electronic device used for text processing; this disclosure does not limit it in this regard.

[0076] When electronic device 120 is a server, electronic device 120 and server 110 can be two independent servers or integrated into the same server. This application does not make specific limitations in this regard.

[0077] It is easy to understand that when electronic device 120 and server 110 are integrated into the same server, the communication method between electronic device 120 and server 110 is the same as the communication between internal modules of the server. In this case, the communication process between the two is the same as that between electronic device 120 and server 110 when they are independent of each other.

[0078] For ease of understanding, this application mainly uses the independent configuration of electronic device 120 and server 110 as an example for illustration.

[0079] The aforementioned server can be a single server, or it can be a server cluster consisting of multiple servers. In some embodiments, the server cluster can also be a distributed cluster. This disclosure does not limit the specific implementation method of the server.

[0080] When electronic device 120 is used as a terminal, it can be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, etc., which can install and use content community applications (such as Kuaishou). This disclosure does not impose any special restrictions on the specific form of the terminal. It can interact with users through one or more methods such as keyboard, touchpad, touch screen, remote control, voice interaction, or handwriting device.

[0081] Optionally, the above Figure 1 In the text processing system shown, electronic device 120 can be connected to at least one server 110. This disclosure does not limit the number or type of servers 110.

[0082] The text processing method provided in this disclosure can be applied to the aforementioned... Figure 1 Electronic device 120 in the application scenario shown.

[0083] The text processing method provided in the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0084] like Figure 2 As shown, when a text processing method is applied to an electronic device, the text processing method may include:

[0085] S201. The electronic device acquires the text to be modified.

[0086] In some embodiments, the text to be modified may be a news hot topic recommendation, a user interest recommendation determined based on user profiles, or other types of recommendation. This disclosure does not limit this.

[0087] In one possible implementation, when an electronic device obtains text to be modified, it can send a request message to a server storing the text to be modified after receiving a processing instruction (e.g., a text processing instruction sent by a user to the electronic device). Correspondingly, upon receiving the request message, the server can retrieve the text to be modified from a database or other storage platform and send it to the electronic device. Thus, the electronic device can obtain the text to be modified.

[0088] In another possible implementation, the server can proactively send the text to be modified to the electronic device after receiving a processing instruction (such as a text processing instruction sent by a user to an electronic device), so that the electronic device can process the text to be modified.

[0089] S202. The electronic device determines the target text based on the text to be modified, the preset number of characters, and the pre-trained text modification model.

[0090] Specifically, after obtaining the text to be modified, in order to ensure that the text processing obtains the target text's word count, the electronic device can determine the target text based on the text to be modified, the preset word count, and the pre-trained text modification model.

[0091] The target text has a preset number of characters; the text modification model is trained based on sample text pairs and sample numbers; the sample text pairs include: the sample text to be modified and the modified sample text; the sample numbers are used to represent the number of characters remaining after the byte corresponding to the sample number in the modified sample text.

[0092] In one possible approach, the preset word count can be determined based on the size of the display box for the target text.

[0093] For example, the target text might be a recommended term from a content community application. This application can display this target text in the recommended term display box on the search page. This recommended term display box typically shows 5 bytes of recommended terms. In this case, the electronic device can determine that the preset character count is 5.

[0094] In one feasible approach, electronic devices can train text modification models based on different neural network models.

[0095] In some examples, the neural network model described above may be a Long Short-Term Memory (LSTM) network model, a seq2seq model based on LSTM, a Recurrent Neural Network (RNN) model, or other types of neural network models, which are not limited in this disclosure.

[0096] In one possible implementation, the sample text pair includes the sample text to be modified and the modified sample text as corresponding samples.

[0097] In some examples, the sample text to be modified could be "Team 1 won the championship today." Correspondingly, the modified sample text could be "Team 1 won the championship today." Thus, the modified sample text is more concise and has fewer words than the sample text to be modified. By training on sample text pairs including both the sample text to be modified and the modified sample text, the target text output by the subsequent text modification model can be made more concise and more accurate.

[0098] Furthermore, to ensure the accurate word count of the target text obtained through text processing, this application includes sample numbers in the training samples of the text modification model. These sample numbers represent the remaining word count after the byte corresponding to the sample number in the modified sample text.

[0099] Based on the above example, if the sample text to be modified is "The first team won the championship today.", and the modified sample text can be "The first team won the championship today", the sample number can include: the numerical code of the remaining number of characters after each byte in the modified sample text "The first team won the championship today". This sample number is shown in Table 1.

[0100] Table 1

[0101]

[0102] The technical solution provided by the above embodiments brings at least the following beneficial effects: As shown in S201-S202, after acquiring the text to be modified, the electronic device can determine the target text based on the text to be modified, a preset number of characters, and a pre-trained text modification model. The target text has a preset number of characters; the text modification model is trained based on sample text pairs and sample numbers; the sample text pairs include the sample text to be modified and the modified sample text; the sample numbers represent the number of remaining characters after the byte corresponding to the sample number in the modified sample text.

[0103] In this way, electronic devices can modify the text to be modified using a text modification model, and then add a preset number of characters to the text modification model to obtain the target text with the preset number of characters. This ensures the correct number of characters in the text processing and improves the accuracy and efficiency of text processing. Subsequently, the target text with the preset number of characters can be displayed at the target text display location, ensuring the completeness of the target text display and thus enriching the user experience.

[0104] In one embodiment, such as Figure 3 As shown, the text processing method provided in this disclosure also includes:

[0105] S301. Electronic device acquires sample text pairs and sample numbers.

[0106] In one possible implementation, when an electronic device acquires sample text pairs and sample numbers, it can send a request message to a server storing the sample text pairs and sample numbers after receiving a model training instruction (e.g., a model training instruction sent by a user to the electronic device). Correspondingly, upon receiving the request message, the server can retrieve the sample text pairs and sample numbers from a database or other storage platform and send them to the electronic device. Thus, the electronic device can acquire the sample text pairs and sample numbers.

[0107] In another possible implementation, after receiving a model training instruction (e.g., a model training instruction sent by a user to an electronic device), the server can proactively send sample text pairs and sample numbers to the electronic device, so that the electronic device can train the model based on the sample text pairs and sample numbers.

[0108] For a description of the sample text pairs and sample numbers, please refer to the relevant description in S202 above, which will not be repeated here.

[0109] S302. The electronic device trains a text modification model on sample text pairs and sample numbers based on a preset model.

[0110] Specifically, after obtaining sample text pairs and sample numbers, the electronic device can train the sample text pairs and sample numbers based on a preset model to obtain a text modification model, so as to determine the target text with a preset number of characters based on the text modification model.

[0111] The preset models include: LSTM model.

[0112] In some embodiments, the preset model may be an LSTM-based seq2seq model, an RNN model, or other types of neural network models, and this disclosure does not limit it.

[0113] The aforementioned model can be used for machine translation, text summarization, conversation modeling, image captioning, etc. In this disclosure, an electronic device can train a text modification model based on a preset model using sample text pairs and sample numbers.

[0114] In one feasible approach, the electronic device can first convert sample text pairs into text vectors and sample numbers into sample number codes, and then train the text vectors and sample number codes of the sample text pairs based on a preset model to obtain a text modification model.

[0115] Optionally, the content of the text vector can be a specific vector or matrix, etc. The content of the sample numerical encoding can be binary encoding, etc., and this disclosure does not limit it.

[0116] In another possible approach, the electronic device trains a model based on a preset model for sample text pairs and sample numbers. This process can be referenced in the model training process of general technologies and will not be elaborated here.

[0117] The technical solution provided by the above embodiments brings at least the following beneficial effects: As shown in S301-S302, a specific implementation method is given for an electronic device to train a text modification model based on a preset model using sample text pairs and sample numbers. When the electronic device trains the text modification model, it can add sample numbers to the training sample data summary, thereby training a text modification model that can output a preset number of characters, so as to facilitate the subsequent output of target text with a preset number of characters, thus improving the accuracy and efficiency of text processing.

[0118] In one embodiment, combined with Figure 2 ,like Figure 4 As shown, in S202 above, the method by which the electronic device determines the target text based on the text to be modified, the preset number of characters, and the pre-trained text modification model specifically includes:

[0119] S401. The electronic device inputs the text to be modified and the preset number of characters into the encoder of the text modification model to obtain the text vector set of the text to be modified and the digital code set of the preset number of characters.

[0120] The text vector set includes the byte vector of each byte in the text to be modified; the number encoding set includes the number encoding of each digit in the preset number of characters.

[0121] The encoder in the text modification model is used to encode the acquired data (data types that machines cannot recognize but users can) to obtain data of a machine-recognizable type.

[0122] In this disclosure, the encoder of the text modification model can encode the text to be modified into a set of text vectors, and encode a preset number of characters into a set of numeric codes.

[0123] Optionally, in the text vector set, the content format of each byte vector can be a specific vector or matrix, etc. In the number encoding set, the content format of each number encoding can be binary encoding, etc., and this disclosure does not limit it.

[0124] S402, The electronic device inputs the text vector set and the digital code set into the decoder of the text modification model to obtain the target text.

[0125] The decoder in the text modification model is used to decode the encoded data (a data type that the machine can recognize) to obtain data of a data type that the user can recognize.

[0126] In this disclosure, the decoder of the text modification model can decode the data after processing the set of text vectors and the set of numerical codes into the target text.

[0127] The technical solution provided by the above embodiments brings at least the following beneficial effects: As shown in S401-S402, a method is provided to determine the target text based on the text to be modified, a preset number of characters, and a pre-trained text modification model. The electronic device can obtain the target text with the preset number of characters through the text modification model, ensuring the number of characters processed and improving the accuracy and efficiency of text processing.

[0128] In one embodiment, combined with Figure 4 ,like Figure 5 As shown, in step S401 above, the method by which the electronic device inputs the text to be modified and a preset number of characters into the encoder of the text modification model to obtain the text vector set of the text to be modified and the digital code set of the preset number of characters specifically includes:

[0129] S501, The electronic device inputs the first byte of the text to be modified into the first encoder of the text modification model to obtain the byte vector of the first byte.

[0130] In one possible implementation, the text modification model can include multiple encoders with the same number of bytes as the text to be modified, with different encoders used to encode different bytes. After acquiring the text to be modified, the electronic device can input the first byte of the text to be modified into the first encoder of the text modification model to obtain the byte vector of the first byte.

[0131] The first encoder is an encoder used to encode the first byte of the text to be modified.

[0132] In another implementable manner, the text modification model may only include one encoder. In this case, the electronic device may input the first byte of the text to be modified into this one encoder to obtain the byte vector of the first byte.

[0133] In another implementable manner, the text modification model may include multiple encoders whose quantity is less than the number of bytes in the text to be modified. One encoder is used to encode one or more bytes. That is, when one encoder is used to encode multiple bytes, at least two different bytes may be encoded by one encoder. In this case, the electronic device may input the first byte of the text to be modified into the encoder corresponding to the first byte to obtain the byte vector of the first byte.

[0134] S502. The electronic device performs a first operation on all bytes other than the first byte in the text to be modified to obtain the byte vectors of the other bytes.

[0135] Among them, the first operation includes: inputting the target byte of the text to be modified and the first byte vector into the target encoder of the text modification model to obtain the byte vector of the target byte; the first byte vector is the byte vector of the previous byte adjacent to the target byte; the target encoder is the encoder corresponding to the target byte; the target byte is any one of the other bytes.

[0136] Specifically, after obtaining the byte vector of the first byte, the electronic device inputs the second byte in the text to be modified and the byte vector of the first byte into the second encoder of the text modification model to obtain the byte vector of the second byte.

[0137] Based on the same method as above, the byte vectors of the third byte, the fourth byte, etc. are obtained in sequence until the byte vectors of each byte among the other bytes except the first byte are obtained.

[0138] Exemplarily, it is preset that the text to be modified is "The first team won the championship today.", and the electronic device may input the first byte "第" of the text to be modified into the first encoder of the text modification model to obtain the byte "vector A" of the first byte.

[0139] Then, the electronic device inputs the second byte "一" in the text to be modified and the byte vector "vector A" of the first byte into the second encoder of the text modification model to obtain the byte vector "vector B" of the second byte.

[0140] Then, the electronic device inputs the third byte "球" in the text to be modified and the byte vector "vector B" of the second byte into the third encoder of the text modification model to obtain the byte vector "vector C" of the third byte.

[0141] Using the same method described above, the byte vector of each of the remaining bytes is obtained sequentially. Table 2 shows each byte in the text to be modified and its corresponding byte vector.

[0142] Table 2

[0143]

[0144] S503, The electronic device determines that the set of text vectors of the text to be modified includes the byte vector of the first byte and the byte vectors of the other bytes.

[0145] Specifically, after obtaining the byte vector of the first byte and the byte vectors of the other bytes, the electronic device can determine that the set of text vectors of the text to be modified includes the byte vector of the first byte and the byte vectors of the other bytes.

[0146] Based on Table 2 above, the set of text vectors for the text to be modified can include vectors A through N.

[0147] S504. The electronic device inputs a preset number of characters into the second encoder of the text modification model to obtain a set of digital codes.

[0148] In one possible implementation, the text modification model may further include a second encoder for encoding a preset number of characters. After obtaining the preset number of characters, the electronic device can input each digit of the preset number of characters into the second encoder of the text modification model to obtain a set of digit codes.

[0149] In one possible implementation, the text modification model can include multiple second encoders, the same number of digits as the preset number of characters, with different second encoders used to encode different digits. After obtaining the preset number of characters, the electronic device can input each digit in the preset number of characters into the corresponding second encoder to obtain a set of digit codes.

[0150] In another possible implementation, the text modification model can consist of only a second encoder. In this case, the electronic device can input each digit from a preset number of words into this second encoder to obtain a set of digit codes.

[0151] In another possible implementation, the text modification model can include multiple second encoders, fewer than the number of digits in a preset word count. One second encoder is used to encode one or more word counts. That is, when one second encoder is used to encode multiple digits, at least two distinct digits can be encoded by that single second encoder. In this case, the electronic device can input each digit in the preset word count into its corresponding second encoder to obtain a set of digit codes.

[0152] In practical applications, the first encoder and the second encoder can be the same encoder or different encoders, and this disclosure does not limit them.

[0153] For example, the preset number of characters is "5", and the second encoder is used to encode the decimal number into a binary number. In this case, the electronic device can input the first digit "5" of the preset number of characters into the second encoder to obtain the first digit "5" as the digital code "101".

[0154] Next, the electronic device inputs the second number "4" (preset number of characters) into the second encoder to obtain the digital code "100" for the second number "4".

[0155] Using the same method described above, the numerical code for each of the remaining numbers is obtained sequentially. Table 3 shows each number in the preset number of characters and its corresponding numerical code.

[0156] Table 3

[0157]

[0158] The technical solution provided by the above embodiments brings at least the following beneficial effects: As shown in S501-S504, a specific implementation method is given for an electronic device to input the text to be modified and a preset number of characters into the encoder of a text modification model to obtain a set of text vectors of the text to be modified and a set of digital codes for the preset number of characters. The electronic device can encode the text to be modified and the preset number of characters into a machine-recognizable data type, namely, a set of text vectors of the text to be modified and a set of digital codes for the preset number of characters, so that the electronic device can quickly and accurately determine the target text based on the set of text vectors of the text to be modified and the set of digital codes for the preset number of characters, thereby improving the efficiency of text processing.

[0159] In one feasible example, combining Figure 5 ,like Figure 6 As shown, in S402 above, the electronic device inputs the text vector set and the digital encoding set into the decoder of the text modification model to obtain the target text, including:

[0160] S601, The electronic device inputs the end identifier byte of the text to be modified, the numeric code of the largest number in the numeric code set, and the byte vector of the first byte in the text vector set into the first decoder of the text modification model to obtain the first byte of the target text.

[0161] It should be noted that the End Of Sentence (EOS) byte of the text to be modified is used to indicate that the encoding process of the text to be modified has ended. After obtaining the EOS byte of the text to be modified, the electronic device can respond to the EOS byte and begin decoding the text vector set and the number encoding set to obtain the target text.

[0162] In one possible implementation, the text modification model can include multiple decoders equal to the preset number of characters, with different decoders used to encode different byte vectors and numeric codes. After obtaining the set of text vectors and the set of numeric codes, the electronic device can input the end-of-text marker byte, the numeric code of the largest number in the numeric code set, and the byte vector of the first byte in the text vector set into the first decoder of the text modification model to obtain the first byte of the target text.

[0163] The first decoder is used to decode the numeric code of the largest number in the numeric code set and the byte vector of the first byte in the text vector set.

[0164] In another possible implementation, the text modification model can include only one decoder. In this case, the electronic device can input the end-of-text marker byte of the text to be modified, the numeric code of the largest number in the numeric code set, and the byte vector of the first byte in the text vector set into this single decoder to obtain the first byte of the target text.

[0165] In another possible implementation, the text modification model can include multiple decoders, fewer than the number of bytes in a preset word count. One decoder is used to decode one or more byte vectors and one or more numeric codes. That is, when one decoder is used to decode multiple byte vectors and multiple numeric codes, at least two different byte vectors and at least two different numeric codes can be decoded by one decoder. In this case, the electronic device can input the end-of-text marker byte of the text to be modified, the numeric code of the largest number in the numeric code set, and the byte vector of the first byte in the text vector set into the corresponding decoder to obtain the first byte of the target text.

[0166] S602. The electronic device performs a second operation on all byte vectors in the text vector set except for the first byte, and on all numeric codes in the numeric code set except for the numeric code of the largest number, to obtain the other bytes of the target text.

[0167] Among them, the second operation includes: inputting the digital code of the target digit in the digital code set, the byte vector of the target byte in the text vector set, and the previous byte adjacent to the target byte into the target decoder of the text modification model to obtain the target byte; the target digit is used to represent the remaining number of characters after the target byte in the target text; the target decoder is the decoder corresponding to the target byte.

[0168] In an implementable manner, when the target digit is the last digit of the preset number of characters (such as "0"), the target byte is the end identification byte of the target text. In this way, the electronic device can output the end identification byte to indicate that the target text has been processed. Subsequently, the electronic device can send the target text to different display platforms for display in response to the generated end identification byte.

[0169] Specifically, after obtaining the first byte of the target text, the electronic device can input the first byte of the target text, the byte vector of the second byte in the text vector, and the second digital code in the digital code set into the second decoder of the text modification model to obtain the second byte.

[0170] Based on the same method as above, the third byte, the fourth byte, etc. are obtained in sequence until each byte among the other bytes except the first byte is obtained.

[0171] Exemplarily, as Figure 7 shown, the preset text to be modified is "The first team won the championship today.", and each byte in the text to be modified and its corresponding byte vector are shown in Table 2, and each digit in the preset number of characters and its corresponding digital code are shown in Table 3. The electronic device can first input the text to be modified "The first team won the championship today." into the corresponding encoder in sequence. Then, the electronic device can input the end identification byte "." of the text to be modified, the binary digital code "101" of the largest digit "5" in the digital code set, and the byte vector "vector A" of the first byte "第" in the text vector set into the first decoder of the text modification model to obtain the first byte "第" of the target text.

[0172] Based on the same method as above, the second byte "一", the third byte "队", the fourth byte "冠", the fifth byte "军", and the end identification byte "0" are obtained in sequence, that is, the target text is "第一队冠军。".

[0173] It should be noted that when modifying the number of characters of the text to be modified, the text modification model can learn and train according to the semantics of the sample text pair, so as to obtain the target text with smooth semantics and the preset number of characters.

[0174] S603, The electronic device determines that the target text includes the first byte and other bytes.

[0175] The technical solution provided by the above embodiments brings at least the following beneficial effects: As shown in S601-S603, an electronic device is provided that inputs a set of text vectors and a set of digital codes into the decoder of a text modification model to obtain target text. The electronic device can decode the set of text vectors and the set of digital codes into a user-recognizable data type, i.e., target text, so that the electronic device can subsequently send the target text to different display platforms for display.

[0176] It is understood that, in actual implementation, the terminal / server described in the embodiments of this disclosure may include one or more hardware structures and / or software modules for implementing the aforementioned corresponding text processing methods. These hardware structures and / or software modules can constitute an electronic device. Those skilled in the art should readily recognize that, based on the algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0177] Based on this understanding, this disclosure also provides a text processing device that can be applied to electronic devices. Figure 8 A schematic diagram of the structure of a text processing apparatus provided in an embodiment of this disclosure is shown. Figure 8 As shown, the text processing device may include: an acquisition unit 801 and a processing unit 802;

[0178] Acquisition unit 801 is used to acquire the text to be modified;

[0179] The processing unit 802 is used to determine the target text based on the text to be modified, the preset number of characters, and the pre-trained text modification model; the number of characters in the target text is the preset number of characters; the text modification model is trained based on sample text pairs and sample numbers; the sample text pairs include: the sample text to be modified and the modified sample text; the sample numbers are used to represent the number of characters remaining after the byte corresponding to the sample number in the modified sample text.

[0180] Optionally, the acquisition unit 801 is also used to acquire sample text pairs and sample numbers;

[0181] The processing unit 802 is also used to train the sample text pairs and sample numbers based on a preset model to obtain a text modification model; the preset model includes: a Long Short-Term Memory (LSTM) network model.

[0182] Optionally, the processing unit 802 is specifically used for:

[0183] The text to be modified and the preset number of characters are input into the encoder of the text modification model to obtain a set of text vectors of the text to be modified and a set of numerical codes of the preset number of characters; the set of text vectors includes the byte vector of each byte in the text to be modified; the set of numerical codes includes the numerical code of each number in the preset number of characters.

[0184] The text vector set and the number encoding set are input into the decoder of the text modification model to obtain the target text.

[0185] Optionally, the processing unit 802 is specifically used for:

[0186] The first byte of the text to be modified is input into the first encoder of the text modification model to obtain the byte vector of the first byte;

[0187] For the text to be modified, the first operation is performed on all bytes except the first byte to obtain the byte vectors of the other bytes. The first operation includes: inputting the target byte of the text to be modified and the first byte vector into the target encoder of the text modification model to obtain the byte vector of the target byte; the first byte vector is the byte vector of the byte preceding the target byte; the target encoder is the encoder corresponding to the target byte; the target byte is any one of the other bytes.

[0188] The set of text vectors for determining the text to be modified includes the byte vector of the first byte and the byte vectors of the other bytes;

[0189] The preset number of characters is input into the second encoder of the text modification model to obtain a set of numerical codes.

[0190] Optionally, the processing unit 802 is specifically used for:

[0191] The end marker byte of the text to be modified, the numeric code of the largest number in the numeric code set, and the byte vector of the first byte in the text vector set are input into the first decoder of the text modification model to obtain the first byte of the target text.

[0192] For all byte vectors in the text vector set except for the first byte, and for all numeric codes in the numeric code set except for the largest number, the second operation is performed to obtain the other bytes of the target text. The second operation includes: inputting the numeric code of the target number in the preset number of characters, the byte vector of the target byte in the text vector set, and the byte preceding the target byte into the target decoder of the text modification model to obtain the target byte. The target number is used to represent the number of remaining characters after the target byte in the target text. The target decoder is the decoder corresponding to the target byte.

[0193] The target text is determined to include the first byte and the other bytes.

[0194] Optionally, when the target number is the last number of the preset number of characters, the target byte is the end marker byte of the target text.

[0195] As described above, the embodiments of this disclosure can divide the electronic device into functional modules according to the above method examples. The integrated modules can be implemented in hardware or as software functional modules. Furthermore, it should be noted that the module division in these embodiments is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into a single processing module.

[0196] Regarding the text processing device in the above embodiments, the specific methods by which each module performs operations and the beneficial effects thereof have been described in detail in the foregoing method embodiments, and will not be repeated here.

[0197] This disclosure also provides a terminal, which may be a user terminal such as a mobile phone or a computer. Figure 9 A schematic diagram of the structure of a terminal provided in an embodiment of this disclosure is shown. The terminal may be a text processing device and may include at least one processor 61, a communication bus 62, a memory 63, and at least one communication interface 64.

[0198] Processor 61 can be a central processing unit (CPU), a microprocessor unit, an ASIC, or one or more integrated circuits used to control the execution of programs according to the present disclosure. As an example, combined with... Figure 8 The processing unit 802 in the electronic device performs the functions of Figure 9 The processor 61 in it performs the same function.

[0199] The communication bus 62 may include a path for transmitting information between the aforementioned components.

[0200] Communication interface 64 uses any transceiver-like device for communicating with other devices or communication networks, such as servers, Ethernet, radio access networks (RAN), wireless local area networks (WLAN), etc. As an example,

[0201] Memory 63 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory may exist independently and be connected to the processing unit via a bus. Memory may also be integrated with the processing unit.

[0202] The memory 63 stores the application code that executes the present invention, and its execution is controlled by the processor 61. The processor 61 executes the application code stored in the memory 63 to implement the functions of the method of the present invention.

[0203] In a specific implementation, as one example, processor 61 may include one or more CPUs, for example... Figure 9 CPU0 and CPU1 in the CPU.

[0204] In a specific implementation, as one example, the terminal may include multiple processors, for example... Figure 9 Processors 61 and 65 are mentioned. Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor here may refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0205] In a specific implementation, as one embodiment, the terminal may further include an input device 66 and an output device 67. The input device 66 and the output device 67 communicate and can accept user input in various ways. For example, the input device 66 may be a mouse, keyboard, touchscreen device, or sensing device. The output device 67 communicates with the processor 61 and can display information in various ways. For example, the output device 67 may be a liquid crystal display (LCD), a light emitting diode (LED) display device, etc.

[0206] Those skilled in the art will understand that Figure 9 The structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0207] This disclosure also provides a server. Figure 10 A schematic diagram of the server structure provided in an embodiment of this disclosure is shown. This server can be a text processing device. The server can vary significantly due to differences in configuration or performance, and may include one or more processors 71 and one or more memories 72. The memory 72 stores at least one instruction, which is loaded and executed by the processor 71 to implement the text processing methods provided in the various method embodiments described above. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be elaborated here.

[0208] This disclosure also provides a computer-readable storage medium including instructions stored thereon, which, when executed by a processor of a computer device, enable the computer to perform the text processing method provided in the embodiments described above. For example, the computer-readable storage medium may be a memory 63 including instructions, which may be executed by a processor 61 of a terminal to complete the method. As another example, the computer-readable storage medium may be a memory 72 including instructions, which may be executed by a processor 71 of a server to complete the method. Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a ROM, RAM, CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0209] This disclosure also provides a computer program product including computer instructions that, when executed on an electronic device, cause the electronic device to perform the above-described actions. Figures 1-7The text processing method shown in any of the attached figures.

[0210] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0211] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A text processing method, characterized in that, include: Get the text to be modified; The text to be modified and the preset number of characters are input into the encoder of the text modification model to obtain the text vector set of the text to be modified and the digital encoding set of the preset number of characters. The text vector set and the digital encoding set are then input into the decoder of the text modification model to obtain the target text. The target text has the preset number of characters. The text modification model is trained based on sample text pairs and sample numbers; the sample text pairs include: the sample text to be modified and the modified sample text; the sample numbers represent the number of characters remaining after the byte corresponding to the sample number in the modified sample text; the text vector set includes the byte vector of the first byte of the text to be modified and the byte vectors of other bytes; the byte vector of the first byte is obtained by inputting the first byte into the first encoder of the text modification model; the byte vectors of the other bytes are obtained by performing a first operation; the first operation includes: inputting the target byte of the text to be modified and the first byte vector into the target encoder of the text modification model to obtain the byte vector of the target byte; the first byte vector is the byte vector of the byte preceding the target byte; the target encoder is the encoder corresponding to the target byte; the target byte is any one of the other bytes; the number encoding set includes the number encoding of each digit in the preset number of characters; the number encoding set is obtained by inputting the preset number of characters into the second encoder of the text modification model.

2. The text processing method according to claim 1, characterized in that, Also includes: Obtain the sample text pairs and the sample numbers; The text modification model is trained on the sample text pairs and the sample numbers based on a preset model to obtain the text modification model; The preset model includes: Long Short-Term Memory (LSTM) network model.

3. The text processing method according to claim 1, characterized in that, The step of inputting the text vector set and the number encoding set into the decoder of the text modification model to obtain the target text includes: The end identifier byte of the text to be modified, the numeric code of the largest number in the numeric code set, and the byte vector of the first byte in the text vector set are input into the first decoder of the text modification model to obtain the first byte of the target text; For all byte vectors in the text vector set except for the first byte, and for all numeric codes in the numeric code set except for the numeric code of the largest number, a second operation is performed to obtain the other bytes of the target text. The second operation includes: inputting the numeric code of the target number in the preset number of characters, the byte vector of the target byte in the text vector set, and the byte preceding the target byte into the target decoder of the text modification model to obtain the target byte. The target number is used to represent the number of remaining characters after the target byte in the target text. The target decoder is the decoder corresponding to the target byte. The target text is determined to include the first byte and the other bytes.

4. The text processing method according to claim 3, characterized in that, When the target number is the last number of the preset number of characters, the target byte is the end identifier byte of the target text.

5. A text processing device, characterized in that, include: Acquisition unit and processing unit; The acquisition unit is used to acquire the text to be modified; The processing unit is used to input the text to be modified and the preset number of characters into the encoder of the text modification model to obtain the text vector set of the text to be modified and the digital encoding set of the preset number of characters, and input the text vector set and the digital encoding set into the decoder of the text modification model to obtain the target text; The target text has the preset number of characters. The text modification model is trained based on sample text pairs and sample numbers; the sample text pairs include: the sample text to be modified and the modified sample text; the sample numbers represent the number of characters remaining after the byte corresponding to the sample number in the modified sample text; the text vector set includes the byte vector of the first byte of the text to be modified and the byte vectors of other bytes; the byte vector of the first byte is obtained by inputting the first byte into the first encoder of the text modification model; the byte vectors of the other bytes are obtained by performing a first operation; the first operation includes: inputting the target byte of the text to be modified and the first byte vector into the target encoder of the text modification model to obtain the byte vector of the target byte; the first byte vector is the byte vector of the byte preceding the target byte; the target encoder is the encoder corresponding to the target byte; the target byte is any one of the other bytes; the number encoding set includes the number encoding of each digit in the preset number of characters; the number encoding set is obtained by inputting the preset number of characters into the second encoder of the text modification model.

6. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the text processing method as described in any one of claims 1-4.

7. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the text processing method as described in any one of claims 1-4.

8. A computer program product, comprising instructions, characterized in that, When the instructions are executed on the processor of the electronic device, the processor of the electronic device causes the processor of the electronic device to perform the text processing method as described in any one of claims 1-4.

Images