Text processing method and device, equipment, medium and product
By performing word segmentation processing on text and keywords and verifying keywords with word segmentation results, the problem of low accuracy in keyword extraction in the prior art is solved, and more efficient and accurate keyword extraction is achieved.
Patent Information
- Application Number
- CN202510045309.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-10
- Publication Date
- 2025-04-18
AI Technical Summary
In the prior art, the accuracy of extracting keywords from text is low, and how to improve the accuracy of keyword extraction has become an urgent problem.
By obtaining text, extracting keywords, and performing word segmentation processing on the text and keywords, combining the word segmentation results of the two, the verification results of the keywords are determined to improve the accuracy of keyword extraction.
Without affecting the efficiency of keyword extraction, the accuracy of keyword extraction is significantly improved to avoid incorrect keyword extraction.
Smart Images
Figure CN119849496A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a text processing method, device, electronic device, computer-readable storage medium, and computer program product. Background Art
[0002] With the continuous development of computer technology, the ways of text processing have gradually diversified. For example, in some businesses, keywords can be extracted from text.
[0003] In the related art, the accuracy of extracting keywords from text is low, and how to improve the accuracy of keyword extraction has become a problem that needs to be solved urgently. Summary of the invention
[0004] The present application provides a text processing method. The method can improve the accuracy of keyword extraction without affecting the efficiency of keyword extraction. The present application also provides a device, an electronic device, a computer-readable storage medium, and a computer program product corresponding to the above method.
[0005] In a first aspect, the present application provides a text processing method, the method comprising:
[0006] Get the first text;
[0007] Extract keywords from the first text to obtain first keywords;
[0008] Performing word segmentation processing on the first text and the first keyword respectively to obtain a word segmentation result of the first text and a word segmentation result of the first keyword;
[0009] A verification result of the first keyword is determined according to the segmentation result of the first text and the segmentation result of the first keyword.
[0010] In some possible implementations, extracting keywords from the first text to obtain first keywords includes:
[0011] Generate a first prompt word; wherein the first prompt word includes the first text and information indicating that keywords are extracted from the first text;
[0012] The first prompt word is sent to a language model, and a first keyword returned by the language model is received.
[0013] In some possible implementations, performing word segmentation processing on the first text and the first keyword respectively to obtain a word segmentation result of the first text and a word segmentation result of the first keyword includes:
[0014] The first text is sent to a word segmentation model, and a word segmentation result of the first text is received from the word segmentation model; and the first keyword is sent to the word segmentation model, and a word segmentation result of the first keyword is received from the word segmentation model.
[0015] In some possible implementations, determining the verification result of the first keyword according to the word segmentation result of the first text and the word segmentation result of the first keyword includes:
[0016] The verification result of the first keyword is determined according to the position of the separator in the word segmentation result of the first text and the position of the separator in the word segmentation result of the first keyword; wherein the separator is used to indicate different word segments.
[0017] In some possible implementations, before determining the verification result of the first keyword according to the word segmentation result of the first text and the word segmentation result of the first keyword, the method further includes:
[0018] The separator is inserted at a first position and a second position adjacent to the first keyword; wherein the first position is located before the first keyword, and the second position is located after the first keyword.
[0019] In some possible implementations, determining the verification result of the first keyword according to the word segmentation result of the first text and the word segmentation result of the first keyword includes:
[0020] In response to the word segmentation result of the first text and the word segmentation result of the first keyword matching, determining a verification result indicating that the first keyword is correct;
[0021] Otherwise, a verification result indicating that the first keyword is incorrect is determined.
[0022] In some possible implementations, the word segmentation result of the first text and the word segmentation result of the first keyword are indicated by a separator, and in response to the word segmentation result of the first text matching the word segmentation result of the first keyword, determining a verification result indicating that the first keyword is correct includes:
[0023] In response to the position of the separator in the word segmentation result of the first text being the same as the position of the separator in the word segmentation result of the first keyword, a verification result indicating that the first keyword is correct is determined.
[0024] In a second aspect, the present application provides a text processing device, the device comprising:
[0025] An acquisition module, used for acquiring a first text;
[0026] An extraction module, used for extracting keywords from the first text to obtain first keywords;
[0027] A word segmentation module, used to perform word segmentation processing on the first text and the first keyword respectively, to obtain a word segmentation result of the first text and a word segmentation result of the first keyword;
[0028] A determination module is used to determine a verification result of the first keyword according to a segmentation result of the first text and a segmentation result of the first keyword.
[0029] In some possible implementations, the extraction module is specifically used to:
[0030] Generate a first prompt word; wherein the first prompt word includes the first text and information indicating that keywords are extracted from the first text;
[0031] The first prompt word is sent to a language model, and a first keyword returned by the language model is received.
[0032] In some possible implementations, the word segmentation module is specifically used to:
[0033] The first text is sent to a word segmentation model, and a word segmentation result of the first text is received from the word segmentation model; and the first keyword is sent to the word segmentation model, and a word segmentation result of the first keyword is received from the word segmentation model.
[0034] In some possible implementations, the determining module is specifically configured to:
[0035] The verification result of the first keyword is determined according to the position of the separator in the word segmentation result of the first text and the position of the separator in the word segmentation result of the first keyword; wherein the separator is used to indicate different word segments.
[0036] In some possible implementations, the word segmentation module is further used to:
[0037] The separator is inserted at a first position and a second position adjacent to the first keyword; wherein the first position is located before the first keyword, and the second position is located after the first keyword.
[0038] In some possible implementations, the determining module is specifically configured to:
[0039] In response to the word segmentation result of the first text and the word segmentation result of the first keyword matching, determining a verification result indicating that the first keyword is correct;
[0040] Otherwise, a verification result indicating that the first keyword is incorrect is determined.
[0041] In some possible implementations, the word segmentation result of the first text and the word segmentation result of the first keyword are indicated by separators, and the determining module is specifically configured to:
[0042] In response to the position of the separator in the word segmentation result of the first text being the same as the position of the separator in the word segmentation result of the first keyword, a verification result indicating that the first keyword is correct is determined.
[0043] In a third aspect, the present application provides an electronic device, the electronic device comprising a processor and a memory. The processor and the memory communicate with each other. The processor is used to execute instructions stored in the memory, so that the electronic device performs the text processing method in the first aspect or any implementation of the first aspect.
[0044] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, wherein the instructions instruct an electronic device to execute the text processing method described in the first aspect or any one of the implementations of the first aspect.
[0045] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the text processing method described in the first aspect or any one of the implementations of the first aspect.
[0046] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations.
[0047] It can be seen from the above technical solutions that this application has the following advantages:
[0048] The present application provides a text processing method, which first obtains a first text, extracts keywords from the first text, obtains a first keyword, then performs word segmentation processing on the first text and the first keyword respectively, obtains a word segmentation result of the first text and a word segmentation result of the first keyword, and determines a verification result of the first keyword based on the word segmentation result of the first text and the word segmentation result of the first keyword.
[0049] In this method, for the scenario of extracting keywords from text, after extracting the first keyword, the original text (i.e., the first text) and the extracted keyword (i.e., the first keyword) are segmented respectively, and the two segmentation results are combined to verify the keyword extraction to avoid errors in the extracted keywords. Since the execution speed of the segmentation processing is relatively fast, the accuracy of keyword extraction is improved without affecting the efficiency of keyword extraction. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the technical method of the embodiments of the present application, the drawings required for use in the embodiments are briefly introduced below.
[0051] Figure 1 A flowchart of a text processing method provided in an embodiment of the present application;
[0052] Figure 2 A schematic diagram of the structure of a text processing device provided in an embodiment of the present application;
[0053] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0054] The terms "first" and "second" in the embodiments of the present application are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.
[0055] First, some technical terms and application scenarios involved in the embodiments of this application are introduced.
[0056] Keyword extraction usually refers to the process of extracting keywords from the original text. When performing keyword extraction, there is a certain probability of keyword extraction errors. For example, when the keyword is a difficult-to-understand word, keyword extraction is performed on the original text "Let the customer service management help to promote this content within the customer", and the keyword "internal propaganda" is extracted. That is, "internal propaganda" in the above text is a difficult-to-understand word. However, the correct word segmentation of the above original text should be "Let the customer service management help to promote this content within the customer", and the keyword "internal propaganda" does not exist. Due to the word segmentation error, a keyword extraction error occurs.
[0057] In response to the above problems, the related technologies usually adopt the following methods to deal with them: In some examples, keyword extraction is completed by a language model. In this case, the language model can self-correct the extracted keywords. For example, the language model can use a chain of thought to make the language model perform an additional step of determining whether the keyword extraction is correct after extracting the keywords. In other examples, the original text can be pre-segmented using traditional word segmentation methods, and then keywords can be extracted from the segmented text.
[0058] However, the above method has the following defects: In the method of self-correcting the extracted keywords by the language model, the self-correction step may increase the delay, reduce the efficiency of keyword extraction, and make it difficult to meet business needs. In the method of pre-segmenting the text by the traditional word segmentation method and then extracting keywords from the original text after word segmentation, the word segmentation effect is poor for words with low historical frequency, which is easy to cause word segmentation errors, resulting in poor keyword extraction effect.
[0059] In view of this, the present application provides a text processing method, which first obtains a first text, extracts keywords from the first text, obtains a first keyword, then performs word segmentation processing on the first text and the first keyword respectively, obtains the word segmentation result of the first text and the word segmentation result of the first keyword, and determines the verification result of the first keyword based on the word segmentation result of the first text and the word segmentation result of the first keyword.
[0060] In this method, for the scenario of extracting keywords from text, after extracting the first keyword, the original text (i.e., the first text) and the extracted keyword (i.e., the first keyword) are segmented respectively, and the two segmentation results are combined to verify the keyword extraction to avoid errors in the extracted keywords. Since the execution speed of the segmentation processing is relatively fast, the accuracy of keyword extraction is improved without affecting the efficiency of keyword extraction.
[0061] To facilitate understanding of the technical solution provided by the embodiments of the present application, the following will be described in conjunction with the accompanying drawings. Figure 1 A flowchart of a text processing method is shown, the method specifically comprising:
[0062] S101: Obtain a first text.
[0063] In the embodiment of the present application, the first text can be understood as any text that has a keyword extraction requirement.
[0064] The source of the first text may be different. For example, in a document scenario, the first text may be text in a document or text entered by a user in a document page. For another example, in a video conference scenario, the first text may be text in a conference record or text in a conference minutes. For another example, in an instant messaging (IM) scenario, the first text may be text in an IM message (e.g., a single chat message or a group chat message) or text entered by a user in an IM page.
[0065] S102: extracting keywords from the first text to obtain first keywords.
[0066] The process of extracting keywords from the first text can be understood as the process of determining whether the keywords exist in the first text. Under different extraction requirements, keywords can have different types, for example, keywords can be difficult to understand words, entity words, words related to names, etc., which are not limited in the embodiments of the present application.
[0067] In some possible implementations, the process of extracting keywords from the first text can be implemented based on keyword extraction rules. Specifically, keyword extraction rules are pre-formulated, for example, the keyword extraction rules can be related to a keyword set, when the first text includes a keyword in the keyword set, the keyword is extracted to obtain the first keyword, and when the first text does not include a keyword in the keyword set, it is determined that the first text does not include the keyword.
[0068] In some other possible implementations, the process of extracting keywords from the first text can also be implemented with the help of a language model. The language model has natural language processing capabilities, can understand the meaning of natural language, and process different types of natural language tasks. For example, the language model can be a deep learning model trained using text data. In other words, the natural language processing capabilities of the language model are used to analyze the first text, determine whether the first text includes keywords, and implement the process of extracting keywords from the first text.
[0069] In an embodiment of the present application, the language model extracts keywords from the first text based on prompt learning. Among them, the prompt word can be used to guide the language model to perform specific output in a generative task (such as a text generation task, a question-answering task, and a dialogue task). By configuring the prompt word, the language model is helped to understand the background and requirements of the task, so that the language model can handle different types of natural language processing tasks without retraining the language model, thereby increasing the scalability and flexibility of the language model.
[0070] In a specific implementation, a first prompt word is generated, the first prompt word is sent to a language model, and a first keyword returned by the language model is received.
[0071] The first prompt word includes the first text and information indicating that keywords are to be extracted from the first text. That is, by configuring the first text and information for instructing the language model to extract keywords in the first prompt word, the language model can determine whether there are keywords in the first text (for example, whether there are difficult-to-understand words, entity words, words involving names, etc.) based on the prompting ability of the first prompt word, and output the extracted first keywords, thereby improving the applicability of keyword extraction.
[0072] In some possible implementations, considering that the first text may be related to a specific business, there may be certain differences in extracting keywords in different businesses. Therefore, the information in the first prompt word indicating that keywords are extracted from the first text may also specifically indicate that keywords are extracted from the first text in a specific business. For example, the first prompt word may include information indicating that keywords are extracted from the first text in a product sales business, or the first prompt word may also include information indicating that keywords are extracted from the first text in a demand review business.
[0073] Taking the keyword as an incomprehensible word as an example, the first prompt word is exemplified and described. The first prompt word may be:
[0074] “Your task is to identify and extract difficult-to-understand words from a given text. These words may include professional terms in the Internet industry, overly abstract words, or rare English abbreviations.
[0075] Please follow the requirements below:
[0076] You need to ensure the accuracy of the extraction and only extract it when you are very sure that a word is difficult to understand.
[0077] Your thought process should go something like this:
[0078] 1. First, you need to read the entire text and find all the difficult words.
[0079] 2. Next, confirm whether you are very sure that the words you are going to extract are difficult to understand. If you can infer the meaning of the words by combining the context, do not extract them.
[0080] The text is as follows: [First text]
[0081] Please return the extracted incomprehensible words. "
[0082] S103: Perform word segmentation processing on the first text and the first keyword respectively to obtain the word segmentation result of the first text and the word segmentation result of the first keyword.
[0083] The word segmentation process can be understood as a process of segmenting the original text into multiple words. In the embodiment of the present application, the word segmentation results after two word segmentation processes are obtained by performing word segmentation on the first text and performing word segmentation on the first keyword.
[0084] In some possible implementations, the process of performing word segmentation processing on the first text and the first keyword can be implemented based on a preset word segmentation rule. Specifically, the word segmentation rule is pre-configured in combination with the word segmentation requirements, and the word segmentation rule is applied to the process of performing word segmentation processing on the first text and the first keyword, so as to obtain the word segmentation result of the first text and the word segmentation result of the first keyword.
[0085] In some other possible implementations, the process of performing word segmentation processing on the first text and the first keyword can also be implemented with the help of a word segmentation model. Specifically, the first text is sent to the word segmentation model, and the word segmentation result of the first text returned by the word segmentation model is received, and the first keyword is sent to the word segmentation model, and the word segmentation result of the first keyword returned by the word segmentation model is received.
[0086] The word segmentation model can be understood as a model with word segmentation processing capabilities. The embodiment of the present application does not limit the implementation method of the word segmentation model. For example, the word segmentation model can be a hidden Markov model (HMM), or the word segmentation model can also be implemented based on dynamic programming, conditional random fields (CRF), neural network and other technologies.
[0087] By inputting the first text and the first keyword into the word segmentation model respectively, the word segmentation model can automatically analyze the first text and the first keyword and output the word segmentation result of the first text and the word segmentation result of the first keyword. In this way, efficient word segmentation processing is achieved using the word segmentation model.
[0088] In some embodiments, the word segmentation process can be regarded as a process of inserting separators at appropriate positions of the original text. In other words, after the original text is word segmented, the word segmentation result of the original text can include the original text and at least one separator, and the separator can be used to indicate different word segments.
[0089] The embodiments of the present application do not limit the specific form of the separator. For example, the separator can be any symbol such as “<>”, “|”, “[]”, etc., and a suitable separator can be selected based on specific word segmentation requirements.
[0090] Taking the delimiter "<>" as an example, the word segmentation process is explained. The original text is "Increase the penetration rate of advertising", and after processing the original text, the word segmentation result of the original text is "<>Increase<> Advertising<> Penetration Rate<>". That is to say, the original text is divided into three words, and in the word segmentation result of the original text, there are delimiters before and after each word.
[0091] In an embodiment of the present application, after the first text and the first keyword are respectively segmented, the extracted first keyword is verified in combination with the segmentation results. Therefore, in order to accurately and uniformly verify the extracted first keyword, a separator may also be inserted at the first position and the second position adjacent to the first keyword.
[0092] The first position is located before the first keyword, and the second position is located after the first keyword. In other words, the first position is the character position before the first keyword, and the second position is the character position after the first keyword. In this way, by inserting separators before and after the first keyword, the extracted keywords have the same form as the segmentation result of the word segmentation processing, which is convenient for the subsequent verification of the first keyword in combination with the segmentation result.
[0093] Taking the first text "Increase advertising penetration rate" as an example, keywords are extracted from the first text to obtain the first keyword "advertising", and the first keyword is updated to "<> advertising<>" by inserting separators at the first position and the second position of the first keyword.
[0094] S104: Determine a verification result of the first keyword according to the word segmentation result of the first text and the word segmentation result of the first keyword.
[0095] In the embodiment of the present application, the first text and the first keyword are segmented respectively, and the first keyword is verified by comparing the segmentation result of the first text with the segmentation result of the first keyword to determine whether the extracted first keyword is correct.
[0096] In a specific implementation, in response to the word segmentation result of the first text matching the word segmentation result of the first keyword, a verification result representing that the first keyword is correct is determined; otherwise, a verification result representing that the first keyword is incorrect is determined.
[0097] That is to say, when the segmentation result after the segmentation processing for the first text is the same as the segmentation result after the segmentation processing for the first keyword, it indicates that there is no segmentation error in the first keyword extracted from the first text, the first keyword after the correct segmentation is extracted, and the verification of the first keyword passes. When the segmentation result after the segmentation processing for the first text is different from the segmentation result after the segmentation processing for the first keyword, it indicates that there is a segmentation error in the first keyword extracted from the first text, the first keyword is extracted after the incorrect segmentation, and the verification of the first keyword fails.
[0098] It can be understood that since the first keyword is a keyword extracted from the first text, the first keyword is a partial text in the first text. The process of judging whether the word segmentation result of the first text is consistent with the word segmentation result of the first keyword can be understood as the process of judging whether the word segmentation result related to the first keyword in the first text is consistent with the word segmentation result of the first keyword, wherein the word segmentation result related to the first keyword in the first text can be understood as the word segmentation result corresponding to the partial text in the first text including the first keyword.
[0099] In this way, by comparing the segmentation results of the first text with the segmentation results of the first keyword, the first keyword is verified to avoid segmentation errors in the first keyword extracted from the first text. By combining the direct keyword extraction method and the traditional word segmentation processing method, the accuracy of keyword extraction is ensured, and at the same time, the efficiency and performance of keyword extraction are improved.
[0100] In some embodiments, the word segmentation process can be regarded as a process of inserting a separator at an appropriate position of the original text. In this case, the verification result of the first keyword is determined based on the position of the separator in the word segmentation result of the first text and the position of the separator in the word segmentation result of the first keyword.
[0101] That is to say, since the delimiter can be used to indicate different word segmentations, therefore, combined with the position of the delimiter in the two word segmentation results, it is possible to determine whether there is a word segmentation error in the first keyword extracted from the first text. By comparing the position of the delimiter in the word segmentation results, verification of the first keyword is achieved, thereby improving verification efficiency.
[0102] Specifically, in response to the position of the separator in the word segmentation result of the first text being the same as the position of the separator in the word segmentation result of the first keyword, a verification result representing that the first keyword is correct is determined; otherwise, a verification result representing that the first keyword is incorrect is determined.
[0103] That is to say, when the position of the separator in the word segmentation result of the first text is consistent with the position of the separator in the word segmentation result of the first keyword, it indicates that there is no word segmentation error in the first keyword extracted from the first text, and the verification of the first keyword passes. When the position of the separator in the word segmentation result of the first text is inconsistent with the position of the separator in the word segmentation result of the first keyword, it indicates that there is a word segmentation error in the first keyword extracted from the first text, and the verification of the first keyword fails.
[0104] Similarly, since the first keyword is a keyword extracted from the first text, the first keyword is a partial text in the first text, and the process of determining whether the position of the delimiter in the word segmentation result of the first text is consistent with the position of the delimiter in the word segmentation result of the first keyword can be understood as the process of determining whether the position of the delimiter in the word segmentation result related to the first keyword in the first text is consistent with the position of the delimiter in the word segmentation result of the first keyword.
[0105] The text processing method of the embodiment of the present application is explained below with reference to two examples. In the first example, the first text is "empowering small and medium-sized customers to produce materials, diagnose delivery and improve the efficiency of lead management", keywords are extracted from the first text to obtain the first keyword "lead management", and separators (for example, "<>") are inserted before and after the first keyword to obtain the updated first keyword "<>lead management<>". Next, the first text is segmented to obtain the segmentation result of the first text "<>empowering <>small<>customers<> to <>produce<> materials<>, <>delivery<>diagnosis<> and <>lead<>management<>efficiency<>improvement<>", and the first keyword is segmented to obtain the segmentation result of the first keyword "<>lead<>management<>". By comparing the segmentation result related to the first keyword in the first text with the segmentation result of the first keyword, it can be found that the segmentation result related to the first keyword in the first text and the segmentation result of the first keyword are both "<>lead<>management<>", that is, the first keyword is extracted when the segmentation of the first text is correct, and the verification of the first keyword passes.
[0106] In the second example, the first text is "New additions mainly rely on natural traffic, and acquire customers accurately". Keywords are extracted from the first text to obtain the first keyword "natural flow". Delimiters (for example, "<>") are inserted before and after the first keyword to obtain the updated first keyword "<>natural flow<>". Next, the first text is segmented to obtain the segmentation result of the first text "<>New additions<> mainly<> rely on<>natural<> traffic<>, <>Accurate customer<>". The first keyword is segmented to obtain the segmentation result of the first keyword "<>natural<>flow<>". Comparing the segmentation result related to the first keyword in the first text with the segmentation result of the first keyword, it can be found that the segmentation result related to the first keyword in the first text is different from the segmentation result of the first keyword. The segmentation result related to the first keyword in the first text is "<>natural<>traffic<>", and the segmentation result of the first keyword is "<>natural<>flow<>", that is, the first keyword is extracted when the segmentation of the first text is wrong, and the verification of the first keyword fails.
[0107] In this method, for the scenario of extracting keywords from text, after extracting the first keyword, the original text (i.e., the first text) and the extracted keyword (i.e., the first keyword) are segmented respectively, and the two segmentation results are combined to verify the keyword extraction to avoid errors in the extracted keywords. Since the execution speed of the segmentation processing is relatively fast, the accuracy of keyword extraction is improved without affecting the efficiency of keyword extraction.
[0108] Combination of the above Figure 1 The text processing method provided in the embodiment of the present application is introduced in detail. The apparatus and device provided in the embodiment of the present application will be introduced in conjunction with the accompanying drawings.
[0109] See also Figure 2 The structural diagram of the text processing device shown in FIG. 20 includes:
[0110] An acquisition module 201 is used to acquire a first text;
[0111] An extraction module 202, configured to extract keywords from the first text to obtain first keywords;
[0112] A word segmentation module 203 is used to perform word segmentation processing on the first text and the first keyword respectively to obtain a word segmentation result of the first text and a word segmentation result of the first keyword;
[0113] The determination module 204 is used to determine the verification result of the first keyword according to the word segmentation result of the first text and the word segmentation result of the first keyword.
[0114] In some possible implementations, the extraction module 202 is specifically configured to:
[0115] Generate a first prompt word; wherein the first prompt word includes the first text and information indicating that keywords are extracted from the first text;
[0116] The first prompt word is sent to a language model, and a first keyword returned by the language model is received.
[0117] In some possible implementations, the word segmentation module 203 is specifically used to:
[0118] The first text is sent to a word segmentation model, and a word segmentation result of the first text is received from the word segmentation model; and the first keyword is sent to the word segmentation model, and a word segmentation result of the first keyword is received from the word segmentation model.
[0119] In some possible implementations, the determining module 204 is specifically configured to:
[0120] The verification result of the first keyword is determined according to the position of the separator in the word segmentation result of the first text and the position of the separator in the word segmentation result of the first keyword; wherein the separator is used to indicate different word segments.
[0121] In some possible implementations, the word segmentation module 203 is further used to:
[0122] The separator is inserted at a first position and a second position adjacent to the first keyword; wherein the first position is located before the first keyword, and the second position is located after the first keyword.
[0123] In some possible implementations, the determining module 204 is specifically configured to:
[0124] In response to the word segmentation result of the first text and the word segmentation result of the first keyword matching, determining a verification result indicating that the first keyword is correct;
[0125] Otherwise, a verification result indicating that the first keyword is incorrect is determined.
[0126] In some possible implementations, the word segmentation result of the first text and the word segmentation result of the first keyword are indicated by separators, and the determination module 204 is specifically configured to:
[0127] In response to the position of the separator in the word segmentation result of the first text being the same as the position of the separator in the word segmentation result of the first keyword, a verification result indicating that the first keyword is correct is determined.
[0128] The text processing device 20 according to the embodiment of the present application may correspond to executing the method described in the embodiment of the present application, and the above and other operations and / or functions of each module / unit of the text processing device 20 are respectively to implement Figure 1 For the sake of brevity, the corresponding processes of each method in the illustrated embodiment are not described in detail here.
[0129] The present application also provides an electronic device. The electronic device is specifically used to implement Figure 2 The functions of the text processing device 20 in the illustrated embodiment.
[0130] Figure 3 A schematic diagram of the structure of an electronic device 300 is provided. Figure 3 As shown, the electronic device 300 includes a bus 301, a processor 302, a communication interface 303 and a memory 304. The processor 302, the memory 304 and the communication interface 303 communicate with each other via the bus 301.
[0131] The bus 301 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0132] The processor 302 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).
[0133] The communication interface 303 is used for communicating with the outside. For example, the communication interface 303 can be used for communicating with a terminal.
[0134] The memory 304 may include a volatile memory, such as a random access memory (RAM). The memory 304 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).
[0135] The memory 304 stores executable codes, and the processor 302 executes the executable codes to perform the aforementioned text processing method.
[0136] Specifically, in implementing Figure 2 In the case of the embodiment shown, and Figure 2 When each module or unit of the text processing device 20 described in the embodiment is implemented by software, the execution Figure 2 The software or program code required for the functions of each module / unit in the text processing method may be partially or completely stored in the memory 304. The processor 302 executes the program code corresponding to each unit stored in the memory 304 to perform the above-mentioned text processing method.
[0137] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the above-mentioned text processing method applied to the text processing device 20.
[0138] The embodiment of the present application further provides a computer program product, which includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiment of the present application is generated in whole or in part.
[0139] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer or data center to another website, computer or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.
[0140] When the computer program product is executed by a computer, the computer executes any of the aforementioned text processing methods. The computer program product may be a software installation package, and when any of the aforementioned text processing methods is needed, the computer program product may be downloaded and executed on a computer.
[0141] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to each embodiment of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0142] The units involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the unit / module does not, in some cases, constitute a limitation on the unit itself.
[0143] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
[0144] In the context of the present application embodiment, machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable medium can include but is not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the above. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.
[0145] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.
[0146] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0147] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0148] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0149] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A text processing method, characterized in that: The method comprises: Get the first text; Extract keywords from the first text to obtain first keywords; Performing word segmentation processing on the first text and the first keyword respectively to obtain a word segmentation result of the first text and a word segmentation result of the first keyword; A verification result of the first keyword is determined according to the segmentation result of the first text and the segmentation result of the first keyword.
2. The method according to claim 1, characterized in that The step of extracting a keyword from the first text to obtain a first keyword includes: Generate a first prompt word; wherein the first prompt word includes the first text and information indicating that keywords are extracted from the first text; The first prompt word is sent to a language model, and a first keyword returned by the language model is received.
3. The method according to claim 1, characterized in that The performing word segmentation processing on the first text and the first keyword respectively to obtain the word segmentation result of the first text and the word segmentation result of the first keyword includes: The first text is sent to a word segmentation model, and a word segmentation result of the first text is received from the word segmentation model; and the first keyword is sent to the word segmentation model, and a word segmentation result of the first keyword is received from the word segmentation model.
4. The method according to claim 1, characterized in that: The step of determining the verification result of the first keyword according to the word segmentation result of the first text and the word segmentation result of the first keyword includes: The verification result of the first keyword is determined according to the position of the separator in the word segmentation result of the first text and the position of the separator in the word segmentation result of the first keyword; wherein the separator is used to indicate different word segments.
5. The method according to claim 4, characterized in that Before determining the verification result of the first keyword according to the word segmentation result of the first text and the word segmentation result of the first keyword, the method further includes: The separator is inserted at a first position and a second position adjacent to the first keyword; wherein the first position is located before the first keyword, and the second position is located after the first keyword.
6. The method according to any one of claims 1 to 5, characterized in that: The step of determining the verification result of the first keyword according to the word segmentation result of the first text and the word segmentation result of the first keyword includes: In response to the word segmentation result of the first text and the word segmentation result of the first keyword matching, determining a verification result indicating that the first keyword is correct; Otherwise, a verification result indicating that the first keyword is incorrect is determined.
7. The method according to claim 6, characterized in that The word segmentation result of the first text and the word segmentation result of the first keyword are indicated by a separator, and in response to the word segmentation result of the first text matching the word segmentation result of the first keyword, determining a verification result indicating that the first keyword is correct includes: In response to the position of the separator in the word segmentation result of the first text being the same as the position of the separator in the word segmentation result of the first keyword, a verification result indicating that the first keyword is correct is determined.
8. A text processing device, characterized in that: The device comprises: An acquisition module, used for acquiring a first text; An extraction module, used to extract keywords from the first text to obtain first keywords; A word segmentation module, used to perform word segmentation processing on the first text and the first keyword respectively, to obtain a word segmentation result of the first text and a word segmentation result of the first keyword; A determination module is used to determine a verification result of the first keyword according to a segmentation result of the first text and a segmentation result of the first keyword.
9. An electronic device, characterized in that: The electronic device comprises a processor and a memory; The processor is configured to execute instructions stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The method comprises instructions, wherein the instructions instruct an electronic device to execute the method as claimed in any one of claims 1 to 7.
11. A computer program product, characterized in that The computer program product comprises computer readable instructions for implementing the method according to any one of claims 1 to 7.