Text processing method and device, equipment and storage medium

Abstract

The invention relates to the technical field of text processing, and provides a text processing method and device, equipment and a storage medium, and the method comprises the steps: obtaining a to-be-processed text; based on a preset text slicing model, slicing the to-be-processed text to obtain a plurality of initial text fragments; based on a preset abstract extraction model, performing abstract extraction on the initial text fragment to obtain a text abstract corresponding to the initial text fragment; based on a preset relevance evaluation model, according to the to-be-processed text, evaluating the relevance degree between the initial text fragment and the text abstract corresponding to the initial text fragment to obtain an evaluation result; and determining a target text fragment of the to-be-processed text according to the evaluation result, the to-be-processed text and the plurality of initial text fragments. According to the method, the target text fragment is determined by combining the association degree between the text fragment and the text abstract and the plurality of initial text fragments, so that the target text fragment has semantic coherence, and the accuracy of text slicing is improved.

Description

Technical Field

[0001] The present application relates to the field of text processing technology, and in particular to a text processing method, device, equipment and storage medium. Background Art

[0002] In the insurance intelligent question-answering system, text data in the insurance field is generally processed through text slicing methods. The insurance field generally involves a large number of professional terms and specific data structures. For example, an insurance policy may contain professional terms such as "deductible", "employer liability insurance", and "repair outlets".

[0003] Related technologies generally process text data based on rule-based segmentation methods and semantic-based segmentation methods. Among them, the rule-based segmentation method refers to segmenting documents according to pre-defined rules, that is, slicing when the text segment reaches a preset length (such as 512). However, this method of cutting with a fixed block size ignores the semantic coherence of text data, resulting in the segmented text blocks may lose key contextual information. The semantic-based segmentation method refers to using a pre-trained model to evaluate paragraph similarity for segmentation. Although semantic associations are taken into account, they are limited to local document relationships and lack a full document perspective. They may not reflect the overall structure of the document, affecting the global optimization of the segmentation strategy and content performance.

[0004] Related technologies are prone to poor semantic coherence, resulting in low accuracy in text processing. Summary of the invention

[0005] The main purpose of this application is to provide a text processing method, apparatus, device and storage medium, which combines the degree of association between text fragments and text summaries, and multiple initial text fragments obtained by the first segmentation process to determine the final target text fragment, so as to make the target text fragment semantically coherent, thereby further improving the accuracy of text slicing.

[0006] In a first aspect, the present application provides a text processing method, comprising:

[0007] Get the text to be processed;

[0008] Based on a preset text slicing model, the text to be processed is sliced ​​to obtain a plurality of initial text segments;

[0009] Based on a preset summary extraction model, extract a summary of the initial text segment to obtain a text summary corresponding to the initial text segment;

[0010] Based on a preset relevance evaluation model, and according to the text to be processed, the degree of relevance between the initial text segment and the text summary corresponding to the initial text segment is evaluated to obtain an evaluation result;

[0011] A target text segment of the text to be processed is determined according to the evaluation result, the text to be processed and the multiple initial text segments.

[0012] In a second aspect, the present application further provides a text processing device, comprising:

[0013] A text acquisition module is used to acquire the text to be processed;

[0014] A text slicing module, used for slicing the text to be processed based on a preset text slicing model to obtain a plurality of initial text segments;

[0015] A summary extraction module, used to extract a summary of the initial text segment based on a preset summary extraction model to obtain a text summary corresponding to the initial text segment;

[0016] A relevance evaluation module, used to evaluate the relevance between the initial text segment and the text summary corresponding to the initial text segment based on a preset relevance evaluation model and according to the text to be processed, to obtain an evaluation result;

[0017] The slicing adjustment module is used to determine the target text segment of the text to be processed according to the evaluation result, the text to be processed and the multiple initial text segments.

[0018] In a third aspect, the present application further provides a computer device, the computer device comprising a memory and a processor;

[0019] The memory is used to store computer programs;

[0020] The processor is used to execute the computer program and implement the above-mentioned text processing method when executing the computer program.

[0021] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the text processing method as described above are implemented.

[0022] The present application provides a text processing method, device, equipment and storage medium, wherein the method includes: obtaining a text to be processed; slicing the text to be processed based on a preset text slicing model to obtain a plurality of initial text segments; extracting a summary of the initial text segment based on a preset summary extraction model to obtain a text summary corresponding to the initial text segment; evaluating the degree of association between the initial text segment and the text summary corresponding to the initial text segment based on the text to be processed based on a preset relevance evaluation model to obtain an evaluation result; determining a target text segment of the text to be processed based on the evaluation result, the text to be processed and the plurality of initial text segments. The summary extraction model in the present application can extract key information from the text to be processed, provide a basis for determining the target text segment, and determine the final target text segment by combining the degree of association between the text segment and the text summary, and the plurality of initial text segments obtained by the first segmentation process, so as to make the target text segment semantically coherent, thereby further improving the accuracy of text slicing. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 A flowchart of a text processing method provided in an embodiment of the present application;

[0025] Figure 2 A schematic diagram of the connection between the server and the terminal device provided in the embodiment of the present application;

[0026] Figure 3 A schematic diagram of the text processing framework provided in the embodiment of the present application;

[0027] Figure 4 A schematic block diagram of a text processing device provided in an embodiment of the present application;

[0028] Figure 5 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0030] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0031] The embodiments of the present application provide a text processing method, apparatus, device and storage medium. The text processing method can be applied to a terminal device, which can be a mobile phone, a tablet computer, a laptop computer, a desktop computer and the like. It can also be applied to a server, which can be a separate server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0032] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0033] See also Figure 1 , Figure 1 This is a flowchart of a text processing method provided in an embodiment of the present application. It should be noted that the text processing method provided in an embodiment of the present application can be used in a terminal device, and of course can also be used in a server.

[0034] like Figure 2 As shown, the text processing method is applied to a server, the server and the terminal device are connected in communication, and the target text segment obtained by the text processing method can be sent to the terminal device through the server. Of course, it is not limited to this and is not limited here.

[0035] In specific implementation, the terminal device includes but is not limited to: any one of a mobile phone, a tablet computer, a laptop computer, and a desktop computer; the server can be a single server or a server cluster, or a cloud server that provides cloud computing services.

[0036] like Figure 1 As shown, the text processing method includes steps S101 to S105.

[0037] Step S101, obtaining the text to be processed.

[0038] The embodiments of the present application can obtain text to be processed related to the insurance field. The text content related to the insurance field generally includes many professional terms. For example, professional terms such as policyholder, insured, insurance company, insurance exemption, etc. For example, the text to be processed may include "basic information such as the insured's name, gender, age, ID number, contact information and address"; it may also include "detailed descriptions of the name, type, quantity, value, location, etc. of the insured object"; it may also include "guarantees or commitments made by the insured to relevant risks, such as compliance with safety regulations, timely reporting of accidents, etc."

[0039] Step S102: Slice the text to be processed based on a preset text slicing model to obtain a plurality of initial text segments.

[0040] It should be noted that the text to be processed may include sentences corresponding to different index positions. For example, the index position corresponding to sentence A of the first sentence in the text to be processed is 1, and the index position corresponding to sentence B is 8. If slicing is performed at the index position 8, an initial text segment including multiple sentences at index positions 1-8 can be obtained.

[0041] Exemplarily, an embodiment of the present application can obtain text content related to the insurance field and text fragments corresponding to the text content as training data. At the same time, a text slicing model is trained based on the LoRa (Low-Rank Adaptation) algorithm to accurately segment the text to be processed based on the text slicing model to obtain multiple initial text fragments.

[0042] The LoRa algorithm is a technology that performs efficient fine-tuning based on a pre-trained model, and is suitable for large-scale language models and other large deep learning models. The LoRa algorithm adapts and adjusts the model parameters of the model by introducing a low-rank matrix, thereby significantly reducing the computational cost and storage requirements of fine-tuning while maintaining the original capabilities of the pre-trained model. Specifically, the LoRa algorithm can decompose the weight matrix ΔW that needs to be updated in the fine-tuning stage of the model into the product of two low-rank matrices A and B, that is, ΔW≈AB. Among them, the dimensions of A and B are much smaller than the dimensions of the original weight matrix. Therefore, the computing resources and storage resources required to train A and B are greatly reduced.

[0043] The text slicing model in the embodiment of the present application may include the qwen2-7b model. Qwen2-7b is one of the pre-trained and instruction fine-tuning models in the qwen2 large model series released by Alibaba Cloud. It should be noted that the qwen2 series contains 5 sizes of pre-trained and instruction fine-tuning models, including qwen2-0.5B, qwen2-1.5B, qwen2-7B, qwen2-57B-A14B and qwen2-72B. These models provide users with different scales of options to meet different application scenarios and needs.

[0044] like Figure 3 As shown, by slicing the text to be processed based on the text slicing model, K text segments can be obtained. For example, the text to be processed may include basic information such as the insured's name, gender, age, ID number, contact information and address. The multiple initial text segments obtained after the text to be processed are segmented include "the insured...", "gender...", "ID number 4...0", etc. Of course, a text segment may also include one or more sentences. For example, the text segment includes sentence A, and sentence A is "During the payment period, when the insured is due to an accident or after the waiting period... the insured is exempted from the subsequent premiums to be paid, so that the insured's protection rights and interests continue to take effect.".

[0045] The embodiment of the present application trains the qwen2-7b model based on the LoRa algorithm, so that the qwen2-7b model segments the processed text from a global perspective to obtain multiple initial text fragments, thereby ensuring that the initial text fragments after segmentation are more complete and coherent, and improving the semantic coherence of the multiple initial text fragments.

[0046] Step S103: extract a summary of the initial text segment based on a preset summary extraction model to obtain a text summary corresponding to the initial text segment.

[0047] The summary extraction model in the embodiment of the present application may include the qwen2-7b model. Based on the qwen2-7b model, the initial text fragments are subjected to summary extraction, and the text summary corresponding to each initial text fragment can be obtained. Figure 3 As shown, by performing summary extraction on K text segments based on the summary extraction model, K text segments and K text summaries corresponding to the K text segments can be obtained.

[0048] It can be understood that each initial text segment can correspond to a text summary, and each initial text segment can include one sentence or multiple sentences. For example, the initial text segment includes sentence A and sentence B, sentence A is "During the payment period, when the insured due to an accident or after the waiting period... the insured is exempted from the subsequent premiums payable, so that the insured's protection rights and interests continue to take effect"; sentence B is "During the payment period, when the insured due to an accident... the insurance company will exempt the corresponding insurance policy from subsequent premiums payable, so that the insurance policy rights and interests remain valid." This embodiment of the application is based on the qwen2-7b model, and extracts the summary of sentence A and sentence B to obtain the corresponding text summary "Insurance exemption includes policyholder exemption and insured exemption."

[0049] In an exemplary embodiment, the method further includes step S301 and step S302.

[0050] Step S301: Acquire second training data, where the second training data includes a training text and a text summary corresponding to the training text.

[0051] Step S302: training a summary extraction model based on the training text and the text summary.

[0052] For example, in the embodiment of the present application, text content related to the insurance field can be used as training text, and the training text and the corresponding text summary can be used as the second training data. At the same time, based on the LoRa algorithm, the summary extraction model is trained according to the training text and the text summary. The second training data in the embodiment of the present application is text content related to the insurance field. These text contents usually include professional terms in the insurance field (deductibles, exemptions, liability exemptions, etc.) and specific data structures (cash value calculation methods, insurance compensation calculation methods, etc.). Based on this, the summary extraction model obtained by training can more accurately process text content related to the insurance field.

[0053] Step S104: Based on a preset relevance evaluation model and according to the text to be processed, the relevance between the initial text segment and the text summary corresponding to the initial text segment is evaluated to obtain an evaluation result.

[0054] Exemplarily, the relevance evaluation model in the embodiment of the present application may include the qwen2-72b model. The qwen2-72b model is a model with a parameter scale of 72 billion in the qwen2 series, and is also one of the largest and most powerful models in the series. The qwen2-72b model uses the GQA (Grouped Query Attention) mechanism, which makes the model's reasoning speed faster while reducing video memory usage. The qwen2-72b model can provide more efficient and better performance when processing complex tasks, thereby improving the user experience.

[0055] The embodiment of the present application uses the qwen2-72b model to evaluate the degree of association between the initial text fragment and the corresponding text summary based on the content of the text to be processed. Specifically, it determines whether the content of each initial text fragment and the content of the corresponding text summary are similar or whether the degree of association between the two is greater than a set threshold, thereby obtaining an evaluation result.

[0056] In an exemplary embodiment, when the correlation degree between the initial text segment and the text summary corresponding to the initial text segment is greater than a preset correlation degree threshold, the evaluation result is related.

[0057] When the correlation degree between the initial text segment and the text summary corresponding to the initial text segment is less than or equal to a preset correlation degree threshold, the evaluation result is irrelevant.

[0058] For example, in the embodiment of the present application, a relevance threshold may be preset, and when the relevance between each initial text segment and the corresponding text summary is greater than the relevance threshold, the evaluation result may be determined to be relevant. When the relevance between each initial text segment and the corresponding text summary is less than or equal to the preset relevance threshold, the evaluation result may be determined to be irrelevant.

[0059] Furthermore, in the embodiment of the present application, different association degree thresholds may be set. When evaluating the association degree between different initial text segments and corresponding text summaries, the corresponding association degree thresholds may be used, thereby increasing the accuracy of evaluating the association degree.

[0060] In an exemplary implementation, step S104 includes steps S1041 to S1043.

[0061] Step S1041 : Based on a preset relevance evaluation model and according to key data in the text to be processed, the initial text segment and the text summary corresponding to the initial text segment are evaluated to obtain a first evaluation result.

[0062] Step S1042: Evaluate the initial text segment and the text summary corresponding to the initial text segment according to the semantic logical order in the text to be processed to obtain a second evaluation result.

[0063] Step S1043: Obtain an evaluation result according to the first evaluation result and the second evaluation result.

[0064] The embodiment of the present application can first obtain the key data in the text to be processed, and then based on the relevance evaluation model, evaluate the initial text fragment and the corresponding text summary according to the key data in the text to be processed to obtain a first evaluation result. According to the semantic logical order in the text to be processed, the initial text fragment and the corresponding text summary are evaluated to obtain a second evaluation result. Taking the insurance field as an example, the text to be processed is an insurance contract, and the key data in the insurance contract may include the personal information of the policyholder, the personal information of the insured, and the insurance product, etc. The semantic logical order in the insurance contract refers to the semantics between each sentence in the insurance contract and the logical order between multiple sentences. The final evaluation result can be obtained based on the first evaluation result and the second evaluation result.

[0065] For example, when both the first evaluation result and the second evaluation result satisfy the set relevance threshold, it can be determined that the final evaluation result is that the initial text segment is related to the corresponding text summary. When the first evaluation result or the second evaluation result does not satisfy the set relevance threshold, it can be determined that the final evaluation result is that the initial text segment is not related to the corresponding text summary.

[0066] The embodiment of the present application combines the key data of the text to be processed and the overall semantic logical order, evaluates the degree of correlation between the content of the initial text fragment and the content of the corresponding text summary, evaluates from different aspects to obtain different evaluation results, and finally determines whether the initial text fragment is related to the corresponding text summary based on the different evaluation results, thereby further improving the accuracy of the evaluation.

[0067] Step S105: determining a target text segment of the text to be processed according to the evaluation result, the text to be processed and the multiple initial text segments.

[0068] The embodiment of the present application can combine the evaluation result of the degree of association between the initial text segment and the corresponding text summary, and the text to be processed to determine the target text segment, so that the content of the target text segment finally obtained is more complete and coherent.

[0069] In an exemplary embodiment, step S105 includes step S1051 and step S1052. Step S1051: When the evaluation result is irrelevant, based on a preset slice adjustment model, a target text segment of the text to be processed is determined according to the text to be processed and a plurality of initial text segments.

[0070] Step S1052: When the evaluation result is relevant, determine that the multiple initial text segments are target text segments of the text to be processed.

[0071] Exemplarily, if the evaluation result of the degree of association between the initial text fragment and the corresponding text summary is correlated, it means that the index position of the text slicing model for slicing the text to be processed is accurate. Therefore, multiple initial text fragments can be determined as target text fragments of the text to be processed.

[0072] If the evaluation result of the degree of association between the initial text segment and the corresponding text summary is irrelevant, it means that the index position of the text slicing model for slicing the text to be processed is wrong, resulting in errors in the multiple initial text segments obtained. Based on this, the embodiment of the present application can readjust the index position of the slices based on the text to be processed and the multiple initial text segments based on the preset slice adjustment model to obtain the target text segment of the text to be processed.

[0073] like Figure 3 As shown, K text segments are obtained by slicing the text to be processed based on the text slicing model, and M text segments are obtained by slicing the text to be processed based on the slicing adjustment model. For example, the text to be processed includes multiple sentences with index positions of 1-50. The text slicing model is used to slice the text at index positions of 10, 19, 29 and 42, respectively, to obtain 5 text segments. When the evaluation result is irrelevant, the slicing adjustment model can be used to redetermine the index positions for slicing the text to be processed as 12, 22, 33, and 42, respectively, based on the text to be processed and the 5 text segments, thereby obtaining 5 target text segments. Since the index positions of the two slicing processes are different, the 5 text segments obtained in the first time are different from the 5 text segments obtained in the second time.

[0074] The embodiment of the present application combines the strategies of the summary extraction model and the slice adjustment model to optimize the initial text fragments obtained by the text slicing model, thereby obtaining target text fragments with higher accuracy, thereby improving the accuracy of text slicing.

[0075] In an exemplary embodiment, the number of initial text segments obtained based on the text slicing model is different from the number of target text segments obtained based on the slicing adjustment model.

[0076] like Figure 3As shown, the number of initial text segments obtained based on the text slicing model is different from the number of target text segments obtained based on the slicing adjustment model, that is, the number of K text segments is different from the number of M text segments. For example, the text to be processed includes multiple sentences with index positions of 1-50. Based on the text slicing model, the index positions of 10, 19, 29 and 42 in the text to be processed are sliced ​​to obtain 5 text segments. Based on the slicing adjustment model, according to the text to be processed and the 5 text segments, the index positions for slicing the text to be processed are re-determined to be 10, 18, 22, 33, and 42, respectively, thereby obtaining 6 target text segments. Since the index positions of the two slicing processes are different, the number of slicing processes is also different. Therefore, the number of text segments obtained by the two slicing processes is different.

[0077] In an exemplary embodiment, the method further includes step S501 and step S502:

[0078] Step S501, obtaining first training data, the first training data including a training text, a plurality of first text segments corresponding to the training text, and a plurality of second text segments, wherein a degree of association between a second text segment and a text summary corresponding to the second text segment is greater than a degree of association between a first text segment and a text summary corresponding to the first text segment.

[0079] Step S502: training a slice adjustment model according to the training text, the first text segment and the second text segment.

[0080] The slice adjustment model in the embodiment of the present application may include the qwen2-7b model. The first training data may include text content related to the insurance field, a plurality of first text segments corresponding to the text content, and a plurality of second text segments, and the degree of association between the second text segments and the corresponding text summary is greater than the degree of association between the first text segments and the corresponding text summary. In other words, the accuracy of the second text segment is higher than the accuracy of the first text segment.

[0081] Exemplarily, the first text segment may be a text segment obtained based on a text slicing model. The second text segment may be a text segment that has been manually processed in advance and has a high accuracy. In the embodiment of the present application, the slicing adjustment model may be trained based on the text content related to the insurance field, the first text segment, and the second text segment. At the same time, the slicing adjustment model may be fine-tuned based on the LoRa algorithm to improve the accuracy of the slicing adjustment model in slicing text.

[0082] The text processing method provided in the above embodiment includes: obtaining a text to be processed; slicing the text to be processed based on a preset text slicing model to obtain a plurality of initial text segments; extracting a summary of the initial text segment based on a preset summary extraction model to obtain a text summary corresponding to the initial text segment; evaluating the degree of association between the initial text segment and the text summary corresponding to the initial text segment based on the preset relevance evaluation model according to the text to be processed to obtain an evaluation result; determining the target text segment of the text to be processed based on the evaluation result, the text to be processed and the plurality of initial text segments. The summary extraction model in the present application can extract key information from the text to be processed, provide a basis for determining the target text segment, and determine the final target text segment by combining the degree of association between the text segment and the text summary, and the plurality of initial text segments obtained by the first segmentation process, so as to make the target text segment semantically coherent, thereby further improving the accuracy of text slicing.

[0083] See also Figure 4 , Figure 4 A schematic block diagram of a text processing device provided in an embodiment of the present application. The text processing device can be configured in a server or an electronic device to execute the above-mentioned text processing method.

[0084] like Figure 4 As shown, the text processing device includes: a text acquisition module 110 , a text slicing module 120 , a summary extraction module 130 , a relevance evaluation module 140 and a slicing adjustment module 150 .

[0085] The text acquisition module 110 is used to acquire the text to be processed.

[0086] The text slicing module 120 is used to slice the text to be processed based on a preset text slicing model to obtain a plurality of initial text segments.

[0087] The summary extraction module 130 is used to extract a summary of the initial text segment based on a preset summary extraction model to obtain a text summary corresponding to the initial text segment.

[0088] The relevance evaluation module 140 is used to evaluate the relevance between the initial text segment and the text summary corresponding to the initial text segment based on a preset relevance evaluation model and according to the text to be processed, and obtain an evaluation result.

[0089] The slicing adjustment module 150 is used to determine a target text segment of the text to be processed according to the evaluation result, the text to be processed and a plurality of initial text segments.

[0090] In an exemplary embodiment, when the correlation degree between the initial text segment and the text summary corresponding to the initial text segment is greater than a preset correlation degree threshold, the evaluation result is related.

[0091] When the correlation degree between the initial text segment and the text summary corresponding to the initial text segment is less than or equal to a preset correlation degree threshold, the evaluation result is irrelevant.

[0092] In an exemplary implementation, the slice adjustment module 150 includes a first adjustment submodule and a second adjustment submodule.

[0093] The first adjustment submodule is used to determine a target text segment of the text to be processed based on the text to be processed and a plurality of initial text segments based on a preset slice adjustment model when the evaluation result is irrelevant.

[0094] The second adjustment submodule is used to determine the multiple initial text segments as target text segments of the text to be processed when the evaluation result is relevant.

[0095] In an exemplary embodiment, the number of initial text segments obtained based on the text slicing model is different from the number of target text segments obtained based on the slicing adjustment model.

[0096] In an exemplary embodiment, the apparatus further includes a first data acquisition module and a first training module.

[0097] The first data acquisition module is used to acquire first training data, the first training data includes a training text, a plurality of first text segments corresponding to the training text, and a plurality of second text segments, the degree of association between the second text segments and the text summaries corresponding to the second text segments is greater than the degree of association between the first text segments and the text summaries corresponding to the first text segments.

[0098] The first training module is used to train the slice adjustment model according to the training text, the first text segment and the second text segment.

[0099] In an exemplary implementation, the relevance evaluation module 140 includes a first evaluation submodule, a second evaluation submodule, and a third evaluation submodule.

[0100] The first evaluation submodule is used to evaluate the initial text segment and the text summary corresponding to the initial text segment based on a preset relevance evaluation model and according to key data in the text to be processed, so as to obtain a first evaluation result.

[0101] The second evaluation submodule is used to evaluate the initial text segment and the text summary corresponding to the initial text segment according to the semantic logical order in the text to be processed, so as to obtain a second evaluation result.

[0102] The third evaluation submodule is used to obtain an evaluation result according to the first evaluation result and the second evaluation result.

[0103] In an exemplary embodiment, the apparatus further includes a second data acquisition module and a second training module.

[0104] The second data acquisition module is used to acquire second training data, where the second training data includes a training text and a text summary corresponding to the training text.

[0105] The second training module is used to train the summary extraction model according to the training text and the text summary.

[0106] It should be noted that technicians in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0107] The method of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0108] Exemplarily, the above method and apparatus may be implemented in the form of a computer program, which may be run on a computer device.

[0109] See also Figure 5 , Figure 5 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. The computer device may be a server or a terminal device.

[0110] like Figure 5 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a storage medium and an internal memory.

[0111] The storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute the steps of any text processing method.

[0112] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0113] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute the steps of any text processing method.

[0114] The network interface is used for network communication, such as sending assigned tasks.

[0115] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0116] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0117] In one embodiment, the processor is used to execute a computer program and implement the following steps when executing the computer program:

[0118] Get the text to be processed;

[0119] Based on the preset text slicing model, the text to be processed is sliced ​​to obtain multiple initial text segments;

[0120] Based on the preset summary extraction model, the initial text segment is summarized and the text summary corresponding to the initial text segment is obtained;

[0121] Based on a preset relevance evaluation model, and according to the text to be processed, the relevance between the initial text segment and the text summary corresponding to the initial text segment is evaluated to obtain an evaluation result;

[0122] A target text segment of the text to be processed is determined according to the evaluation result, the text to be processed and the multiple initial text segments.

[0123] In an exemplary embodiment, when the correlation degree between the initial text segment and the text summary corresponding to the initial text segment is greater than a preset correlation degree threshold, the evaluation result is related;

[0124] When the correlation degree between the initial text segment and the text summary corresponding to the initial text segment is less than or equal to a preset correlation degree threshold, the evaluation result is irrelevant.

[0125] In an exemplary embodiment, determining a target text segment of the text to be processed according to the evaluation result, the text to be processed, and a plurality of initial text segments includes:

[0126] When the evaluation result is irrelevant, based on a preset slice adjustment model, a target text segment of the text to be processed is determined according to the text to be processed and a plurality of initial text segments.

[0127] When the evaluation result is relevant, the multiple initial text segments are determined as target text segments of the text to be processed.

[0128] In an exemplary embodiment, the number of initial text segments obtained based on the text slicing model is different from the number of target text segments obtained based on the slicing adjustment model.

[0129] In an exemplary embodiment, the method further comprises:

[0130] Acquire first training data, the first training data including a training text, a plurality of first text segments corresponding to the training text, and a plurality of second text segments, wherein a degree of association between the second text segments and text summaries corresponding to the second text segments is greater than a degree of association between the first text segments and text summaries corresponding to the first text segments;

[0131] A slice adjustment model is trained according to the training text, the first text segment and the second text segment.

[0132] In an exemplary embodiment, based on a preset relevance evaluation model, the relevance between the initial text segment and the text summary corresponding to the initial text segment is evaluated according to the text to be processed, and an evaluation result is obtained, including:

[0133] Based on a preset relevance evaluation model, and according to key data in the text to be processed, the initial text segment and the text summary corresponding to the initial text segment are evaluated to obtain a first evaluation result;

[0134] According to the semantic logical order in the text to be processed, the initial text segment and the text summary corresponding to the initial text segment are evaluated to obtain a second evaluation result;

[0135] An evaluation result is obtained according to the first evaluation result and the second evaluation result.

[0136] In an exemplary embodiment, the method further comprises:

[0137] Acquire second training data, where the second training data includes a training text and a text summary corresponding to the training text;

[0138] The summary extraction model is trained based on the training text and text summary.

[0139] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described text processing can refer to the corresponding process in the embodiment of the aforementioned text processing method, and will not be repeated here.

[0140] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The method implemented when the computer program is executed by a processor can refer to the various embodiments of the text processing method of the present application.

[0141] The computer-readable storage medium may be an internal storage unit of the computer device of the aforementioned embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device.

[0142] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0143] It should also be understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0144] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A text processing method, characterized in that: include: Get the text to be processed; Based on a preset text slicing model, the text to be processed is sliced ​​to obtain a plurality of initial text segments; Based on a preset summary extraction model, extract a summary of the initial text segment to obtain a text summary corresponding to the initial text segment; Based on a preset relevance evaluation model, and according to the text to be processed, the degree of relevance between the initial text segment and the text summary corresponding to the initial text segment is evaluated to obtain an evaluation result; A target text segment of the text to be processed is determined according to the evaluation result, the text to be processed and the multiple initial text segments.

2. The text processing method according to claim 1, characterized in that: When the correlation degree between the initial text segment and the text summary corresponding to the initial text segment is greater than a preset correlation degree threshold, the evaluation result is related; When the correlation degree between the initial text segment and the text summary corresponding to the initial text segment is less than or equal to a preset correlation degree threshold, the evaluation result is irrelevant.

3. The text processing method according to claim 2, characterized in that: The step of determining a target text segment of the text to be processed according to the evaluation result, the text to be processed and the plurality of initial text segments comprises: When the evaluation result is irrelevant, based on a preset slice adjustment model, a target text segment of the text to be processed is determined according to the text to be processed and the plurality of initial text segments. When the evaluation result is relevant, the plurality of initial text segments are determined as target text segments of the text to be processed.

4. The text processing method according to claim 3, characterized in that: The number of the initial text segments obtained based on the text slicing model is different from the number of the target text segments obtained based on the slicing adjustment model.

5. The text processing method according to claim 3, characterized in that: The method further comprises: Acquire first training data, the first training data comprising a training text, a plurality of first text segments corresponding to the training text, and a plurality of second text segments, wherein a degree of association between the second text segments and text summaries corresponding to the second text segments is greater than a degree of association between the first text segments and text summaries corresponding to the first text segments; The slice adjustment model is trained according to the training text, the first text segment and the second text segment.

6. The text processing method according to claim 1, characterized in that: The preset relevance evaluation model is used to evaluate the relevance between the initial text segment and the text summary corresponding to the initial text segment according to the text to be processed, and an evaluation result is obtained, including: Based on a preset relevance evaluation model, and according to key data in the text to be processed, the initial text segment and the text summary corresponding to the initial text segment are evaluated to obtain a first evaluation result; According to the semantic logical order in the text to be processed, the initial text segment and the text summary corresponding to the initial text segment are evaluated to obtain a second evaluation result; An evaluation result is obtained according to the first evaluation result and the second evaluation result.

7. The text processing method according to any one of claims 1 to 5, characterized in that: The method further comprises: Acquire second training data, where the second training data includes a training text and a text summary corresponding to the training text; The summary extraction model is trained according to the training text and the text summary.

8. A text processing device, characterized in that: include: A text acquisition module is used to acquire the text to be processed; A text slicing module, used for slicing the text to be processed based on a preset text slicing model to obtain a plurality of initial text segments; A summary extraction module, used to extract a summary of the initial text segment based on a preset summary extraction model to obtain a text summary corresponding to the initial text segment; A relevance evaluation module, used to evaluate the relevance between the initial text segment and the text summary corresponding to the initial text segment based on a preset relevance evaluation model and according to the text to be processed, to obtain an evaluation result; The slicing adjustment module is used to determine the target text segment of the text to be processed according to the evaluation result, the text to be processed and the multiple initial text segments.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the text processing method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the text processing method according to any one of claims 1 to 7 are implemented.

Images