Text processing method and device, electronic equipment, storage medium and program product

Abstract

The application provides a text processing method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: performing feature extraction on a to-be-processed text to obtain initial text features of the to-be-processed text; performing authenticity prediction on the logic of the to-be-processed text in at least one prediction dimension based on the initial text features to obtain authenticity prediction results of the to-be-processed text in each prediction dimension respectively; when the authenticity prediction result indicates that the to-be-processed text does not have authenticity in the corresponding prediction dimension, obtaining a modified feature of the initial text features in the corresponding prediction dimension; performing feature modification on the initial text features based on the modified feature to obtain target text features corresponding to the initial text features; and performing feature decoding on the target text features to obtain a target text corresponding to the to-be-processed text, wherein the target text has authenticity in each prediction dimension. Through the application, the accuracy of text processing can be effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a text processing method and device, electronic equipment, storage medium and program product. BACKGROUND

[0002] Artificial intelligence (AI) is the use of digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. Artificial intelligence basic technology generally includes, such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-training model technology, operation / interaction system, mechatronics, etc. Among them, the pre-training model is also called large model, basic model, which can be widely applied to downstream tasks in various directions of artificial intelligence after fine tuning. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning, etc.

[0003] In the related art, for text processing, the feature extraction and decoding are directly performed on the to-be-processed text to obtain the target text of the to-be-processed text. However, due to the illusion phenomenon in the text processing process, the target text does not have authenticity in the prediction dimension, thereby resulting in low accuracy of the determined target text and low accuracy of the text processing. SUMMARY

[0004] The embodiments of the present application provide a text processing method, device, electronic equipment, computer readable storage medium and computer program product, which can effectively improve the accuracy of text processing.

[0005] The technical solutions of the embodiments of the present application are as follows:

[0006] The embodiments of the present application provide a text processing method, comprising:

[0007] performing feature extraction on the to-be-processed text to obtain initial text features of the to-be-processed text;

[0008] performing authenticity prediction on the logic of the to-be-processed text in at least one prediction dimension based on the initial text features to obtain authenticity prediction results of the to-be-processed text in each prediction dimension;

[0009] when the authenticity prediction result indicates that the to-be-processed text does not have authenticity in the corresponding prediction dimension, obtaining a correction feature of the initial text features in the corresponding prediction dimension;

[0010] based on the correction feature, performing feature correction on the initial text feature to obtain a target text feature corresponding to the initial text feature;

[0011] performing feature decoding on the target text feature to obtain a target text corresponding to the text to be processed, the target text having the authenticity under each of the prediction dimensions.

[0012] Embodiments of the present application provide a text processing apparatus, comprising:

[0013] a feature extraction module configured to perform feature extraction on a text to be processed to obtain an initial text feature of the text to be processed;

[0014] an authenticity prediction module configured to perform authenticity prediction on the logic of the text to be processed in at least one prediction dimension based on the initial text feature to obtain authenticity prediction results of the text to be processed in each of the prediction dimensions respectively;

[0015] a obtaining module configured to, when the authenticity prediction result indicates that the text to be processed does not have authenticity in the corresponding prediction dimension, obtain a correction feature of the initial text feature in the corresponding prediction dimension;

[0016] a feature correction module configured to perform feature correction on the initial text feature based on the correction feature to obtain a target text feature corresponding to the initial text feature;

[0017] a feature decoding module configured to perform feature decoding on the target text feature to obtain a target text corresponding to the text to be processed, the target text having the authenticity under each of the prediction dimensions.

[0018] In the above scheme, the feature extraction is implemented through at least one feature extraction network, and the feature extraction module is further configured to call a first feature extraction network to perform feature extraction on the text to be processed to obtain a first initial text feature; and perform the following processing by iterating i: calling an i-th feature extraction network to perform feature extraction on the text to be processed based on an (i-1)-th initial text feature to obtain an i-th initial text feature; wherein 1

[0019] In the scheme, the text processing apparatus further comprises a feature checking module configured to perform authenticity prediction on the logic of the text to be processed in each of the prediction dimensions based on the i-1th initial text feature, to obtain i-1th authenticity prediction result of the text to be processed in each of the prediction dimensions, and to perform feature checking on the i-1th initial text feature based on the i-1th authenticity prediction result, to obtain i-1th target text feature.

[0020] In the scheme, the feature checking module is further configured to perform feature correction on the i-1th initial text feature to obtain i-1th target text feature when the i-1th authenticity prediction result indicates that the text to be processed does not have the authenticity in the corresponding prediction dimension, and to determine the i-1th initial text feature as the i-1th target text feature when each of the i-1th authenticity prediction result indicates that the text to be processed has the authenticity in the corresponding prediction dimension.

[0021] In the scheme, the authenticity prediction module is further configured to obtain authenticity prediction network corresponding to each of the prediction dimensions, and perform the following processing for each of the prediction dimensions: calling the corresponding authenticity prediction network, performing authenticity prediction on the logic of the text to be processed in the prediction dimension based on the initial text feature, to obtain authenticity score of the text to be processed in the prediction dimension; when the authenticity score is greater than or equal to a score threshold, determining authenticity prediction result of the prediction dimension as a first result, the first result being used to indicate that the text to be processed has the authenticity in the prediction dimension; when the authenticity score is less than the score threshold, determining authenticity prediction result of the prediction dimension as a second result, the second result being used to indicate that the text to be processed does not have the authenticity in the prediction dimension.

[0022] In the scheme, the authenticity prediction module is further configured to obtain an initial prediction network, obtain a plurality of text feature samples corresponding to a text sample and authenticity label scores of each of the text feature samples, call the initial prediction network for each of the text feature samples, perform authenticity prediction on the logic of the text sample in the prediction dimension based on the text feature sample, to obtain authenticity score corresponding to the text feature sample, and determine loss value corresponding to the text feature sample in combination with the authenticity score and the corresponding authenticity label score, and train the initial prediction network based on the loss value corresponding to each of the text feature samples to obtain authenticity prediction network corresponding to the prediction dimension.

[0023] In the above solution, the authenticity prediction module is further configured to obtain a text sample, perform feature extraction on the text sample, and obtain initial text features of the text sample; and perform feature splitting on the initial text features of the text sample to obtain a plurality of text feature samples corresponding to the text sample.

[0024] In the above solution, the authenticity prediction module is further configured to obtain a text sample, perform feature extraction on the text sample, and obtain initial text features of the text sample; and perform feature splitting on the initial text features of the text sample to obtain a plurality of text feature samples corresponding to the text sample.

[0025] In the above solution, the authenticity prediction module is further configured to obtain an initial prediction network, obtain a first text feature sample corresponding to a text sample of a first prediction dimension and a first authenticity label score of the first text feature sample, call the initial prediction network, perform authenticity prediction on a logic of the text sample of the first prediction dimension based on the first text feature sample to obtain a first authenticity score, train the initial prediction network in combination with the first authenticity score and the first authenticity label score to obtain an authenticity prediction network corresponding to the first prediction dimension, and perform the following processing by traversing j: obtain a (j-1)th authenticity score corresponding to a text sample of a (j-1)th prediction dimension, train the initial prediction network based on the (j-1)th authenticity score to obtain an authenticity prediction network corresponding to a jth prediction dimension, where 2≤j≤M, and M is used to indicate a number of the prediction dimensions.

[0026] In the above solution, the authenticity prediction module is further configured to obtain a jth text feature sample corresponding to a text sample of a jth prediction dimension and a jth authenticity label score of the jth text feature sample, call the initial prediction network, perform authenticity prediction on a logic of the text sample of the jth prediction dimension based on the jth text feature sample to obtain a jth authenticity score, determine a first loss value in combination with the jth authenticity score and a (j-1)th authenticity score, determine a second loss value in combination with the jth authenticity score and the jth authenticity label score, and train the initial prediction network in combination with the first loss value and the second loss value to obtain an authenticity prediction network corresponding to the jth prediction dimension.

[0027] In the above solution, the feature decoding module is further configured to, when authenticity prediction results of all the prediction dimensions indicate that the to-be-processed text has the authenticity in the corresponding prediction dimensions, perform feature decoding on the initial text features to obtain the target text corresponding to the to-be-processed text.

[0028] In the scheme, the correction features correspond to the target prediction dimensions one by one, the to-be-processed text does not have the authenticity in the target prediction dimensions, the feature correction module is further configured to acquire authenticity scores of the to-be-processed text in each target prediction dimension, determine each authenticity score as a weight of a corresponding correction feature, perform weighted fusion on each correction feature according to the weight of the correction feature, and obtain the reference correction feature; and perform feature correction on the initial text feature based on the reference correction feature to obtain a target text feature corresponding to the initial text feature.

[0029] In the scheme, the feature correction module is further configured to acquire a feature dimension of the initial text feature and a feature dimension of the reference correction feature, adjust the feature dimension of the reference correction feature to obtain a target correction feature when the feature dimension of the initial text feature is different from the feature dimension of the reference correction feature, determine the reference correction feature as the target correction feature when the feature dimension of the initial text feature is the same as the feature dimension of the reference correction feature, determine a correction strength of the initial text feature based on the number of the correction features, the correction strength being positively correlated with the number of the correction features, determine a product of the correction strength and the target correction feature as a fusion feature, and add the initial text feature and the fusion feature to obtain the target text feature.

[0030] In the scheme, the feature decoding module is further configured to acquire a task type of the to-be-processed text and acquire a task prediction network corresponding to the task type, invoke a task prediction network corresponding to an answer prediction task for answering the to-be-processed text when the task type is the answer prediction task, perform answer prediction on the to-be-processed text based on the target text feature to obtain an answer text corresponding to the to-be-processed text, the answer text having the authenticity in each prediction dimension, invoke a task prediction network corresponding to a translation task for translating the to-be-processed text when the task type is the translation task, perform translation on the to-be-processed text based on the target text feature to obtain a translation text corresponding to the to-be-processed text, and the translation text having the authenticity in each prediction dimension.

[0031] An electronic device is provided in an embodiment of the application, and the electronic device comprises:

[0032] A memory is configured to store computer executable instructions or computer programs.

[0033] A processor is configured to execute the computer executable instructions or computer programs stored in the memory to implement the text processing method provided in the embodiments of the application.

[0034] The embodiment of the present application provides a computer readable storage medium, which stores computer executable instructions, and is used for causing a processor to execute the text processing method provided by the embodiment of the present application.

[0035] The embodiment of the present application provides a computer program product, which comprises a computer program or computer executable instructions stored in a computer readable storage medium. A processor of an electronic device reads the computer executable instructions from the computer readable storage medium, and the processor executes the computer executable instructions, so that the electronic device executes the text processing method provided by the embodiment of the present application.

[0036] The embodiment of the present application has the following beneficial effects:

[0037] By performing feature extraction on the to-be-processed text, initial text features of the to-be-processed text are obtained, authenticity of the logic of the to-be-processed text is predicted in at least one prediction dimension based on the initial text features, authenticity prediction results of the to-be-processed text in each prediction dimension are obtained, when the authenticity prediction result indicates that the to-be-processed text does not have authenticity in the corresponding prediction dimension, a correction feature in the corresponding prediction dimension is obtained, the initial text features are corrected based on the correction feature, target text features corresponding to the initial text features are obtained, and the target text features are decoded to obtain target text which has authenticity in each prediction dimension. In this way, authenticity of the to-be-processed text in each prediction dimension is predicted based on the initial text features, authenticity prediction results of the to-be-processed text in each prediction dimension are obtained, the initial text features are corrected to obtain target text features, and the target text features are decoded, so that the target text has authenticity in each prediction dimension, thereby effectively improving accuracy of the target text and effectively improving accuracy of text processing. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 FIG. 1 is an architectural schematic diagram of a text processing system provided by the embodiment of the present application;

[0039] Figure 2 FIG. 4 is a structural schematic diagram of an electronic device for text processing provided by the embodiment of the present application;

[0040] Figures 3-4 FIG. 6 is a flow schematic diagram of a text processing method provided by the embodiment of the present application;

[0041] Figure 5 FIG. 8 is a principle schematic diagram of a text processing method provided by the embodiment of the present application;

[0042] Figures 6-8is a flowchart of a text processing method provided by an embodiment of the present application.

[0043] Figure 9 is a principle diagram of a text processing method provided by an embodiment of the present application.

[0044] Figure 10 is an experimental effect diagram of a text processing method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the objects, technical solutions and advantages of the present application clearer, the following will further describe the present application in conjunction with the accompanying drawings, and the described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0046] In the following description, "some embodiments" are related to a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0047] In the following description, the terms "first\second\third" are only to distinguish similar objects, and do not represent a specific order of the objects, and it can be understood that "first\second\third" can be interchanged in a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0049] Before the embodiments of the present application are further described in detail, the terms and phrases involved in the embodiments of the present application are explained, and the terms and phrases involved in the embodiments of the present application are applicable to the following explanations.

[0050] 1) Artificial Intelligence (AI): is to use digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain optimal results. Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc.

[0051] 2) Convolutional Neural Networks (CNN): These are a type of feedforward neural network (FNN) that incorporates convolutional computations and possesses a deep structure. They are a representative algorithm for deep learning. CNNs possess representation learning capabilities and can perform shift-invariant classification on input images based on their hierarchical structure.

[0052] 3) Machine Learning (ML): This is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specializes in studying how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance.

[0053] 4) In response to: used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed can be in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed are executed.

[0054] 5) Large Language Model (LLM): A large language model is a deep learning model trained using large amounts of text data. It can generate natural language text or understand the meaning of text. Large language models can handle a variety of natural language tasks, such as text classification, question-answering, and conversation, and are an important path to artificial intelligence. Large language models are designed to understand and generate human language. They are trained on large amounts of text data and can perform a wide range of tasks, including text summarization, translation, and sentiment analysis. Large language models are characterized by their large scale and billions of parameters, which help them learn complex patterns in language data. These models are often based on deep learning architectures such as transformers, which contributes to their impressive performance on various natural language processing tasks. Pre-trained models, a key technology for model training in the field of artificial intelligence, are derived from large language models in the field of natural language processing. After fine-tuning, large language models can be widely applied to downstream tasks.

[0055] 6) Natural Language Processing (NLP): is an important direction in the field of computer science and artificial intelligence. It studies the various theories and methods that can realize effective communication between people and computers with natural language. Natural language processing is a scientific field that integrates linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, i.e. the language used in daily life, so it is closely related to the study of linguistics, but also has important differences. Natural language processing is not generally studying natural language, but developing computer systems, especially software systems, that can effectively implement natural language communication. Thus it is part of computer science. Natural language processing is mainly applied to machine translation, public opinion detection, automatic abstract, opinion extraction, text classification, question answering, text semantic comparison, speech recognition, etc.

[0056] In the implementation process of the embodiments of the present application, the applicant finds that the related art has the following problems:

[0057] In the related art, for text processing, the features of the text to be processed are usually extracted and decoded directly to obtain the target text of the text to be processed. As a result of the hallucination phenomenon in the text processing process, the target text does not have authenticity in the prediction dimension, thereby resulting in low accuracy of the determined target text and low accuracy of the text processing.

[0058] The embodiments of the present application provide a text processing method, device, electronic equipment, computer readable storage medium and computer program product, which can effectively improve the accuracy of text processing. The exemplary application of the text processing system provided by the embodiments of the present application is described below.

[0059] Referring to Figure 1 , Figure 1 is an architecture schematic diagram of the text processing system 100 provided by the embodiments of the present application. The terminal (exemplarily shows the terminal 400) connects the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0060] The terminal 400 is used for a user to use the client 410 to display the target text on the graphical interface 410-1 (exemplarily shows the graphical interface 410-1). The terminal 400 and the server 200 are connected to each other through wired or wireless networks.

[0061] In some embodiments, the server 200 can be a stand-alone physical server, a server cluster composed of multiple physical servers, or a distributed system, and can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal 400 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart television, a smart watch, a vehicle-mounted terminal, and the like, but is not limited thereto. The electronic device provided by the embodiments of the present application can be implemented as a terminal or a server. The terminal and the server can be connected directly or indirectly through wired or wireless communication, which is not limited in the embodiments of the present application.

[0062] In some embodiments, the server 200 extracts features from the to-be-processed text to obtain initial text features of the to-be-processed text, determines target text features corresponding to the initial text features, decodes the target text features to obtain a target text corresponding to the to-be-processed text, and sends the target text to the terminal 400.

[0063] In some other embodiments, the terminal 400 extracts features from the to-be-processed text to obtain initial text features of the to-be-processed text, determines target text features corresponding to the initial text features, decodes the target text features to obtain a target text corresponding to the to-be-processed text, and sends the target text to the server 200.

[0064] In some other embodiments, the embodiments of the present application can be implemented by means of cloud technology. Cloud technology refers to a kind of hosting technology that unifies a series of resources such as hardware, software, and network in a wide area network or a local area network to realize data calculation, storage, processing, and sharing.

[0065] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology applied based on cloud computing business model, can form a resource pool, and can be used on demand, flexibly and conveniently. Cloud computing technology will become an important support. The background service of the technical network system needs a large amount of computing and storage resources.

[0066] Referring to Figure 2 , Figure 2 is a structural schematic diagram of an electronic device 500 for text processing provided by the embodiments of the present application, wherein Figure 2 The electronic device 500 shown in the figure can be a server 200 or a terminal 400 in Figure 1 Figure 2 ​The electronic device 500 shown includes at least one processor 430, a memory 450, at least one network interface 420. The various components in the electronic device 500 are coupled together by a bus system 440. It is understood that the bus system 440 is used for communicating data between the components. The bus system 440 includes a data bus, a power bus, a control bus, and a state signal bus. However, for clarity, only the data bus is shown in Figure 2

[0067] The processor 430 can be an integrated circuit chip that has processing capability, such as a general purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc.

[0068] The memory 450 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 450 optionally includes one or more storage devices remotely located from the processor 430.

[0069] The memory 450 includes volatile memory or non-volatile memory, and can also include both volatile and non-volatile memory. Non-volatile memory can be read only memory (ROM), and volatile memory can be random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0070] In some embodiments, the memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, which are described below.

[0071] The operating system 451 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks.

[0072] The network communication module 452 is used to communicate with other electronic devices via one or more (wired or wireless) network interfaces 420, examples of which include Bluetooth, wireless compatibility authentication (WiFi), and universal serial bus (USB), etc.

[0073] ​In some embodiments, the text processing apparatus provided by the embodiments of the present application can be implemented in a software manner, Figure 2 A text processing apparatus 455 stored in the memory 450 is shown, which can be software in the form of programs and plug-ins, etc., including the following software modules: a feature extraction module 4551, a authenticity prediction module 4552, an acquisition module 4553, a feature correction module 4554, and a feature decoding module 4555, which are logical, and thus can be combined or further split according to the implemented functions. The functions of the various modules will be described below.

[0074] In some embodiments, the text processing apparatus provided by the embodiments of the present application can be implemented in a hardware manner. As an example, the text processing apparatus provided by the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to perform the text processing method provided by the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can use one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other electronic elements.

[0075] In some embodiments, the terminal or server can implement the text processing method provided by the embodiments of the present application by running a computer program or computer executable instructions. For example, the computer program can be a native program (e.g., a dedicated text processing program) in the operating system or a software module, for example, a deblurring module embedded in any program (such as an instant messaging client, a photo album program, an electronic map client, a navigation client); for example, it can be a native (Native) application (APP), i.e., a program that needs to be installed in the operating system to run. In summary, the above computer program can be any form of application program, module or plug-in.

[0076] The text processing method provided by the embodiments of the present application will be described in conjunction with exemplary applications and implementations of the server or terminal provided by the embodiments of the present application.

[0077] Referring to Figure 3 , Figure 3 is a flowchart of the text processing method provided by the embodiments of the present application. The text processing method provided by the embodiments of the present application will be described in conjunction with Figure 3The steps 101 to 105 are shown to illustrate that the text processing method provided by the embodiments of the present application can be implemented by a server or a terminal alone, or by a server and a terminal in cooperation. The following will be described by taking the implementation of the server alone as an example.

[0078] In step 101, feature extraction is performed on the to-be-processed text to obtain initial text features of the to-be-processed text.

[0079] In some embodiments, the feature extraction mentioned above refers to a processing process of converting the to-be-processed text in the form of text into initial text features in the form of a vector. In machine learning, pattern recognition and image processing, feature extraction starts from an initial set of measurement data, and establishes derived values (features) that aim to provide information and non-redundancy, so as to facilitate subsequent learning and generalization steps, and in some cases, better interpretability. Feature extraction is related to dimensionality reduction. The quality of features has a crucial influence on the generalization ability.

[0080] In some embodiments, the feature extraction mentioned above is implemented by at least one feature extraction network. When the number of feature extraction networks is one, the step 101 mentioned above can be implemented in the following manner: calling the feature extraction network, performing feature extraction on the to-be-processed text, and obtaining initial text features of the to-be-processed text.

[0081] In some embodiments, the feature extraction network mentioned above can be implemented by an encoding network. The encoding network can be a machine learning network with a multi-head self-attention network as a network framework. The specific implementation manner of the feature extraction network mentioned above does not constitute a limitation on the embodiments of the present application.

[0082] In some embodiments, the feature extraction mentioned above is implemented by at least one feature extraction network. For details, refer to Figure 4 , Figure 4 is a flowchart of the text processing method provided by the embodiments of the present application. When the number of feature extraction networks is multiple, the extraction scales of the feature extraction networks are different, Figure 3 The step 101 shown can be implemented by Figure 4 The steps 1011 to 1013 shown.

[0083] In step 1011, the first feature extraction network is called to perform feature extraction on the to-be-processed text to obtain first initial text features.

[0084] For example, refer to Figure 5 , Figure 5 is a principle diagram of the text processing method provided by the embodiments of the present application. The first feature extraction network 51 is called to perform feature extraction on the to-be-processed text to obtain first initial text features.

[0085] In step 1012, the following processing is performed for i: calling the ith feature extraction network, performing feature extraction on the to-be-processed text based on the (i-1)th initial text feature to obtain the ith initial text feature.

[0086] In some embodiments, 1 < i ≤ N, where N is used to indicate the number of feature extraction networks.

[0087] As an example, referring to Figure 5 , the second feature extraction network 52 is called to perform feature extraction on the to-be-processed text based on the first initial text feature to obtain the second initial text feature, and the nth feature extraction network 5n is called to perform feature extraction on the to-be-processed text based on the (n-1)th initial text feature to obtain the nth initial text feature.

[0088] In some embodiments, before performing the above step 1012, the (i-1)th target text feature can be determined by: performing authenticity prediction on the logic of the to-be-processed text in each prediction dimension based on the (i-1)th initial text feature to obtain (i-1)th authenticity prediction results of the to-be-processed text in each prediction dimension; and performing feature checking on the (i-1)th initial text feature based on the (i-1)th authenticity prediction results to obtain the (i-1)th target text feature.

[0089] In some embodiments, the above authenticity prediction on the logic of the to-be-processed text in each prediction dimension based on the (i-1)th initial text feature to obtain (i-1)th authenticity prediction results of the to-be-processed text in each prediction dimension can be implemented by: obtaining authenticity prediction networks respectively corresponding to each of the prediction dimensions, and performing the following processing for each of the prediction dimensions: calling the corresponding authenticity prediction network, performing authenticity prediction on the logic of the to-be-processed text in the prediction dimension based on the (i-1)th initial text feature to obtain an (i-1)th authenticity score of the to-be-processed text in the prediction dimension; when the authenticity score is greater than or equal to a score threshold, determining the authenticity prediction result of the prediction dimension as a first result; and when the authenticity score is less than the score threshold, determining the authenticity prediction result of the prediction dimension as a second result.

[0090] In some embodiments, the first result is used to indicate that the to-be-processed text has authenticity in the prediction dimension, and the second result is used to indicate that the to-be-processed text does not have authenticity in the prediction dimension.

[0091] In some embodiments, when the number of the prediction dimensions is one, the obtaining of the authenticity prediction network corresponding to each prediction dimension can be implemented in the following manner: an initial prediction network is obtained, a plurality of text feature samples corresponding to a text sample are obtained, and authenticity label scores of each text feature sample are obtained; for each text feature sample, the initial prediction network is invoked, authenticity prediction of the logic of the text sample is performed in the prediction dimension based on the text feature sample, an authenticity score corresponding to the text feature sample is obtained, and a loss value corresponding to the text feature sample is determined in combination of the authenticity score and the corresponding authenticity label score; the initial prediction network is trained based on the loss value corresponding to each text feature sample, and the authenticity prediction network corresponding to the prediction dimension is obtained.

[0092] In some embodiments, when the number of the prediction dimensions is multiple, the obtaining of the authenticity prediction network corresponding to each prediction dimension can be implemented in the following manner: an initial prediction network is obtained, a first text feature sample corresponding to a text sample of a first prediction dimension is obtained, and a first authenticity label score of the first text feature sample is obtained; the initial prediction network is invoked, authenticity prediction of the logic of the text sample of the first prediction dimension is performed in the first text feature sample, a first authenticity score is obtained, and the initial prediction network is trained in combination of the first authenticity score and the first authenticity label score, to obtain the authenticity prediction network corresponding to the first prediction dimension; the following processing is performed by traversing j: a (j-1)th authenticity score corresponding to a text sample of a (j-1)th prediction dimension is obtained, the initial prediction network is trained based on the (j-1)th authenticity score, and the authenticity prediction network corresponding to a jth prediction dimension is obtained.

[0093] In some embodiments, 2≤j≤M, and M is used to indicate the number of the prediction dimensions.

[0094] In some embodiments, the obtaining of the (i-1)th target text feature based on the (i-1)th authenticity prediction result can be implemented in the following manner: when there is an (i-1)th authenticity prediction result indicating that the to-be-processed text does not have authenticity in the corresponding prediction dimension, the (i-1)th initial text feature is modified to obtain the (i-1)th target text feature; and when each (i-1)th authenticity prediction result indicates that the to-be-processed text has authenticity in the corresponding prediction dimension, the (i-1)th initial text feature is determined as the (i-1)th target text feature.

[0095] As an example, the i-1th authenticity prediction result corresponds to the prediction dimension one by one, the prediction dimension includes prediction dimension A, prediction dimension B and prediction dimension C, the i-1th authenticity prediction result includes the prediction result corresponding to the prediction dimension A, the prediction result corresponding to the prediction dimension B and the prediction result corresponding to the prediction dimension C, the prediction result corresponding to the prediction dimension A indicates that the to-be-processed text does not have authenticity under the prediction dimension A, the prediction result corresponding to the prediction dimension B indicates that the to-be-processed text has authenticity under the prediction dimension B, and the prediction result corresponding to the prediction dimension C indicates that the to-be-processed text does not have authenticity under the prediction dimension C, that is, there is the i-1th authenticity prediction result indicating that the to-be-processed text does not have authenticity under the corresponding prediction dimension, at this time, the i-1th initial text feature needs to be corrected to obtain the i-1th target text feature.

[0096] As an example, the i-1th authenticity prediction result corresponds to the prediction dimension one by one, the prediction dimension includes prediction dimension A, prediction dimension B and prediction dimension C, the i-1th authenticity prediction result includes the prediction result corresponding to the prediction dimension A, the prediction result corresponding to the prediction dimension B and the prediction result corresponding to the prediction dimension C, the prediction result corresponding to the prediction dimension A indicates that the to-be-processed text does not have authenticity under the prediction dimension A, the prediction result corresponding to the prediction dimension B indicates that the to-be-processed text has authenticity under the prediction dimension B, and the prediction result corresponding to the prediction dimension C indicates that the to-be-processed text does not have authenticity under the prediction dimension C, that is, there is the i-1th authenticity prediction result indicating that the to-be-processed text does not have authenticity under the corresponding prediction dimension, at this time, the i-1th initial text feature needs to be corrected to obtain the i-1th target text feature.

[0097] In some embodiments, the above step 1012 can be implemented by calling the i-th feature extraction network, performing feature extraction on the to-be-processed text based on the i-1th target text feature to obtain the i-th initial text feature.

[0098] In some embodiments, before calling the i-th feature extraction network, the i-1th target text feature is obtained by performing feature checking on the i-1th initial text feature, so that the i-th feature extraction network is called to perform feature extraction on the to-be-processed text based on the i-1th target text feature to obtain the i-th initial text feature, so that in the plurality of feature extraction networks, the feature checking is performed layer by layer to ensure that the input of each layer of feature extraction network is the target text feature after strict feature checking, so that the feature extraction network can gradually realize the optimization of feature extraction on the to-be-processed text, thereby effectively improving the accuracy of feature extraction.

[0099] In step 1013, the Nth initial text feature is determined as the initial text feature of the to-be-processed text.

[0100] As an example, refer to Figure 5 The Nth initial text feature (i.e., the output of the Nth feature extraction network 5n) is determined as the initial text feature of the text to be processed.

[0101] In this way, before calling the i-th feature extraction network, the i-1th target text feature is obtained by performing feature inspection on the i-1th initial text feature, so as to call the i-th feature extraction network, perform feature extraction on the text to be processed based on the i-1th target text feature, and obtain the i-th initial text feature, so that in the plurality of feature extraction networks, the feature inspection is performed layer by layer to ensure that the input of each layer of feature extraction network is the target text feature after strict feature inspection, so that the feature extraction network can gradually realize the optimization of feature extraction on the text to be processed, thereby effectively improving the accuracy of feature extraction.

[0102] In step 102, based on the initial text feature, the authenticity of the logic of the text to be processed is predicted in at least one prediction dimension, and the authenticity prediction result of the text to be processed in each prediction dimension is obtained.

[0103] In some embodiments, the authenticity prediction described above can be realized by an authenticity prediction network, and the authenticity prediction network corresponds to the prediction dimension one by one, that is, the authenticity prediction networks corresponding to different prediction dimensions are different, the network structures of the authenticity prediction networks under different prediction dimensions are the same, and the network parameters are different. The network structure of the authenticity prediction network can include a convolution layer, a pooling layer, and a normalization layer.

[0104] In some embodiments, the prediction dimension is used to indicate the logical dimension of the text to be processed, and the prediction dimension corresponds to the logical dimension of the text to be processed one by one. The logical dimension of the text to be processed includes various language logic types such as common sense logic and syntax logic. For example, the common sense logic includes that the refrigerator is smaller than the elephant, and the moon has no oxygen.

[0105] In some embodiments, refer to Figure 6 , Figure 6 is a flowchart of a text processing method provided by an embodiment of the present application, Figure 3 The step 102 shown in the figure can be implemented by Figure 6 The steps 1021 to 1024 shown in the figure.

[0106] In step 1021, the authenticity prediction network corresponding to each prediction dimension is obtained, and the following steps 1022 to 1024 are performed for each prediction dimension.

[0107] In some embodiments, the authenticity prediction described above can be implemented by an authenticity prediction network, the authenticity prediction network corresponds to the prediction dimension one by one, that is, different authenticity prediction networks correspond to different prediction dimensions, the network structures of the authenticity prediction networks under different prediction dimensions are the same, and the network parameters are different. The network structure of the authenticity prediction network can include a convolutional layer, a pooling layer, and a normalization layer.

[0108] In some embodiments, when the number of prediction dimensions is one, the authenticity prediction network corresponding to each prediction dimension described above can be obtained by the following method: obtaining an initial prediction network, and obtaining a plurality of text feature samples corresponding to the text sample and authenticity label scores of each text feature sample; for each text feature sample, calling the initial prediction network, performing authenticity prediction on the logic of the text sample in the prediction dimension based on the text feature sample, obtaining an authenticity score corresponding to the text feature sample, and combining the authenticity score and the corresponding authenticity label score to determine a loss value corresponding to the text feature sample; based on the loss value corresponding to each text feature sample, training the initial prediction network to obtain the authenticity prediction network corresponding to the prediction dimension.

[0109] In some embodiments, the authenticity score corresponds to the text feature sample one by one, and the text feature sample corresponds to the authenticity label score one by one. The combination of the authenticity score and the corresponding authenticity label score to determine the loss value corresponding to the text feature sample can be implemented by the following method: subtracting the authenticity label score of the text feature sample from the authenticity score of the text feature sample to obtain the loss value corresponding to the text feature sample.

[0110] As an example, the expression of the loss value of the text feature sample can be:

[0111] L1 = F1 - F2 (1)

[0112] Wherein, L1 is used to indicate the loss value of the text feature sample, F1 is used to indicate the authenticity score of the text feature sample, and F2 is used to indicate the authenticity label score of the text feature sample.

[0113] In some embodiments, the obtaining of the plurality of text feature samples corresponding to the text sample can be implemented by the following method: obtaining the text sample, and performing feature extraction on the text sample to obtain initial text features of the text sample; performing feature splitting on the initial text features of the text sample to obtain the plurality of text feature samples corresponding to the text sample.

[0114] In some embodiments, the feature splitting on the initial text feature of the text sample to obtain the plurality of text feature samples corresponding to the text sample can be implemented in the following manner: the following processing is respectively performed on each feature character in the initial text feature: the feature character is determined as a target feature character, and the target feature character and other feature characters in the initial text feature are randomly combined to obtain at least one text feature sample corresponding to the target feature character.

[0115] In some embodiments, each text feature sample corresponding to the text sample is a sub-feature of the initial text feature of the text sample.

[0116] As an example, the initial text feature can be: 1234567, and the plurality of text feature samples corresponding to the text sample can be: 12, 123, 1234, 12345, 123456, 23, 234, and the like.

[0117] In this way, by performing feature extraction on the text sample to obtain the initial text feature of the text sample, and performing feature splitting on the initial text feature of the text sample to obtain the plurality of text feature samples corresponding to the text sample, the number of training samples of the initial prediction network is effectively expanded, and the prediction performance of the authenticity prediction network obtained by training is effectively improved.

[0118] In some embodiments, when the number of prediction dimensions is a plurality, the above-mentioned obtaining of the authenticity prediction network corresponding to each prediction dimension can be implemented in the following manner: obtaining an initial prediction network, and obtaining a first text feature sample corresponding to a text sample of a first prediction dimension, and a first authenticity label score of the first text feature sample; calling the initial prediction network, performing authenticity prediction on the logic of the text sample of the first prediction dimension based on the first text feature sample to obtain a first authenticity score, and training the initial prediction network in combination with the first authenticity score and the first authenticity label score to obtain an authenticity prediction network corresponding to the first prediction dimension; performing the following processing by traversing j: obtaining a (j-1)th authenticity score corresponding to a text sample of a (j-1)th prediction dimension, and training the initial prediction network based on the (j-1)th authenticity score to obtain an authenticity prediction network corresponding to a jth prediction dimension.

[0119] In some embodiments, 2≤j≤M, and M is used to indicate the number of prediction dimensions.

[0120] In some embodiments, the obtaining the first text feature sample corresponding to the text sample of the first prediction dimension can be achieved by: obtaining the text sample of the first prediction dimension, performing feature extraction on the text sample of the first prediction dimension to obtain initial text features of the text sample of the first prediction dimension, and performing feature splitting on the initial text features of the text sample of the first prediction dimension to obtain the first text feature sample of the text sample of the first prediction dimension.

[0121] In some embodiments, the training the initial prediction network based on the first authenticity score and the first authenticity label score to obtain the authenticity prediction network corresponding to the first prediction dimension can be achieved by: determining the difference between the first authenticity score and the first authenticity label score as a loss value of the first prediction dimension, and training the initial prediction network based on the loss value of the first prediction dimension to obtain the authenticity prediction network corresponding to the first prediction dimension.

[0122] In some embodiments, by obtaining the j-1 authenticity score corresponding to the text sample of the j-1 prediction dimension, training the initial prediction network based on the j-1 authenticity score to obtain the authenticity prediction network corresponding to the j prediction dimension, the authenticity prediction network corresponding to the j prediction dimension can effectively learn from the network parameters of the authenticity prediction network corresponding to the j-1 prediction dimension, so that the prediction direction of the authenticity prediction network corresponding to the j prediction dimension and the prediction direction of the authenticity prediction network corresponding to the j-1 prediction dimension remain orthogonal, thereby effectively improving the prediction independence between the authenticity prediction networks of different prediction dimensions.

[0123] As an example, the first authenticity score corresponding to the text sample of the first prediction dimension is obtained, and the initial prediction network is trained based on the first authenticity score to obtain the authenticity prediction network corresponding to the second prediction dimension; the second authenticity score corresponding to the text sample of the second prediction dimension is obtained, and the initial prediction network is trained based on the second authenticity score to obtain the authenticity prediction network corresponding to the third prediction dimension.

[0124] In some embodiments, the training of the initial prediction network based on the (j-1)-th authenticity score to obtain the authenticity prediction network corresponding to the j-th prediction dimension can be implemented in the following manner: obtaining a j-th text feature sample corresponding to a text sample of the j-th prediction dimension and a j-th authenticity label score of the j-th text feature sample; calling the initial prediction network to perform authenticity prediction on the logic of the text sample of the j-th prediction dimension based on the j-th text feature sample to obtain a j-th authenticity score; determining a first loss value in combination with the j-th authenticity score and the (j-1)-th authenticity score, and determining a second loss value in combination with the j-th authenticity score and the j-th authenticity label score; and training the initial prediction network in combination with the first loss value and the second loss value to obtain the authenticity prediction network corresponding to the j-th prediction dimension.

[0125] In some embodiments, the expression of the first loss value can be as follows:

[0126]

[0127] wherein, L orth is used to indicate the first loss value, θ t is used to indicate the j-th authenticity score, θ r is used to indicate the (j-1)-th authenticity score.

[0128] In some embodiments, the expression of the second loss value can be as follows:

[0129] L2=F3-F4 (3)

[0130] wherein, L2 is used to indicate the second loss value, F3 is used to indicate the j-th authenticity score, and F4 is used to indicate the j-th authenticity label score.

[0131] In some embodiments, the training of the initial prediction network in combination with the first loss value and the second loss value to obtain the authenticity prediction network corresponding to the j-th prediction dimension can be implemented in the following manner: summing the first loss value and the second loss value to obtain a total loss of the j-th prediction dimension, and training the initial prediction network based on the total loss to obtain the authenticity prediction network corresponding to the j-th prediction dimension.

[0132] Thus, by obtaining the j-1th authenticity score corresponding to the text sample of the j-1th prediction dimension, training the initial prediction network based on the j-1th authenticity score, and obtaining the authenticity prediction network corresponding to the jth prediction dimension, the authenticity prediction network corresponding to the jth prediction dimension can effectively learn from the network parameters of the authenticity prediction network corresponding to the j-1th prediction dimension, and the prediction direction of the authenticity prediction network corresponding to the jth prediction dimension is orthogonal to the prediction direction of the authenticity prediction network corresponding to the j-1th prediction dimension, thereby effectively improving the prediction independence between authenticity prediction networks of different prediction dimensions.

[0133] In step 1022, the corresponding authenticity prediction network is called, and the authenticity of the logic of the to-be-processed text in the prediction dimension is predicted based on the initial text features to obtain the authenticity score of the to-be-processed text in the prediction dimension.

[0134] In some embodiments, the above step 1022 can be implemented in the following manner: for each prediction dimension, the authenticity prediction network corresponding to the prediction dimension is called, the authenticity of the logic of the to-be-processed text in the prediction dimension is predicted based on the initial text features to obtain the authenticity score of the to-be-processed text in the prediction dimension.

[0135] As an example, the prediction dimensions include prediction dimension A, prediction dimension B, and prediction dimension C, the authenticity prediction network corresponding to the prediction dimension A is called, the authenticity of the logic of the to-be-processed text in the prediction dimension A is predicted based on the initial text features to obtain the authenticity score of the to-be-processed text in the prediction dimension A; the authenticity prediction network corresponding to the prediction dimension B is called, the authenticity of the logic of the to-be-processed text in the prediction dimension B is predicted based on the initial text features to obtain the authenticity score of the to-be-processed text in the prediction dimension B; the authenticity prediction network corresponding to the prediction dimension C is called, the authenticity of the logic of the to-be-processed text in the prediction dimension C is predicted based on the initial text features to obtain the authenticity score of the to-be-processed text in the prediction dimension C.

[0136] In step 1023, when the authenticity score is greater than or equal to the score threshold, the authenticity prediction result of the prediction dimension is determined as the first result.

[0137] In some embodiments, the above first result is used to indicate that the to-be-processed text has authenticity in the prediction dimension.

[0138] In some embodiments, the above score threshold can be specifically set according to the actual application scenario, and the score threshold is used to determine whether the to-be-processed text has authenticity in the prediction dimension.

[0139] In step 1024, when the authenticity score is less than the score threshold, the authenticity prediction result of the prediction dimension is determined as the second result.

[0140] In some embodiments, the second result described above is used to indicate that the text to be processed does not have authenticity in the prediction dimension.

[0141] In some embodiments, after step 102 described above, the target text can also be determined by the following manner: when the authenticity prediction results of each prediction dimension all indicate that the text to be processed has authenticity in the corresponding prediction dimension, the initial text feature is subjected to feature decoding to obtain the target text corresponding to the text to be processed.

[0142] In some embodiments, when the authenticity prediction results of each prediction dimension all indicate that the text to be processed has authenticity in the corresponding prediction dimension, it indicates that the text obtained by decoding the initial text feature can have authenticity in each prediction dimension, and at this time, the text obtained by decoding the initial text feature can be determined as the target text corresponding to the text to be processed.

[0143] In step 103, when the authenticity prediction result indicates that the text to be processed does not have authenticity in the corresponding prediction dimension, the modified feature of the initial text feature in the corresponding prediction dimension is obtained.

[0144] As an example, the prediction dimension corresponds to the authenticity prediction result one by one, when the authenticity prediction result indicates that the text to be processed does not have authenticity in prediction dimension A, the modified feature of the initial text feature in prediction dimension A is obtained, and when the authenticity prediction result indicates that the text to be processed does not have authenticity in prediction dimension B, the modified feature of the initial text feature in prediction dimension B is obtained.

[0145] In some embodiments, the modified feature described above is used to modify the corresponding feature dimension of the initial text feature, so that the text obtained by decoding the modified initial text feature can have authenticity in the corresponding feature dimension.

[0146] In some embodiments, the modified feature of the initial text feature in the corresponding prediction dimension described above can be realized by the following manner: the dimension-feature mapping relationship is obtained, when the authenticity prediction result indicates that the text to be processed does not have authenticity in the corresponding prediction dimension, the corresponding prediction dimension is determined as a target prediction dimension, a target index entry including the target prediction dimension is queried from the dimension-feature mapping relationship, and the feature in the target index entry is determined as the modified feature of the target prediction dimension.

[0147] In step 104, the initial text feature is subjected to feature modification based on the modified feature to obtain the target text feature corresponding to the initial text feature.

[0148] In some embodiments, the feature modification is used to modify the initial text feature, so that the target text feature obtained by feature decoding has authenticity in each prediction dimension.

[0149] In some embodiments, when the number of modified features is one, the modified feature is used as the reference modified feature. Figure 3 The step 104 shown can be implemented by modifying the initial text feature based on the reference modified feature to obtain the target text feature corresponding to the initial text feature.

[0150] In some embodiments, referring to Figure 7 , Figure 7 is a flowchart of a text processing method provided by an embodiment of the present application. The modified feature corresponds to the target prediction dimension one-to-one, and the to-be-processed text does not have authenticity in the target prediction dimension. When the number of modified features is multiple, Figure 3 The step 104 shown can be implemented by Figure 7 The steps 1041 to 1043 shown.

[0151] In step 1041, the authenticity scores of the to-be-processed text in each target prediction dimension are obtained, and each authenticity score is determined as the weight of the corresponding modified feature.

[0152] As an example, the modified feature corresponds to the target prediction dimension one-to-one, the target prediction dimension includes prediction dimension 1, prediction dimension 2 and prediction dimension 3, the authenticity score 11 of the to-be-processed text in the target prediction dimension 1 is obtained, the authenticity score 21 of the to-be-processed text in the target prediction dimension 2 is obtained, the authenticity score 31 of the to-be-processed text in the target prediction dimension 3 is obtained, the authenticity score 11 is determined as the weight of the modified feature corresponding to the target prediction dimension 1, the authenticity score 21 is determined as the weight of the modified feature corresponding to the target prediction dimension 2, and the authenticity score 31 is determined as the weight of the modified feature corresponding to the target prediction dimension 3.

[0153] In step 1042, each modified feature is weighted and fused according to the weight of each modified feature to obtain the reference modified feature.

[0154] As an example, the expression of the reference modified feature can be:

[0155] T = ω1T1 + ω2T2 + … ω t T t (4)

[0156] Wherein, T is used to indicate the reference modified feature, ω1 to ω t indicate the weight of each modified feature, T1 to T t indicate the modified feature.

[0157] In step 1043, based on the reference correction feature, the initial text feature is corrected to obtain the target text feature corresponding to the initial text feature.

[0158] In some embodiments, the above step 1043 can be implemented in the following way: obtaining the feature dimension of the initial text feature and the feature dimension of the reference correction feature; when the feature dimension of the initial text feature is different from the feature dimension of the reference correction feature, adjusting the feature dimension of the reference correction feature to obtain a target correction feature; when the feature dimension of the initial text feature is the same as the feature dimension of the reference correction feature, determining the reference correction feature as the target correction feature; based on the number of correction features, determining the correction strength of the initial text feature, the correction strength being positively correlated with the number of correction features; multiplying the correction strength and the target correction feature to determine a fusion feature, and adding the initial text feature and the fusion feature to obtain the target text feature.

[0159] In some embodiments, when the feature dimension of the initial text feature is different from the feature dimension of the reference correction feature, the feature dimension of the reference correction feature is adjusted to obtain a target correction feature, and the feature dimension of the target correction feature is the same as the feature dimension of the initial text feature.

[0160] As an example, the expression of the above target text feature can be:

[0161] Q = Q1 + Q2 (5)

[0162] Wherein, Q is used to indicate the target text feature, Q1 is used to indicate the initial text feature, and Q2 is used to indicate the fusion feature.

[0163] As an example, the expression of the above fusion feature can be:

[0164] Q2 = αT m (6)

[0165] Wherein, Q2 is used to indicate the fusion feature, α is used to indicate the correction strength, and T m is used to indicate the target correction feature.

[0166] In this way, by correcting the corresponding feature dimension of the initial text feature, the text obtained by decoding the corrected initial text feature in the corresponding feature dimension has authenticity, thereby effectively improving the accuracy of the generated target text.

[0167] In step 105, the target text feature is decoded to obtain the target text corresponding to the text to be processed, and the target text has authenticity in each prediction dimension.

[0168] In some embodiments, referring to Figure 8 , Figure 8 is a flowchart of a text processing method provided by an embodiment of the present application, Figure 3 The step 105 shown in the figure can be implemented by Figure 8 The steps 1051 to 1053 shown in the figure.

[0169] In step 1051, the task type of the text to be processed is obtained, and a task prediction network corresponding to the task type is obtained.

[0170] In some embodiments, the task type of the text to be processed can include a translation task type, an opinion detection task type, an automatic summary task type, a viewpoint extraction task type, a text classification task type, a question answering task type, a text semantic comparison task type, a speech recognition task type, and various natural language processing task types, and the task type corresponds to the task prediction network one by one.

[0171] In step 1052, when the task type is an answer prediction task for answering the text to be processed, the task prediction network corresponding to the answer prediction task is called, the text to be processed is predicted based on the target text feature, and the answer text corresponding to the text to be processed is obtained.

[0172] In some embodiments, the answer text has authenticity under each prediction dimension.

[0173] In some embodiments, the network structure of the task prediction network can include a convolution layer and a prediction layer, and the above calling the task prediction network corresponding to the answer prediction task, predicting the text to be processed based on the target text feature, and obtaining the answer text corresponding to the text to be processed can be implemented by: calling the convolution layer of the task prediction network corresponding to the answer prediction task, performing feature convolution on the target text feature to obtain target convolution feature, calling the prediction layer of the task prediction network corresponding to the answer prediction task, performing text prediction on the target text feature to obtain the answer text corresponding to the text to be processed.

[0174] As an example, the text to be processed is "How old are you today?", and the answer text corresponding to the text to be processed can be "I am 26 years old".

[0175] In step 1053, when the task type is a translation task for translating the text to be processed, the task prediction network corresponding to the translation task is called, the text to be processed is translated based on the target text feature, and the translation text corresponding to the text to be processed is obtained.

[0176] In some embodiments, the translation text has authenticity under each prediction dimension.

[0177] In some embodiments, the network structure of the task prediction network can include a convolutional layer and a prediction layer, and the task prediction network corresponding to the calling translation task is used to translate the to-be-processed text based on the target text feature to obtain the translation text corresponding to the to-be-processed text, which can be realized by the following manner: the convolutional layer of the task prediction network corresponding to the calling translation task is called to perform feature convolution on the target text feature to obtain target convolutional features, and the prediction layer of the task prediction network corresponding to the calling translation task is called to perform text prediction on the target text feature to obtain the translation text corresponding to the to-be-processed text.

[0178] For example, the to-be-processed text is "How old are you this year?", and the translation text corresponding to the to-be-processed text can be "You how old are you this year?".

[0179] In this way, by performing feature extraction on the to-be-processed text to obtain initial text features of the to-be-processed text, based on the initial text features, the authenticity of the logic of the to-be-processed text is predicted in at least one prediction dimension to obtain authenticity prediction results of the to-be-processed text in each prediction dimension, when the authenticity prediction result indicates that the to-be-processed text does not have authenticity in the corresponding prediction dimension, the correction feature in the corresponding prediction dimension is obtained, based on the correction feature, the initial text feature is corrected to obtain target text features corresponding to the initial text features, and the target text features are decoded to obtain target text that has authenticity in each prediction dimension. In this way, by predicting the authenticity of the logic of the to-be-processed text in at least one prediction dimension based on the initial text features to obtain authenticity prediction results of the to-be-processed text in each prediction dimension, respectively, by correcting the initial text features to obtain target text features, and decoding the target text features, the target text has authenticity in each prediction dimension, thereby effectively improving the accuracy of the target text and effectively improving the accuracy of the text processing.

[0180] In the following, an exemplary application of the embodiments of the present application in an actual application scenario of answering questions will be described.

[0181] In recent years, pre-training language models based on Transformer have achieved remarkable success in natural language processing tasks. However, these models often generate unrealistic information in the generation task. Through the text processing method provided by the embodiments of the present application, the multi-directional probe can be used to identify the direction of the model pointing to the real direction and intervene in the generation process of the language model to improve the authenticity of the generation result.

[0182] For clarity of notation and context, some key elements of the Transformer architecture will be briefly introduced below, and the multi-head attention mechanism (MHA) will be considered as a way to add attention-weighted vectors independently to the residual flow.

[0183] The core component of the Transformer is a sequence of Transformer layers of the same size. The variable l is used in the embodiments of the present application to represent these layers. Each Transformer layer contains two key modules: one is the multi-head attention (MHA) mechanism, and the other is the standard multi-layer perceptron (MLP) layer. The MHA layer is mainly introduced in the embodiments of the present application, which is the position of the TrFr implementation training probe and intervention generation.

[0184] In each Transformer layer, MHA consists of H independent linear operations, while MLP is responsible for all nonlinear operations. Specifically, MHA can be represented as:

[0185]

[0186] where, map the activation to a low-dimensional head space (Head) of dimension D, map it back to the original high-dimensional space. Att is an attention computation operator that communicates with other input tokens. The probe training and intervention in the embodiments of the present application occur after Att, Before that, the activation is represented by x l ∈R D .

[0187] The text processing method provided by the embodiments of the present application aims to identify the real direction inside the large model through multi-directional probes, and intervene in the generation process of the large language model to improve the authenticity of the generated results.

[0188] The embodiments of the present application are based on the following assumptions: the internal states of the large model are different when outputting real and illusory content. Specifically, when the large model inputs a text sequence, the neural network inside the large model will generate some implicit vector outputs (the head output of the multi-head attention is used as a probe for the authenticity of the large model in the embodiments of the present application). The embodiments of the present application judge whether the generated content of the large model at this time is real or illusory based on these implicit vector outputs. These probes can also assist the model in intervening in the authenticity of the results generated by the large model, so that the large model can make relevant but more objective and real replies.

[0189] In some embodiments, referring to Figure 9 , Figure 9is a principle schematic diagram of a text processing method provided by an embodiment of the present application, which focuses on the head in the multi-head attention (MHA) which is the smallest state unit in the Transformer, and targets positioning and intervention. The embodiment of the present application introduces multiple probes for each head (such as the first layer, the second layer, and the nth layer shown in Figure 9 The embodiment of the present application forces an orthogonal constraint between the probes to prevent model collapse. The embodiment of the present application optimizes the orthogonal probes and introduces an orthogonality loss function to maintain the orthogonality between the probes. The internal state probe research of the language model shows that the language model often has the ability to distinguish between lies and truths, but cannot effectively generate facts. The embodiment of the present application extracts features by considering the extended range in the sequence. Specifically, the embodiment of the present application samples from a predefined distribution and truncates the sequence at different positions to obtain different features, so that the learned direction is more stable and can be generalized to different positions in the generation process. After the training is completed, the orthogonal vector pointing to the truth can be obtained. The embodiment of the present application calculates the final orthogonal vector by using an exponentially decaying weight, sorts the heads, and obtains the final intervention vector. When intervening in the MHA layer, it is modified to a constant. Since the extra term of each step is a constant, the time complexity of TrFr is O(1).

[0190] In some embodiments, referring to Figure 9 For Figure 9 the probes shown, for each head, the embodiment of the present application introduces a classifier as a probe, the input is wherein is the result of l2-norm.

[0191] The embodiment of the present application introduces multiple probes for each head and forces an orthogonal constraint between the probes:

[0192] Θ={θ1,θ2,...,θ k},θ i ⊥θ j ,i≠j (8)

[0193] The embodiment of the present application optimizes the orthogonal probes and introduces an orthogonality loss function:

[0194]

[0195] By minimizing the loss, the embodiment of the present application encourages the probes to remain orthogonal to each other, thereby capturing different aspects of the model's internal representation of truth. The total loss function of each probe is:

[0196] L total =L ce +λL orth+ μL2 (10)

[0197] By adjusting λ and μ, embodiments of the present application can control the trade-off between the accuracy and orthogonality of the probe.

[0198] Embodiments of the present application extract features by considering the range of expansion in the sequence. Specifically, embodiments of the present application sample from a predefined distribution and truncate the sequence at different positions to obtain different features, so that the learned direction is more stable and can be generalized to different positions in the generation process.

[0199] In some embodiments, embodiments of the present application define D to be a certain hallucination question and answer data set, each question containing false and correct answers; Φ can be an arbitrarily defined distribution, and the Transformer is a generation model to be intervened by embodiments of the present application.

[0200] As an example, the pseudo code for extracting features by considering the range of expansion in the sequence provided by embodiments of the present application is described in detail as follows:

[0201] Input: data D, LM, predefined distribution num_layers, num_heads;

[0202] Output: MHA feature F;

[0203] Initialize function list: F;

[0204] For each (Q, A) ∈ D do;

[0205]

[0206] S = (Q, A), S1 = (Q, A1), where A1 = (a1, a2, …, az), and A = (a1, a2, …, aL);

[0207] For each layer l in range(num_layer) do;

[0208] For each head h in range(num_heads) do;

[0209] Transformer(S1) = Xh;

[0210] Append Xh to F;

[0211] End for;

[0212] End for;

[0213] End for;

[0214] Return F;

[0215] In some embodiments, after the training is completed, the embodiments of the present application obtain the orthogonal vectors pointing to the authenticity. The embodiments of the present application calculate the final orthogonal vector by using the exponential decay weight, and sort the heads to obtain the final intervention vector:

[0216]

[0217] where w k is the weight factor, θ l，h，k is the kth orthogonal vector at position (l, h).

[0218] When intervention is performed in the MHA layer, the embodiments of the present application modify it to a constant:

[0219]

[0220] where x l and x l+1 represent the input and output of the lth layer, and are MHA components, H is the number of heads, and a is the intervention strength, is the standard deviation of the direction module length before l2-norm normalization calculated by the embodiments of the present application using another same distribution data set to restore, is the effective intervention vector of the probe after Top-K screening of the accuracy. Since the intervention term of each step is a constant, the time complexity of using TrFr is O(1).

[0221] In some embodiments, a question and answer dataset is obtained, which contains questions and their corresponding commonly correct or incorrect answers. The embodiments of the present application first extract features using a language model to be intervened (such as LLaMA-7B). Then, the embodiments of the present application select a suitable sampling distribution according to hyperparameters, use the random peeking method to truncate the input sequence at different positions, and obtain the features of each position inside the language model after inputting the model. After that, the embodiments of the present application train probes that are orthogonal to each other at each position using these features. The embodiments of the present application can capture different aspects of the model's internal representation of truth. After the training is completed, the embodiments of the present application obtain orthogonal vectors pointing to truth. After integrating the directions of the probe set, the final intervention vector is obtained. In the generation task, that is, in actual reasoning, the embodiments of the present application select effective probes for intervention (the embodiments of the present application obtain effective probes through some strategies such as threshold or topk screening), and through the embodiments of the present application, the truthfulness of the generated results can be controlled by adjusting the intervention strength. For example, when the embodiments of the present application want to generate an answer that is related to the question and neutral in truth, a larger intervention strength can be used. In this way, the generated answer will be more able to avoid hallucination results.

[0222] In this way, the embodiments of the present application improve the truthfulness of the generated results through multi-directional intervention. The text processing method provided by the embodiments of the present application can include using orthogonal probes to represent truth, using the random peeking method to alleviate the generation-discrimination gap, and implementing the truth forest and intervention process. By training orthogonal probes, the embodiments of the present application can capture different aspects of the model's internal representation of truth, and the random peeking method makes the learned directions more stable and can be generalized to different positions in the generation process. In the generation task, the embodiments of the present application can control the truthfulness of the generated results by adjusting the intervention strength, and in actual applications, the embodiments of the present application can be applied to various natural language processing tasks such as text generation, question and answer systems, dialogue systems, etc. In addition, the embodiments of the present application can also be used in combination with other generation models (such as other large models, etc.) to further improve the truthfulness and reliability of the generated results.

[0223] The effectiveness of the method is tested in open source datasets and open source models, and compared with mainstream fact-enhanced schemes. By selecting an experimental dataset that is strongly related to the illusion problem, and using a variety of open source models for experiments. In this experiment, the index of the present embodiment can be: determination index (True%): if the GPT model determines that the answer given by the language model is false, it is 0, otherwise it is 1, and the average is calculated in all questions and answers. Detailed level index (True*Info%): Info% measures the detail level of the language model answer, and Info% is generated in the same way and multiplied by True%, which prevents high True% caused by the language model continuously refusing to answer. Probability index (MC%): calculate the generation probability of each TruthfulQA given Ground True answer, if the correct answer is ranked first, it is 1, otherwise it is 0, and then calculate the average of all samples. First intervention index (CE): calculate the CrossEntropy in the pre-training data, which represents the pre-training task loss of the language model, as one of the indicators to measure the intervention strength. Second intervention index (KL): calculate the distribution distance of each word generation before and after intervention, as one of the indicators to measure the intervention strength.

[0224] As an example, see Table 1 below, Table 1 is an experimental parameter schematic table (1) provided by the present embodiment.

[0225] Table 1 Experimental parameter schematic table (1) provided by the present embodiment

[0226]

[0227] As an example, see Table 2 below, Table 2 is an experimental parameter schematic table (2) provided by the present embodiment.

[0228] Table 2 Experimental parameter schematic table (2) provided by the present embodiment

[0229]

[0230]

[0231] In some embodiments, the base model (Baseline) is the LLaMA-7B model before intervention, RandomDirection is a sample randomly sampled from a normal distribution as the direction, and TOP-K positions are randomly selected for intervention, which serves as a control group; ITI is two methods to improve the factuality of the language model, which serve as the baseline; SupervisedFinetuning is a common downstream fine-tuning scheme in the language model, and Few-shot Prompting is a contextual learning method. In this embodiment of the application, 80 correct question-answer pairs are extracted from TruthfulQA as prompt learning.

[0232] In some embodiments, in order to compare fairly with Few-shot Prompting, all methods in Few-shot setting use only 80 samples. In Full Data setting, the present embodiment uses the full dataset of TruthfulQA for 2-fold cross validation, with the ratio of train:valid:test = 4:1:5 for each fold.

[0233] Experimental results show that the text processing method provided by the embodiment of the present application has achieved significant performance improvements in a variety of scenarios. On a complete data set, the embodiment of the present application has better performance than the implementation method of the related art. Experimental results show that the embodiment of the present application can significantly improve True*Info% with minimal intervention at any stage. The embodiment of the present application was compared with the related art. In a small sample setting, the embodiment of the present application achieved better results while being compatible with FSP. CE and KL results show that the embodiment of the present application achieved better performance with minimal intervention while maintaining the amount of information.

[0234] Table 3 Schematic table of experimental parameters provided in the examples of this application (3)

[0235]

[0236]

[0237] The embodiments of the present application were tested on the fine-tuning model before fine-tuning the relevant model (pre-training model) and after fine-tuning. After introducing the text processing method provided by the embodiments of the present application, the performance of the models at different stages was significantly improved.

[0238] In some embodiments, see Figure 10 , Figure 10is an experimental effect schematic diagram of the text processing method provided by the embodiment of the present application. By comparing the performance of the text generation model of the present application and related technologies on different types of data sets (for example, an education topic data set, a finance topic data set, a fine-tuning data set, etc.), it is found that Figure 10 It can be seen that the embodiments of the present application can improve the performance of the text generation model on almost all types of data sets.

[0239] In this way, the embodiments of the present application improve the authenticity of the generated results through multi-directional intervention, effectively solving the problem of generating unrealistic information by pre-training language models in the generation task. The embodiments of the present application can effectively alleviate the generation-discrimination gap, make the learned direction more stable, and generalize to different positions in the generation process. The embodiments of the present application have low time complexity, are easy to implement and apply. The embodiments of the present application can be applied to various natural language processing tasks and have wide applicability. The embodiments of the present application can be used in combination with other generation models to further improve the authenticity and reliability of the generated results. By adjusting the intervention strength, users can control the authenticity of the generated results as needed to improve the quality of the generated results.

[0240] In this way, by extracting features from the text to be processed, the initial text features of the text to be processed are obtained. Based on the initial text features, the authenticity of the logic of the text to be processed is predicted in at least one prediction dimension to obtain authenticity prediction results of the text to be processed in each prediction dimension. When the authenticity prediction result indicates that the text to be processed does not have authenticity in the corresponding prediction dimension, the correction feature in the corresponding prediction dimension is obtained. Based on the correction feature, the initial text feature is corrected to obtain the target text feature corresponding to the initial text feature. The target text feature is decoded to obtain the target text which has authenticity in each prediction dimension. In this way, by predicting the authenticity of the logic of the text to be processed in at least one prediction dimension based on the initial text features, authenticity prediction results of the text to be processed in each prediction dimension are obtained. By correcting the initial text features, the target text features are obtained, and the target text features are decoded, so that the target text has authenticity in each prediction dimension, thereby effectively improving the accuracy of the target text and effectively improving the accuracy of the text processing.

[0241] It can be understood that in the embodiments of the present application, data related to the text to be processed is involved. When the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0242] The following continues to illustrate an exemplary structure of the implementation of the text processing device 455 provided by the embodiments of the present application as a software module. In some embodiments, as shown in FIG. 7, the text processing device 455 includes a text processing module 710, a text feature extraction module 720, a text feature correction module 730, a text feature decoding module 740, and a text generation module 750.Figure 2 As shown, the software modules stored in the text processing apparatus 455 of the memory 450 can include: a feature extraction module 4551 configured to perform feature extraction on the to-be-processed text to obtain initial text features of the to-be-processed text; a reality prediction module 4552 configured to perform reality prediction on the logic of the to-be-processed text in at least one prediction dimension based on the initial text features to obtain reality prediction results of the to-be-processed text in each of the prediction dimensions; an acquisition module 4553 configured to acquire a modified feature of the initial text features in a corresponding prediction dimension when the reality prediction result indicates that the to-be-processed text does not have reality in the corresponding prediction dimension; a feature modification module 4554 configured to perform feature modification on the initial text features based on the modified feature to obtain target text features corresponding to the initial text features; and a feature decoding module 4555 configured to perform feature decoding on the target text features to obtain a target text corresponding to the to-be-processed text, the target text having the reality in each of the prediction dimensions.

[0243] In some embodiments, the feature extraction is implemented by at least one feature extraction network, and the feature extraction module is further configured to invoke a first feature extraction network to perform feature extraction on the to-be-processed text to obtain first initial text features; and perform the following processing by iterating i: invoke an i-th feature extraction network to perform feature extraction on the to-be-processed text based on (i-1)-th initial text features to obtain i-th initial text features; wherein 1

[0244] In some embodiments, the text processing apparatus further includes a feature checking module configured to perform reality prediction on the logic of the to-be-processed text in each of the prediction dimensions based on the (i-1)-th initial text features to obtain (i-1)-th reality prediction results of the to-be-processed text in each of the prediction dimensions; and perform feature checking on the (i-1)-th initial text features based on the (i-1)-th reality prediction results to obtain (i-1)-th target text features; and the feature extraction module is further configured to invoke an i-th feature extraction network to perform feature extraction on the to-be-processed text based on the (i-1)-th target text features to obtain the i-th initial text features.

[0245] In some embodiments, the feature checking module is further configured to, when the i-1th authenticity prediction result indicates that the to-be-processed text does not have the authenticity in the corresponding prediction dimension, perform feature correction on the i-1th initial text feature to obtain an i-1th target text feature; and when each of the i-1th authenticity prediction results indicates that the to-be-processed text has the authenticity in the corresponding prediction dimension, determine the i-1th initial text feature as the i-1th target text feature.

[0246] In some embodiments, the authenticity prediction module is further configured to obtain an authenticity prediction network corresponding to each of the prediction dimensions, and perform the following processing for each of the prediction dimensions: calling the authenticity prediction network corresponding to the prediction dimension, performing authenticity prediction on the logic of the text sample in the prediction dimension based on the initial text feature of the text sample to obtain an authenticity score of the text sample in the prediction dimension; when the authenticity score is greater than or equal to a score threshold, determining a first result of authenticity prediction of the prediction dimension, the first result being used to indicate that the to-be-processed text has the authenticity in the prediction dimension; and when the authenticity score is less than the score threshold, determining a second result of authenticity prediction of the prediction dimension, the second result being used to indicate that the to-be-processed text does not have the authenticity in the prediction dimension.

[0247] In some embodiments, the authenticity prediction module is further configured to obtain an initial prediction network, obtain a plurality of text feature samples corresponding to a text sample, and obtain authenticity label scores of the text feature samples; for each of the text feature samples, call the initial prediction network, perform authenticity prediction on the logic of the text sample in the prediction dimension based on the text feature sample to obtain an authenticity score corresponding to the text feature sample, and determine a loss value corresponding to the text feature sample by combining the authenticity score and the authenticity label score corresponding to the text feature sample; and train the initial prediction network based on the loss values corresponding to the text feature samples to obtain the authenticity prediction network corresponding to the prediction dimension.

[0248] In some embodiments, the authenticity prediction module is further configured to obtain a text sample, and perform feature extraction on the text sample to obtain an initial text feature of the text sample; and perform feature splitting on the initial text feature of the text sample to obtain a plurality of text feature samples corresponding to the text sample.

[0249] In some embodiments, the authenticity prediction module is further configured to obtain a text sample, and perform feature extraction on the text sample to obtain initial text features of the text sample; and perform feature splitting on the initial text features of the text sample to obtain a plurality of text feature samples corresponding to the text sample.

[0250] In some embodiments, the authenticity prediction module is further configured to obtain an initial prediction network, and obtain a first text feature sample corresponding to a text sample of a first prediction dimension, and a first authenticity label score of the first text feature sample; invoke the initial prediction network, perform authenticity prediction on a logic of the text sample of the first prediction dimension based on the first text feature sample to obtain a first authenticity score, and train the initial prediction network based on the first authenticity score and the first authenticity label score to obtain an authenticity prediction network corresponding to the first prediction dimension; and perform the following processing by traversing j: obtain a (j-1)th authenticity score corresponding to a text sample of a (j-1)th prediction dimension, train the initial prediction network based on the (j-1)th authenticity score to obtain an authenticity prediction network corresponding to a jth prediction dimension; wherein 2≤j≤M, and M is used to indicate a number of the prediction dimensions.

[0251] In some embodiments, the authenticity prediction module is further configured to obtain a jth text feature sample corresponding to a text sample of a jth prediction dimension, and a jth authenticity label score of the jth text feature sample; invoke the initial prediction network, perform authenticity prediction on a logic of the text sample of the jth prediction dimension based on the jth text feature sample to obtain a jth authenticity score; determine a first loss value based on the jth authenticity score and the (j-1)th authenticity score, and determine a second loss value based on the jth authenticity score and the jth authenticity label score; and train the initial prediction network based on the first loss value and the second loss value to obtain an authenticity prediction network corresponding to the jth prediction dimension.

[0252] In some embodiments, the feature decoding module is further configured to, when authenticity prediction results of all the prediction dimensions indicate that the to-be-processed text has the authenticity in the corresponding prediction dimensions, perform feature decoding on the initial text features to obtain the target text corresponding to the to-be-processed text.

[0253] In some embodiments, the above-mentioned correction features correspond to target prediction dimensions one by one, the to-be-processed text does not have the authenticity in the target prediction dimensions, and the feature correction module is further configured to obtain authenticity scores of the to-be-processed text in each target prediction dimension, determine each authenticity score as a weight of a corresponding correction feature, perform weighted fusion on each correction feature according to the weight of the correction feature, obtain a reference correction feature, and perform feature correction on the initial text feature based on the reference correction feature to obtain a target text feature corresponding to the initial text feature.

[0254] In some embodiments, the feature correction module is further configured to obtain a feature dimension of the initial text feature and a feature dimension of the reference correction feature, adjust the feature dimension of the reference correction feature to obtain a target correction feature when the feature dimension of the initial text feature is different from the feature dimension of the reference correction feature, determine the reference correction feature as the target correction feature when the feature dimension of the initial text feature is the same as the feature dimension of the reference correction feature, determine a correction strength of the initial text feature based on a number of the correction features, the correction strength being positively correlated with the number of the correction features, determine a product of the correction strength and the target correction feature as a fusion feature, and add the initial text feature and the fusion feature to obtain the target text feature.

[0255] In some embodiments, the feature decoding module is further configured to obtain a task type of the to-be-processed text and obtain a task prediction network corresponding to the task type, call a task prediction network corresponding to an answer prediction task for answering the to-be-processed text when the task type is the answer prediction task, perform answer prediction on the to-be-processed text based on the target text feature to obtain an answer text corresponding to the to-be-processed text, the answer text having the authenticity in each prediction dimension, call a task prediction network corresponding to a translation task for translating the to-be-processed text when the task type is the translation task, perform translation on the to-be-processed text based on the target text feature to obtain a translation text corresponding to the to-be-processed text, and the translation text having the authenticity in each prediction dimension.

[0256] The embodiment of the present application provides a computer program product, which comprises a computer program or computer executable instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer executable instructions from the computer readable storage medium, and the processor executes the computer executable instructions, so that the electronic device executes the text processing method provided in the embodiment of the present application.

[0257] The embodiment of the present application provides a computer readable storage medium storing computer executable instructions, wherein the computer executable instructions are stored, and when the computer executable instructions are executed by a processor, the processor executes a text processing method provided by the embodiment of the present application, for example, as shown in the text processing method. Figure 3 The text processing method.

[0258] In some embodiments, the computer readable storage medium can be FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM memory, and the like; and can also be various electronic devices including one or any combination of the above memories.

[0259] In some embodiments, the computer executable instructions can be in the form of programs, software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and can be deployed in any form, including being deployed as independent programs or being deployed as modules, components, subroutines or other units suitable for use in a computing environment.

[0260] As an example, the computer executable instructions can but not necessarily correspond to files in a file system, can be stored in a part of a file storing other programs or data, for example, stored in one or more scripts in a HyperText Markup Language (HTML) document, stored in a single file dedicated to the program in question, or stored in multiple cooperative files (for example, files storing one or more modules, subroutines or code parts).

[0261] As an example, the computer executable instructions can be deployed to be executed on one electronic device, or executed on multiple electronic devices located in one place, or executed on multiple electronic devices distributed in multiple places and interconnected through a communication network.

[0262] In summary, the embodiment of the present application has the following beneficial effects:

[0263] (1) The initial text features of the to-be-processed text are obtained by feature extraction on the to-be-processed text, the authenticity of the logic of the to-be-processed text is predicted in at least one prediction dimension based on the initial text features, the authenticity prediction results of the to-be-processed text in each prediction dimension are obtained, when the authenticity prediction result indicates that the to-be-processed text does not have authenticity in the corresponding prediction dimension, the correction features in the corresponding prediction dimension are obtained, the initial text features are corrected based on the correction features, the target text features corresponding to the initial text features are obtained, and the target text with authenticity in each prediction dimension is obtained by decoding the target text features. In this way, by predicting the authenticity of the logic of the to-be-processed text in at least one prediction dimension based on the initial text features, the authenticity prediction results of the to-be-processed text in each prediction dimension are obtained, the initial text features are corrected to obtain target text features, and the target text features are decoded, so that the target text has authenticity in each prediction dimension, thereby effectively improving the accuracy of the target text and the accuracy of the text processing.

[0264] (2) Before calling the i-th feature extraction network, the i-1-th target text features are obtained by feature checking on the i-1-th initial text features, so as to call the i-th feature extraction network, and the i-th initial text features are obtained by feature extraction on the to-be-processed text based on the i-1-th target text features, so that the feature checking is performed layer by layer in the plurality of feature extraction networks to ensure that the input of each layer of the feature extraction network is the target text features after strict feature checking, and the feature extraction network can gradually realize the optimization of feature extraction on the to-be-processed text, thereby effectively improving the accuracy of feature extraction.

[0265] (3) The initial text features of the text sample are obtained by feature extraction on the text sample, the initial text features of the text sample are split to obtain a plurality of text feature samples corresponding to the text sample, thereby effectively expanding the number of training samples of the initial prediction network, and effectively improving the prediction performance of the authenticity prediction network obtained by training.

[0266] (4) The j-1-th authenticity score corresponding to the text sample of the j-1-th prediction dimension is obtained, the initial prediction network is trained based on the j-1-th authenticity score, and the authenticity prediction network corresponding to the j-th prediction dimension is obtained, so that the authenticity prediction network corresponding to the j-th prediction dimension can effectively learn the network parameters of the authenticity prediction network corresponding to the j-1-th prediction dimension, the prediction direction of the authenticity prediction network corresponding to the j-th prediction dimension is orthogonal to the prediction direction of the authenticity prediction network corresponding to the j-1-th prediction dimension, and the prediction independence between the authenticity prediction networks of different prediction dimensions is effectively improved.

[0267] (5) By modifying the corresponding feature dimensions of the initial text features, the text obtained by decoding the modified initial text features in the corresponding feature dimensions has authenticity, thereby effectively improving the accuracy of the generated target text.

[0268] (6) The embodiment of the present application improves the authenticity of the generated results by multi-directional intervention, which can effectively solve the problem of generating unrealistic information in the pre-training language model in the generation task. The embodiment of the present application can effectively alleviate the generation-discrimination gap, make the learned direction more stable, and generalize to different positions in the generation process. The embodiment of the present application has low time complexity, is easy to implement and apply. The embodiment of the present application can be applied to various natural language processing tasks and has wide applicability. The embodiment of the present application can be used in combination with other generation models to further improve the authenticity and reliability of the generated results. By adjusting the intervention intensity, users can control the authenticity of the generated results according to their needs and improve the quality of the generated results.

[0269] (7) The embodiment of the present application tests the related model before fine-tuning (pre-training model) and the fine-tuned model after fine-tuning. After introducing the text processing method provided by the embodiment of the present application, the performance of the model at different stages is significantly improved.

[0270] (8) The experimental results show that the text processing method provided by the embodiment of the present application has achieved significant performance improvement in various scenarios. Compared with the implementation manner of the related technology, the embodiment of the present application has better performance on the complete data set. The experimental results show that the embodiment of the present application can significantly improve True*Info% with minimal intervention at any stage. Compared with the related technology, the embodiment of the present application achieves better results in the compatible FSP under the few-shot setting. The CE and KL results show that the embodiment of the present application achieves better performance with minimal intervention while maintaining the amount of information.

[0271] The above is only an embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement and improvement made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A text processing method, characterized in that: The method comprises: Performing feature extraction on the text to be processed to obtain initial text features of the text to be processed; Obtain the authenticity prediction network corresponding to each prediction dimension, and perform the following processing for each prediction dimension: call the corresponding authenticity prediction network, and based on the initial text features, perform authenticity prediction on the logic of the text to be processed in the prediction dimension to obtain the authenticity score of the text to be processed in the prediction dimension; when the authenticity score is greater than or equal to the score threshold, determine the authenticity prediction result of the prediction dimension as a first result, and the first result is used to indicate that the text to be processed has the authenticity under the prediction dimension; when the authenticity score is less than the score threshold, determine the authenticity prediction result of the prediction dimension as a second result, and the second result is used to indicate that the text to be processed does not have the authenticity under the prediction dimension; When the authenticity prediction result indicates that the to-be-processed text is not authentic under the corresponding prediction dimension, obtaining a modified feature of the initial text feature under the corresponding prediction dimension; Based on the modified features, the initial text features are modified to obtain target text features corresponding to the initial text features; Decoding the target text features to obtain a target text corresponding to the text to be processed, wherein the target text has the authenticity under each of the prediction dimensions; Wherein, when the number of the prediction dimensions is multiple, the obtaining of the authenticity prediction network corresponding to each prediction dimension includes: obtaining an initial prediction network, and obtaining the first text feature sample corresponding to the text sample of the first prediction dimension, and the first authenticity label score of the first text feature sample; calling the initial prediction network, and based on the first text feature sample, performing authenticity prediction on the logic of the text sample of the first prediction dimension to obtain the first authenticity score, and combining the first authenticity score and the first authenticity label score to train the initial prediction network to obtain the authenticity prediction network corresponding to the first prediction dimension; traversing j to perform the following processing: obtaining the j-1th authenticity score corresponding to the text sample of the j-1th prediction dimension, and training the initial prediction network based on the j-1th authenticity score to obtain the authenticity prediction network corresponding to the j-1th prediction dimension; wherein, , M is used to indicate the number of prediction dimensions.

2. The method according to claim 1, characterized in that The feature extraction is implemented by at least one feature extraction network. When there are multiple feature extraction networks, the feature extraction is performed on the text to be processed to obtain the initial text features of the text to be processed, including: Calling a first feature extraction network to perform feature extraction on the text to be processed to obtain a first initial text feature; Traversing i, the following processing is performed: calling the i-th feature extraction network, performing feature extraction on the text to be processed based on the i-1-th initial text feature, and obtaining the i-th initial text feature; in, , N is used to indicate the number of feature extraction networks; The Nth initial text feature is determined as the initial text feature of the text to be processed.

3. The method according to claim 2, characterized in that Before calling the i-th feature extraction network and performing feature extraction on the to-be-processed text based on the i-1-th initial text feature to obtain the i-th initial text feature, the method further includes: Based on the i-1th initial text feature, performing authenticity prediction on the logic of the text to be processed in each of the prediction dimensions, and obtaining the i-1th authenticity prediction result of the text to be processed in each of the prediction dimensions; Based on the (i-1)th authenticity prediction result, performing a feature check on the (i-1)th initial text feature to obtain the (i-1)th target text feature; The calling of the i-th feature extraction network to extract features of the text to be processed based on the i-1-th initial text feature to obtain the i-th initial text feature includes: The i-th feature extraction network is called to perform feature extraction on the text to be processed based on the i-1-th target text feature to obtain the i-th initial text feature.

4. The method according to claim 3, characterized in that The step of performing feature checking on the i-1th initial text feature based on the i-1th authenticity prediction result to obtain the i-1th target text feature includes: When there is the (i-1)th authenticity prediction result indicating that the to-be-processed text does not have the authenticity under the corresponding prediction dimension, performing feature correction on the (i-1)th initial text feature to obtain the (i-1)th target text feature; When each of the (i-1)th authenticity prediction results indicates that the text to be processed has the authenticity under the corresponding prediction dimension, the (i-1)th initial text feature is determined as the (i-1)th target text feature.

5. The method according to claim 1, wherein When the number of the prediction dimension is one, obtaining the authenticity prediction network corresponding to each prediction dimension includes: Obtaining an initial prediction network, and obtaining multiple text feature samples corresponding to the text sample, and a truth label score for each of the text feature samples; For each of the text feature samples, calling the initial prediction network, based on the text feature sample, performing authenticity prediction on the logic of the text sample in the prediction dimension, obtaining an authenticity score corresponding to the text feature sample, and combining the authenticity score and the corresponding authenticity label score to determine the loss value corresponding to the text feature sample; Based on the loss value corresponding to each of the text feature samples, the initial prediction network is trained to obtain a authenticity prediction network corresponding to the prediction dimension.

6. The method according to claim 5, characterized in that The obtaining of multiple text feature samples corresponding to the text sample includes: Acquire a text sample, and perform feature extraction on the text sample to obtain initial text features of the text sample; Perform feature splitting on the initial text features of the text sample to obtain multiple text feature samples corresponding to the text sample.

7. The method according to claim 1, characterized in that The training of the initial prediction network based on the j-1th authenticity score to obtain the authenticity prediction network corresponding to the jth prediction dimension includes: Obtaining a j-th text feature sample corresponding to the text sample of the j-th prediction dimension, and a j-th authenticity label score of the j-th text feature sample; Calling the initial prediction network, and performing a logic authenticity prediction on the text sample of the j-th prediction dimension based on the j-th text feature sample to obtain a j-th authenticity score; Determine a first loss value by combining the j-th authenticity score and the j-1-th authenticity score, and determine a second loss value by combining the j-th authenticity score and the j-th authenticity label score; The initial prediction network is trained in combination with the first loss value and the second loss value to obtain a authenticity prediction network corresponding to the j-th prediction dimension.

8. The method according to claim 1, characterized in that After performing a logic authenticity prediction on the text to be processed in the prediction dimension based on the initial text features and obtaining the authenticity score of the text to be processed in the prediction dimension, the method further includes: When the authenticity prediction results of each of the prediction dimensions indicate that the text to be processed has the authenticity under the corresponding prediction dimension, feature decoding is performed on the initial text features to obtain the target text corresponding to the text to be processed.

9. The method according to claim 1, characterized in that The step of decoding the target text features to obtain the target text corresponding to the text to be processed includes: Obtaining a task type for the text to be processed and obtaining a task prediction network corresponding to the task type; When the task type is an answer prediction task for answering the text to be processed, calling the task prediction network corresponding to the answer prediction task, and performing answer prediction on the text to be processed based on the target text features to obtain an answer text corresponding to the text to be processed, wherein the answer text has the authenticity under each of the prediction dimensions; When the task type is a translation task for translating the text to be processed, the task prediction network corresponding to the translation task is called, and the text to be processed is translated based on the target text features to obtain a translation text corresponding to the text to be processed, and the translation text has the authenticity under each of the prediction dimensions.

10. A text processing device, characterized in that: The device comprises: A feature extraction module is used to extract features of the text to be processed to obtain initial text features of the text to be processed; The authenticity prediction module is used to obtain the authenticity prediction network corresponding to each prediction dimension, and perform the following processing for each prediction dimension: call the corresponding authenticity prediction network, and based on the initial text features, perform authenticity prediction on the logic of the text to be processed in the prediction dimension to obtain the authenticity score of the text to be processed in the prediction dimension; when the authenticity score is greater than or equal to the score threshold, determine the authenticity prediction result of the prediction dimension as a first result, and the first result is used to indicate that the text to be processed has the authenticity under the prediction dimension; when the authenticity score is less than the score threshold, determine the authenticity prediction result of the prediction dimension as a second result, and the second result is used to indicate that the text to be processed does not have the authenticity under the prediction dimension; an acquisition module, configured to acquire, when the authenticity prediction result indicates that the to-be-processed text is not authentic under the corresponding prediction dimension, a modified feature of the initial text feature under the corresponding prediction dimension; A feature correction module, configured to perform feature correction on the initial text feature based on the correction feature to obtain a target text feature corresponding to the initial text feature; A feature decoding module is used to decode the target text features to obtain a target text corresponding to the text to be processed, wherein the target text has the authenticity under each of the prediction dimensions; Wherein, when the number of the prediction dimensions is multiple, the obtaining of the authenticity prediction network corresponding to each prediction dimension includes: obtaining an initial prediction network, and obtaining the first text feature sample corresponding to the text sample of the first prediction dimension, and the first authenticity label score of the first text feature sample; calling the initial prediction network, and based on the first text feature sample, performing authenticity prediction on the logic of the text sample of the first prediction dimension to obtain the first authenticity score, and combining the first authenticity score and the first authenticity label score to train the initial prediction network to obtain the authenticity prediction network corresponding to the first prediction dimension; traversing j to perform the following processing: obtaining the j-1th authenticity score corresponding to the text sample of the j-1th prediction dimension, and training the initial prediction network based on the j-1th authenticity score to obtain the authenticity prediction network corresponding to the j-1th prediction dimension; wherein, , M is used to indicate the number of prediction dimensions.

11. An electronic device, characterized in that: The electronic device comprises: a memory for storing computer-executable instructions or computer programs; The processor is configured to implement the text processing method according to any one of claims 1 to 9 when executing the computer-executable instructions or computer programs stored in the memory.

12. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer-executable instructions are executed by a processor, the text processing method according to any one of claims 1 to 9 is implemented.

13. A computer program product comprising a computer program or computer executable instructions, characterized in that When the computer program or computer executable instructions are executed by a processor, the text processing method according to any one of claims 1 to 9 is implemented.

Images