Common sense error correction method and system, storage medium and terminal

Abstract

The invention provides a common sense error correction method and system, a storage medium and a terminal, and the method comprises the following steps: obtaining a training sample set of common sense error correction, each training sample comprising common sense related questions and a plurality of corresponding candidate answers; obtaining the number of correct answers of the candidate answers; calculating a sample weight of the training sample based on the number of the candidate answers and the number of the correct answers; based on the sample weight, dividing the training sample into an effective sample, a difficult sample and an invalid sample, and distributing a training weight for the effective sample; training a common sense error correction reinforcement learning model based on the effective samples and the training weights; and performing instruction fine tuning on the common sense error correction reinforcement learning model based on the difficult sample so as to perform common sense error correction based on the trained common sense error correction reinforcement learning model. According to the common sense error correction method and system, the storage medium and the terminal, high-precision common sense error correction is realized through the reinforcement learning algorithm for adaptively adjusting the training data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, and in particular to a common sense error correction method and system, a storage medium and a terminal. BACKGROUND

[0002] Common sense refers to the basic knowledge, experience and judgment criteria that most people universally possess in a certain social and cultural context. It runs through the process of daily life, social interaction and cognitive world, and has three core characteristics of universality, practicality and experience. Common sense errors include factual errors, conceptual errors, logical errors, etc.

[0003] Common sense error correction is a highly challenging knowledge-based error correction task. For example, the user input: "The moonlight in front of the bed" is from the Song Dynasty masterpiece "Quiet Night Thoughts", and the author is Du Fu. After error correction analysis, "The moonlight in front of the bed" is from "Quiet Night Thoughts", but the dynasty of this work is the Tang Dynasty, and the author is the famous poet of the Tang Dynasty, Li Bai, not Du Fu. In the prior art, a machine learning model can be used to implement common sense error correction. Reinforcement learning (RL) is an important branch of machine learning, and its core idea is to guide the model to learn the optimal decision strategy through the interaction between the agent and the environment, and the "reward-punishment" mechanism. With the development of large language models (LLM), reinforcement learning from human feedback (RLHF) has become a key technology to improve the alignment of model preferences with human preferences, and is widely used in dialogue systems, content generation, intelligent question answering and other fields.

[0004] Unlike other tasks, the difficulty of common sense error correction cases varies greatly. For example, the fact error case "Xiaoming's grandfather is 400 years old this year" is relatively simple, and the model can directly point out that the age is unreasonable. However, the logic error case "Last year's revenue was 4 billion, this year's revenue was 5 billion, an increase of 10%" is a difficult case, and the model needs to list the expression for calculation. In traditional reinforcement learning training, all cases are equally weighted, i.e. the learning value of different difficulty cases is the same by default, which can cause the following problems:

[0005] (1) Low learning efficiency

[0006] All sample weights are equal, resulting in the model consuming a large amount of computing resources on simple samples that it has already mastered, and insufficient investment on complex samples that it has not mastered. For example, the model's error correction rate for simple cases such as "Xiaoming's grandfather is 400 years old this year" has reached 99%, but it still obtains the same training weight as the "complex mathematical modeling problem" with an error correction rate of only 10%, resulting in a serious waste of computing power and time resources, and slow improvement of the model's ability.

[0007] (2) The ability short board cannot be positioned

[0008] Due to the lack of correlation analysis of the sample difficulty and the model mastery degree, the model cannot specifically strengthen the key samples of "like to not like", and cannot identify invalid samples of "completely not". For example, when the case "Li Bai died in 476", if the internal knowledge of the model and the external reference information do not cover this information, the model cannot make accurate corrections through reinforcement learning. Repeatedly inputting this kind of error sample into training will only lead to the model learning error mode and further reduce the generalization performance. SUMMARY

[0009] In view of the above-mentioned defects of the prior art, the purpose of the present application is to provide a common sense correction method, system, storage medium and terminal, which realizes high-precision common sense correction through self-adaptive adjustment of the reinforcement learning algorithm of the training data.

[0010] In a first aspect, the present application provides a common sense correction method, which comprises the following steps: obtaining a training sample set for common sense correction, each training sample comprising a common sense related question and a plurality of candidate answers corresponding thereto; obtaining the number of correct answers of the candidate answers; calculating the sample weight of the training sample based on the number of candidate answers and the number of correct answers; dividing the training sample into effective samples, difficult samples and invalid samples based on the sample weight, and assigning a training weight to the effective samples; training a common sense correction reinforcement learning model based on the effective samples and the training weight; and performing instruction fine-tuning on the common sense correction reinforcement learning model based on the difficult samples, so as to perform common sense correction based on the trained common sense correction reinforcement learning model.

[0011] In an implementation form of the first aspect, the number of candidate answers is not less than 3.

[0012] In an implementation form of the first aspect, the number of correct answers of the candidate answers is obtained based on artificial annotation or evaluation model.

[0013] In an implementation form of the first aspect, calculating the sample weight of the training sample based on the number of candidate answers and the number of correct answers comprises the following steps:

[0014] calculating the correct rate P of the training sample, wherein M represents the number of correct answers, and N represents the number of candidate answers;

[0015] calculating the sample weight W of the training sample, wherein W=1-P.

[0016] In an implementation of the first aspect, based on the sample weights, dividing the training samples into valid samples, difficult samples, and invalid samples includes the following steps:

[0017] Dividing the training samples with 0 < P < 1 into valid samples; where P represents the correct rate of the training samples;

[0018] Dividing the training samples with P = 0 into difficult samples;

[0019] Dividing the training samples with P = 1 into invalid samples.

[0020] In an implementation of the first aspect, training the common sense error correction reinforcement learning model based on the valid samples and the training weights includes the following steps:

[0021] Inputting the valid samples and the training weights into the common sense error correction reinforcement learning model;

[0022] Obtaining the weighted reward value of the valid samples;

[0023] Updating the parameters of the common sense error correction reinforcement learning model based on the weighted reward value.

[0024] In an implementation of the first aspect, the weighted reward value R' of the valid samples = R × W, where R represents the basic reward value of the valid samples and W represents the sample weight.

[0025] In the second aspect, the present invention provides a common sense error correction system, which includes a first acquisition module, a second acquisition module, a calculation module, a division module, a training module, and a fine-tuning module;

[0026] The first acquisition module is used to acquire a training sample set for common sense error correction, and each training sample includes a common sense-related question and a corresponding plurality of candidate answers;

[0027] The second acquisition module is used to acquire the number of correct answers of the candidate answers;

[0028] The calculation module is used to calculate the sample weight of the training sample based on the number of candidate answers and the number of correct answers;

[0029] The division module is used to divide the training samples into valid samples, difficult samples, and invalid samples based on the sample weight, and assign training weights to the valid samples;

[0030] The training module is used to train the common sense error correction reinforcement learning model based on the valid samples and the training weights;

[0031] The fine-tuning module is used to fine-tune the common sense error correction reinforcement learning model based on the difficult samples, so as to perform common sense error correction based on the trained common sense error correction reinforcement learning model.

[0032] Thirdly, the present invention provides a storage medium on which a computer program is stored, which, when executed by a processor, implements the aforementioned common sense error correction method.

[0033] Fourthly, the present invention provides a terminal, comprising: a processor and a memory;

[0034] The memory is used to store computer programs;

[0035] The processor is used to execute the computer program stored in the memory, so that the terminal performs the common sense error correction method described above.

[0036] As described above, the common sense error correction method, system, medium, and device of the present invention have the following beneficial effects:

[0037] (1) In view of the complex types and large differences in difficulty of common sense error correction cases, a reinforcement learning algorithm that adaptively adjusts the training data is used to achieve high-precision common sense error correction;

[0038] (2) Achieve an efficient learning mechanism of "accurately identifying sample value - allocating training resources on demand - and targeted strengthening of model capabilities"; among which, reduce the training weight of samples that the model is already proficient in, concentrate resources on high-value samples, and under the same computing power conditions, the model capability improvement speed is increased by more than 30%; increase the weight of key samples with "low accuracy but not zero" to help the model solidify "inspiration" type capabilities, and the answer accuracy of such samples can be increased by 40%-60%; filter out invalid samples with "completely wrong answers" to avoid wasting resources, and the effective utilization rate of training data can be increased by 25%-35%; clarify the training focus, and the training cycle for the model to reach the target performance can be shortened by 30%-45%;

[0039] (3) It meets the common sense error correction needs in different application scenarios. Attached Figure Description

[0040] Figure 1 The flowchart shown is an embodiment of the common sense error correction method of the present invention;

[0041] Figure 2 The diagram shows a framework schematic of the common sense error correction method of the present invention in one embodiment;

[0042] Figure 3 The diagram shown is a structural schematic of the common sense error correction system of the present invention in one embodiment.

[0043] Figure 4The diagram shown is a structural schematic of the terminal of the present invention in one embodiment. Detailed Implementation

[0044] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0045] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0046] The following embodiments of the present invention provide a common-sense error correction method, which can be applied to terminals. The terminals described in this invention may include mobile phones with wireless charging capabilities, tablet computers, laptops, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiments of the present invention do not impose any restrictions on the specific type of terminal.

[0047] For example, the terminal may be a station (STAION, ST) in a WLAN with wireless charging capability, a cellular phone, cordless phone, Session Initiation Protocol (SIP) phone, Wireless Local Loop (WLL) station, Personal Digital Assistant (PDA) device, handheld device with wireless charging capability, computing device or other processing device, computer, laptop computer, handheld communication device, handheld computing device, and / or other devices for communication over a wireless system, as well as next-generation communication systems, such as mobile terminals in 5G networks, mobile terminals in future evolved Public Land Mobile Networks (PLMNs), or mobile terminals in future evolved Non-terrestrial Networks (NTNs).

[0048] For example, the terminal can communicate with networks and other devices wirelessly. The wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), BT, GNSS, WLAN, NFC, FM, and / or IR technologies. The GNSS can include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), BeiDou Navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).

[0049] The technical solutions in the embodiments of the present invention will be described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0050] As Figure 1 and Figure 2 shown, in one embodiment, the common sense error correction method of the present invention includes steps S1 to S6.

[0051] Step S1, obtain a training sample set for common sense error correction, and each training sample includes a common sense-related question and a corresponding plurality of candidate answers.

[0052] Specifically, construct a training sample set for common sense error correction. Among them, for each training sample, a common sense-related question needs to be generated, and N candidate answers are generated based on an answer generation model, and N≥3. Each common sense-related question and the corresponding N candidate answers form a training sample.

[0053] Step S2, obtain the number of correct answers of the candidate answers.

[0054] Specifically, based on manual annotation of the number of correct answers in the candidate answers, or based on an evaluation model to obtain the number of correct answers in the candidate answers.

[0055] Step S3, calculate the sample weight of the training sample based on the number of candidate answers and the number of correct answers.

[0056] Specifically, different sample weights are set for different training samples to improve computing power resources in subsequent model training. The present invention first calculates the correct rate P = M / N of the training sample, where M represents the number of correct answers and N represents the number of candidate answers; then calculates the sample weight W = 1 - P of the training sample, thereby establishing a mapping relationship between the correct rate and the sample weight.

[0057] Step S4, based on the sample weight, divide the training sample into valid samples, difficult samples and invalid samples, and assign training weights to the valid samples.

[0058] Specifically, according to the sample weight, classify and divide the training samples. Among them, the training samples with 0 < P < 1 are divided into valid samples. The training samples with P = 0 are divided into difficult samples. The training samples with P = 1 are divided into invalid samples and directly discarded to avoid resource waste.

[0059] Step S5, train a common sense error correction reinforcement learning model based on the valid samples and the training weights.

[0060] Specifically, the common sense error correction reinforcement learning model is trained using the Proximal Policy Optimization (PPO) algorithm. The effective samples and their corresponding sample weights are input into the common sense error correction reinforcement learning model to obtain the weighted reward value of the effective samples, R' = R × W. The parameters of the common sense error correction reinforcement learning model are updated based on the weighted reward value, so as to ensure that the training samples with high sample weights have a greater impact on the model parameters, effectively improving the model accuracy. Here, R represents the basic reward value of the effective samples, which is obtained through the reward model of the common sense error correction reinforcement learning model.

[0061] Step S6: Perform instruction fine-tuning on the common sense error correction reinforcement learning model based on the difficult samples, so as to perform common sense error correction based on the trained common sense error correction reinforcement learning model.

[0062] Specifically, the difficult samples are used to perform instruction fine-tuning on the common sense error correction reinforcement learning model, so as to re-incorporate it into the reinforcement learning process after improving the basic capabilities of the model. Here, the instruction fine-tuning is to perform supervised fine-tuning (SFT). In the present invention, for cases where the model cannot answer correctly at all in the current stage, the correct answer needs to be clearly told to the model to let it remember. For example, for the wrong common sense "Li Bai died in 476 AD", if the common sense error correction reinforcement learning model lacks this knowledge, the correct answer is supplemented to inject this knowledge into the common sense error correction reinforcement learning model. For example, construct a case "User input: Please detect whether there is a common sense error in the following sentence - Li Bai died in 476 AD. Answer: There is a common sense error. Li Bai did not die in 476 AD, but in 762 AD". Therefore, through the screening of invalid samples and instruction fine-tuning based on difficult samples, a closed-loop of "reinforcement learning to fill in the short board - instruction fine-tuning to lay the foundation" is formed, solving the problem that the model cannot answer at all, and improving the overall utilization rate of training samples.

[0063] For example, in an embodiment, the text to be detected is: "Before my bed a pool of light" is from the famous poem "Thoughts in the Silent Night" in the Song Dynasty, and the author is Du Fu. Construct an instruction: Please analyze whether there is a common sense error in the text to be detected. Input the text to be detected and the instruction into the common sense error correction reinforcement learning model, and the output is: According to generally recognized historical knowledge, there are the following common sense errors in the text to be detected:

[0064] 1. "Thoughts in the Silent Night" is a famous poem in the Tang Dynasty, not the Song Dynasty;

[0065] 2. The author of "Thoughts in the Silent Night" is Li Bai, not Du Fu.

[0066] The scope of protection of the common sense error correction method described in this embodiment is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this invention is included within the scope of protection of this invention.

[0067] This invention also provides a common sense error correction system, which can implement the common sense error correction method described in this invention. However, the implementation device of the common sense error correction system described in this invention includes, but is not limited to, the structure of the common sense error correction system listed in this embodiment. All structural modifications and substitutions of the prior art made in accordance with the principles of this invention are included within the protection scope of this invention.

[0068] like Figure 3 As shown, in one embodiment, the common sense error correction system of the present invention includes a first acquisition module 31, a second acquisition module 32, a calculation module 33, a partitioning module 34, a training module 35, and a fine-tuning module 36.

[0069] The first acquisition module 31 is used to acquire a training sample set for common sense error correction. Each training sample includes common sense related questions and multiple corresponding candidate answers.

[0070] The second acquisition module 32 is connected to the first acquisition module 31 and is used to acquire the number of correct answers among the candidate answers.

[0071] The calculation module 33 is connected to the second acquisition module 32 and is used to calculate the sample weight of the training sample based on the number of candidate answers and the number of correct answers.

[0072] The partitioning module 34 is connected to the calculation module 33 and is used to partition the training samples into effective samples, difficult samples and invalid samples based on the sample weights, and to assign training weights to the effective samples.

[0073] The training module 35 is connected to the partitioning module 34 and is used to train a common sense error correction reinforcement learning model based on the effective samples and the training weights.

[0074] The fine-tuning module 36 is connected to the training module 35 and is used to fine-tune the common sense error correction reinforcement learning model based on the difficult samples, so as to perform common sense error correction based on the trained common sense error correction reinforcement learning model.

[0075] The structure and principle of the first acquisition module 31, the second acquisition module 32, the calculation module 33, the partitioning module 34, the training module 35 and the fine-tuning module 36 correspond one-to-one with the steps in the above common sense error correction method, so they will not be described again here.

[0076] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0077] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of the present invention, depending on actual needs. For example, the functional modules / units in the various embodiments of the present invention may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0078] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0079] This invention also provides a computer-readable storage medium. Those skilled in the art will understand that all or part of the steps in the common-sense error correction method of the above embodiments can be implemented by a program instructing a processor. The program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof. The storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. This available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state drive (SSD)).

[0080] This invention also provides a terminal. The terminal includes a processor and a memory.

[0081] The memory is used to store computer programs.

[0082] The memory includes various media capable of storing program code, such as ROM, RAM, magnetic disk, USB flash drive, memory card, or optical disk.

[0083] The processor is connected to the memory and is used to execute the computer program stored in the memory so that the terminal performs the aforementioned common sense error correction method.

[0084] Preferably, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0085] like Figure 4As shown, the terminal of the present invention is presented in the form of a general-purpose computing device. The components of the terminal may include, but are not limited to: one or more processors or processing units 41, a memory 42, and a bus 43 connecting different system components (including the memory 42 and the processing unit 41).

[0086] Bus 43 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0087] Terminals typically include various computer system-readable media. These media can be any available media that can be accessed by the terminal, including volatile and non-volatile media, and removable and non-removable media.

[0088] Memory 42 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 421 and / or cache memory 422. The terminal may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 423 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 43 via one or more data media interfaces. Memory 42 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0089] A program / utility 424 having a set (at least one) of program modules 4241 may be stored, for example, in memory 42. Such program modules 4241 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 4241 typically perform the functions and / or methods described in the embodiments of the present invention.

[0090] The terminal can also communicate with one or more external devices (e.g., keyboard, pointing device, display, etc.), one or more devices that enable user interaction with the terminal, and / or any device that enables the terminal to communicate with one or more other computing devices (e.g., network interface card, modem, etc.). This communication can be performed through input / output (I / O) interface 44. Furthermore, the terminal can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) through network adapter 45. Figure 4 As shown, network adapter 45 communicates with other modules of the terminal via bus 43. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the terminal, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0091] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A common sense error correction method, characterized in that, The method includes the following steps: Obtain a training sample set for common sense error correction, where each training sample includes a common sense-related question and multiple corresponding candidate answers; Obtain the number of correct answers for the candidate answers; Calculate the sample weight of the training sample based on the number of candidate answers and the number of correct answers; Based on the sample weight, divide the training samples into valid samples, difficult samples, and invalid samples, and assign training weights to the valid samples; Train a common sense error correction reinforcement learning model based on the valid samples and the training weights; Perform instruction fine-tuning on the common sense error correction reinforcement learning model based on the difficult samples, so as to perform common sense error correction based on the trained common sense error correction reinforcement learning model.

2. The common sense error correction method according to claim 1, characterized in that, The number of candidate answers is not less than 3.

3. The common sense error correction method according to claim 1, characterized in that, Obtain the number of correct answers for the candidate answers based on manual annotation or an evaluation model.

4. The common sense error correction method according to claim 1, characterized in that, Calculating the sample weight of the training sample based on the number of candidate answers and the number of correct answers includes the following steps: Calculate the accuracy rate P = M / N of the training sample, where M represents the number of correct answers and N represents the number of candidate answers; Calculate the sample weight W = 1 - P of the training sample.

5. The common sense error correction method according to claim 1, characterized in that, Dividing the training samples into valid samples, difficult samples, and invalid samples based on the sample weight includes the following steps: Divide the training samples with 0 < P < 1 into valid samples; where P represents the accuracy rate of the training sample; Divide the training samples with P = 0 into difficult samples; Divide the training samples with P = 1 into invalid samples.

6. The common sense error correction method according to claim 1, characterized in that, Training the common sense error correction reinforcement learning model based on the valid samples and the training weights includes the following steps: Input the valid samples and the training weights into the common sense error correction reinforcement learning model; Obtain the weighted reward value of the valid samples; Update the parameters of the common sense error correction reinforcement learning model based on the weighted reward value.

7. The common sense error correction method according to claim 6, characterized in that, The weighted reward value R' of the valid sample = R × W, where R represents the basic reward value of the valid sample and W represents the sample weight.

8. A common sense error correction system, characterized in that, The system includes a first acquisition module, a second acquisition module, a calculation module, a division module, a training module, and a fine-tuning module; The first acquisition module is used to obtain a training sample set for common sense error correction, where each training sample includes a common sense-related question and multiple corresponding candidate answers; The second acquisition module is used to obtain the number of correct answers for the candidate answers; The calculation module is used to calculate the sample weight of the training sample based on the number of candidate answers and the number of correct answers; The division module is used to divide the training samples into valid samples, difficult samples, and invalid samples based on the sample weight, and assign training weights to the valid samples; The training module is used to train a common sense error correction reinforcement learning model based on the valid samples and the training weights; The fine-tuning module is used to perform instruction fine-tuning on the common sense error correction reinforcement learning model based on the difficult samples, so as to perform common sense error correction based on the trained common sense error correction reinforcement learning model.

9. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the common sense error correction method described in any one of claims 1 to 7.

10. A terminal, characterized in that, Including: A processor and a memory; The memory is used to store a computer program; The processor is used to execute the computer program stored in the memory to cause the terminal to perform the common sense error correction method according to any one of claims 1 to 7.

Images