Hard negative sample screening method, device, apparatus and storage medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SF TECH CO LTD
- Filing Date
- 2022-06-28
- Publication Date
- 2026-07-21
Smart Images
Figure CN117349426B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of sample screening, specifically to a method, apparatus, device, and storage medium for screening difficult-to-handle samples. Background Technology
[0002] Currently, with the rapid development of artificial intelligence, natural language processing (NLP) technology has also been applied, for example, in question-answering systems and recommendation systems. Text retrieval is a crucial foundational task of NLP. Current text retrieval methods involve coarse recall of samples, followed by fine-tuning using a deep learning model to output relevant recommendations. The accuracy of these recommendations depends on the model's ability to identify and exclude difficult-to-reject samples. However, the quality of existing samples varies greatly, making it difficult to effectively identify and filter difficult-to-reject samples. This leads to a decrease in application accuracy due to the presence of difficult-to-reject samples. Summary of the Invention
[0003] This application provides a method, apparatus, device, and storage medium for screening difficult-to-bear samples, aiming to solve the technical problem in the prior art where inaccurate screening of difficult-to-bear samples leads to low recognition rates in subsequent applications.
[0004] On the one hand, embodiments of this application provide a method for screening difficult-to-bear samples, the method comprising the following steps:
[0005] Obtain a sample of the target problem;
[0006] Based on the morphemes of the target question sample and the morphemes of the preset candidate samples, recall the first candidate sample of the target question sample;
[0007] Based on the semantic features of the target question sample and the semantic features of the preset candidate samples, a second candidate sample of the target question sample is recalled;
[0008] The first and second candidate samples are filtered to obtain the hard negative samples of the target problem sample.
[0009] In one possible implementation of this application, the step of recalling a first candidate sample of the target question sample based on the morphemes of the target question sample and the morphemes of preset candidate samples includes:
[0010] The target question sample is parsed to obtain the morphemes of the target question sample;
[0011] Access the sample database and obtain the morphemes of each preset candidate sample in the sample database;
[0012] Calculate the morpheme similarity between the morphemes of the target question sample and the morphemes of each of the preset candidate samples, and recall the first candidate sample of the target question sample based on the similarity.
[0013] In one possible implementation of this application, the step of calculating the morpheme similarity between the morphemes of the target question sample and the morphemes of each of the preset candidate samples, and recalling a first candidate sample of the target question sample based on the similarity, includes:
[0014] Calculate the morpheme similarity between the morphemes of the target question sample and the morphemes of each preset candidate sample, and obtain morpheme candidate samples whose morpheme similarity is higher than the preset morpheme threshold.
[0015] Calculate the question similarity between the standard question of the target question sample and the standard question of the morpheme candidate sample, and recall the first candidate sample whose question similarity is lower than a preset question similarity threshold.
[0016] In one possible implementation of this application, the step of recalling a second candidate sample of the target question sample based on the semantic features of the target question sample and the semantic features of preset candidate samples includes:
[0017] The target question sample is encoded to obtain its semantic features;
[0018] The semantic features are subjected to high-dimensional mapping processing to obtain the high-dimensional semantic features of the target problem sample;
[0019] Based on the high-dimensional semantic features of the target question sample and the high-dimensional semantic features of the preset candidate samples, a second candidate sample of the target question sample is recalled.
[0020] In one possible implementation of this application, the step of recalling a second candidate sample of the target question sample based on the high-dimensional semantic features of the target question sample and the high-dimensional semantic features of the preset candidate samples includes:
[0021] Calculate the semantic similarity between the high-dimensional semantic features of the target problem sample and the high-dimensional semantic features of the preset candidate sample, and obtain semantic candidate samples whose semantic similarity is higher than the preset semantic threshold;
[0022] Calculate the semantic similarity between the standard question of the target question sample and the standard question of the semantic candidate sample, and recall a second candidate sample whose question similarity is lower than a preset question similarity threshold.
[0023] In one possible implementation of this application, the step of filtering the first candidate sample and the second candidate sample to obtain the hard-burden sample of the target problem sample includes:
[0024] The morpheme similarity of the first candidate sample and the semantic similarity of the second candidate sample are normalized to obtain the sample similarity.
[0025] By comparing the sample similarity with a preset sample similarity threshold, a first candidate sample and / or a second candidate sample whose sample similarity is higher than the sample similarity threshold are obtained, thus obtaining the hard negative sample of the target problem sample.
[0026] In one possible implementation of this application, after filtering the first candidate sample and the second candidate sample to obtain the hard-to-bear samples of the target problem sample, the method further includes:
[0027] Obtain the standard problem of the target problem sample and the standard problem of the hard-to-bear sample;
[0028] The target problem sample and the standard problem of the target problem sample are used as positive samples, and the hard negative sample and the standard problem of the hard negative sample are used as negative samples. They are input into a preset model for iterative training to obtain the model loss value of the preset model.
[0029] The model parameters in the preset model are corrected based on the model loss value to generate a question-answering model.
[0030] In one possible implementation of this application, after generating the question-answering model, the method further includes:
[0031] Receive a question-and-answer request and obtain the initial question text from the request.
[0032] Input the initial question text into the question-answering model to obtain the initial question sample corresponding to the initial question text;
[0033] The initial problem samples are subjected to hard-to-bear sample identification, and hard-to-bear problem samples are filtered out to obtain the filtered problem samples;
[0034] Obtain the target standard question corresponding to the filtered question sample, and output the preset answer text of the target standard question.
[0035] On the other hand, this application provides a difficult-to-bear sample screening device, the difficult-to-bear sample screening device comprising:
[0036] The sample acquisition module is configured to acquire samples of the target problem.
[0037] The statistical matching module is configured to recall the first candidate sample of the target question sample based on the morphemes of the target question sample and the morphemes of the preset candidate samples.
[0038] The semantic matching module is configured to recall a second candidate sample of the target question sample based on the semantic features of the target question sample and the semantic features of a preset candidate sample;
[0039] The secondary screening module is configured to screen the first candidate sample and the second candidate sample to obtain the hard negative sample of the target problem sample.
[0040] On the other hand, this application also provides a difficult-to-bear sample screening device, the difficult-to-bear sample screening device comprising:
[0041] One or more processors;
[0042] Memory; and
[0043] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the difficult sample screening method.
[0044] On the other hand, this application also provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps in the difficult sample screening method.
[0045] This application obtains a target question sample, along with its morphemes and those of pre-defined candidate samples. Statistical retrieval is then performed based on these morphemes to recall a first candidate sample of the target question sample. Semantic features of both the target question sample and the pre-defined candidate samples are then obtained, and a second candidate sample is recalled based on these features. Finally, the first and second candidate samples are further filtered to obtain the hard-negative samples of the target question sample. This effectively identifies hard-negative samples in the sample database, improving sample selection quality and reducing the probability of false recalls due to semantic or statistical matching. Attached Figure Description
[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0047] Figure 1 This is a schematic diagram illustrating a scenario for the sample screening method for difficult-to-bear samples in an embodiment of this application.
[0048] Figure 2This is a flowchart illustrating one embodiment of the difficult-to-bear sample screening method in this application.
[0049] Figure 3 A flowchart illustrating an embodiment of the difficult-to-bear sample screening method provided in this application, which recalls the first candidate sample of the target problem sample based on the similarity;
[0050] Figure 4 A flowchart illustrating an embodiment of the difficult-to-bear sample screening method provided in this application for recalling second candidate samples of target problem samples;
[0051] Figure 5 This is a schematic diagram of the structure of one embodiment of the difficult-to-bear sample screening device provided in this application;
[0052] Figure 6 This is a schematic diagram of one embodiment of the difficult-to-bear sample screening device provided in this application. Detailed Implementation
[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0054] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.
[0055] In this application, the term "exemplary" is used to mean "serving as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.
[0056] Currently, with the rapid development of artificial intelligence, natural language processing (NLP) technology has also been applied, for example, in question-answering systems and recommendation systems. Text retrieval is a crucial foundational task of NLP. Current text retrieval methods involve coarsely recalling samples and then using a deep learning model as a fine-tuning module to output relevant recommendations. The accuracy of these recommendations depends on the model's ability to identify and exclude difficult-to-reject samples. However, the quality of existing samples varies greatly, making it difficult to effectively identify and filter difficult-to-reject samples. This leads to a decrease in application accuracy due to the presence of difficult-to-reject samples.
[0057] Based on this, this application proposes a method, apparatus, device, and computer-readable storage medium for screening difficult-to-bear samples, in order to solve the technical problem that inaccurate screening of difficult-to-bear samples in the prior art leads to low recognition rates in subsequent applications.
[0058] The difficult-to-load sample screening method in this embodiment of the invention is applied to a difficult-to-load sample screening device, which is set in a difficult-to-load sample screening equipment. The difficult-to-load sample screening equipment is equipped with one or more processors, a memory, and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the difficult-to-load sample screening method. The difficult-to-load sample screening equipment can be a smart terminal, such as a mobile phone, tablet computer, smart TV, network device, and smart computer. Optionally, the difficult-to-load sample screening equipment can also be a server or a service cluster composed of multiple servers.
[0059] like Figure 1 As shown, Figure 1This is a schematic diagram of a scenario for a difficult-to-bear sample screening method according to an embodiment of this application. The difficult-to-bear sample screening scenario in this embodiment includes multiple difficult-to-bear sample screening devices 100 (each difficult-to-bear sample screening device 100 integrates a difficult-to-bear sample screening apparatus). Each difficult-to-bear sample screening device 100 is equipped with a computer-readable storage medium corresponding to the difficult-to-bear sample screening method to execute the steps of the difficult-to-bear sample screening method.
[0060] Understandable, Figure 1 The difficult sample screening device in the scenario of the difficult sample screening method shown, or the device included in the difficult sample screening device, does not constitute a limitation on the embodiments of the present invention. That is, the number or type of difficult sample screening device included in the scenario of the difficult sample screening method, or the number or type of device included in each device, does not affect the overall implementation of the technical solution in the embodiments of the present invention, and can all be considered as equivalent substitutions or derivatives of the technical solutions claimed in the embodiments of the present invention.
[0061] In this embodiment of the invention, the difficult-to-bear sample screening device 100 is mainly used for: obtaining target problem samples;
[0062] Based on the morphemes of the target question sample and the morphemes of the preset candidate samples, recall the first candidate sample of the target question sample;
[0063] Based on the semantic features of the target question sample and the semantic features of the preset candidate samples, a second candidate sample of the target question sample is recalled;
[0064] The first and second candidate samples are filtered to obtain the hard negative samples of the target problem sample.
[0065] The difficult-to-bearing sample screening device 100 in this embodiment of the invention can be multiple independent difficult-to-bearing sample screening devices, such as mobile phones, tablets, smart TVs, network devices, servers, and smart computers, or it can be a difficult-to-bearing sample screening network or a difficult-to-bearing sample screening cluster composed of multiple difficult-to-bearing sample screening devices.
[0066] This application provides a method, apparatus, device, and computer-readable storage medium for screening difficult-to-bear samples, which will be described in detail below.
[0067] It will be understood by those skilled in the art that Figure 1 The application environment shown is only one application scenario related to the solution of this application and does not constitute a limitation on the application scenario of this application. Other application environments may include more than one application scenario. Figure 1 The diagram shows more or fewer hard-to-load sample screening devices, or hard-to-load sample screening network connections, for example... Figure 1Only one difficult sample screening device is shown in the figure. It can be understood that the scenario of this difficult sample screening method may also include one or more difficult sample screening devices, which are not limited here; the difficult sample screening device may also include a memory for storing other data.
[0068] It should be noted that, Figure 1 The schematic diagram of the difficult-to-bear sample screening method shown is merely an example. The scenario of the difficult-to-bear sample screening method described in the embodiments of the present invention is for the purpose of more clearly illustrating the technical solution of the embodiments of the present invention, and does not constitute a limitation on the technical solution provided by the embodiments of the present invention.
[0069] Based on the scenarios described above for screening difficult-to-bear samples, various embodiments of the difficult-to-bear sample screening method disclosed in this invention are proposed.
[0070] like Figure 2 As shown, Figure 2 This is a flowchart illustrating one embodiment of the difficult-to-bear sample screening method in this application. The image processing method includes the following steps 201-204:
[0071] 201. Obtain a sample of the target problem;
[0072] The difficult-to-bearing sample screening method in this embodiment is applied to a difficult-to-bearing sample screening device. The type and number of difficult-to-bearing sample screening devices are not specifically limited. That is, the difficult-to-bearing sample screening device can be one or more smart terminals or servers. In a specific embodiment, the difficult-to-bearing sample screening device is a smart computer.
[0073] This difficult-negative sample screening device is configured to screen target question samples in a preset sample database to obtain one or more difficult-negative samples of each target question sample. The target question samples are training samples used to train question-answering and recommendation systems for text retrieval; the difficult-negative samples are negative samples that are similar to the target question sample but have different representational meanings and are difficult to identify. By screening each target question sample and its associated difficult-negative samples, the training accuracy of the question-answering and recommendation systems for text retrieval can be improved.
[0074] Specifically, when the difficult-to-handle sample screening device is in operation, it receives difficult-to-handle sample screening requests. These requests are instructions to the device to perform difficult-to-handle sample screening on a target question sample, obtaining difficult-to-handle samples of that target question sample. Optionally, the triggering method for these requests is not specifically limited; that is, the requests can be actively triggered by the user. For example, a user, such as a developer of a question-and-answer system or recommendation system, can trigger the request by clicking the difficult-to-handle sample screening button on the device and entering the target question sample or related search instructions. Alternatively, the requests can be automatically triggered by the device itself. For instance, the device may have a pre-set difficult-to-handle sample screening process, automatically triggering requests within a preset time period.
[0075] After receiving a difficult-to-handle sample screening request, the difficult-to-handle sample screening device obtains the target question sample pointed to by the request. This target question sample is the text of a question that a user might ask in a real-world application scenario. For example, in one specific embodiment, the target question sample pointed to by the difficult-to-handle sample screening request is "How to install application A".
[0076] After acquiring the target problem sample, the difficult-to-bear sample screening device further acquires the standard problem for that target sample. This standard problem is a pre-configured, clearly described problem. Then, using the target problem sample and its standard problem, the device performs statistical and semantic vector searches on the problem data samples in a pre-defined sample database to obtain the difficult-to-bear sample of the target problem.
[0077] 202. Based on the morphemes of the target question sample and the morphemes of the preset candidate samples, recall the first candidate sample of the target question sample;
[0078] After obtaining the target question sample and the corresponding standard question, the difficult-to-handle sample screening device further obtains the morphemes of the target question sample and the standard question, and recalls the first candidate sample of the target question sample based on the morphemes of the target question sample and the morphemes of the preset candidate sample.
[0079] Specifically, the difficult-to-task sample screening device performs morpheme analysis on the target problem sample and the standard problem to obtain each morpheme and its corresponding weight in the target problem sample, as well as each morpheme and its corresponding weight in the standard problem. Furthermore, the difficult-to-task sample screening device also obtains morphemes from each preset candidate sample in a preset sample database.
[0080] After acquiring the morphemes of the target question sample and the morphemes of each preset candidate sample, the difficult-to-bear sample screening device calculates the morpheme similarity between each morpheme of the target question sample and the morphemes of each preset candidate sample. Optionally, the difficult-to-bear sample screening device can use statistical matching algorithms such as bm25 to calculate the morpheme similarity between each morpheme of the target question sample and the morphemes of each preset candidate sample.
[0081] After obtaining the morpheme similarity between each morpheme of the target question sample and each morpheme of the preset candidate sample, the difficult negative sample screening device obtains a number of preset candidate samples that have a high similarity to the target question sample. It then compares the preset candidate samples with the standard question of the target question sample to determine the high-quality negative samples of the target question sample and uses the high-quality negative samples as the first candidate samples of the target question sample for recall.
[0082] 203. Based on the semantic features of the target question sample and the semantic features of the preset candidate samples, recall the second candidate sample of the target question sample;
[0083] After obtaining the target problem sample and the corresponding standard problem, the difficult-to-bear sample screening device further encodes the target problem sample to obtain its semantic features. Optionally, in a specific embodiment, the difficult-to-bear sample screening device encodes the target problem sample by inputting the target problem sample into a preset BERT (Bidirectional Encoder Representations from Transformers) model and vectorizing the target problem sample through the BERT model.
[0084] Specifically, after obtaining the semantic features of the target problem sample, the difficult-to-bear sample screening device performs high-dimensional mapping processing on the semantic features of the target problem sample, thereby mapping the semantic features of the target problem sample to a high-dimensional space and obtaining the high-dimensional semantic features of the target problem sample.
[0085] After obtaining the high-dimensional semantic features of the target problem sample, the difficult negative sample screening device performs a proximity search based on the high-dimensional semantic features and the high-dimensional semantic features of the preset candidate samples to determine the high-quality negative samples of the target problem sample, and uses the high-quality negative samples as the second candidate samples of the target sample for recall.
[0086] Specifically, after obtaining the high-dimensional semantic features of the target problem sample, the difficult-to-bear sample screening device calculates the semantic similarity between the high-dimensional semantic features of the target problem sample and the high-dimensional semantic features of the preset candidate samples, and recalls the second candidate sample by using the semantic similarity and the standard question of the target problem sample.
[0087] 204. Filter the first candidate sample and the second candidate sample to obtain the hard negative sample of the target problem sample.
[0088] The difficult-to-bear sample screening device performs statistical retrieval and semantic matching on the sample data in the preset sample database through the target problem sample to obtain the first candidate sample and the second candidate sample. It then further screens the first candidate sample and the second candidate sample to obtain the difficult-to-bear sample of the target problem sample.
[0089] Specifically, since the first and second candidate samples are obtained using different methods and have different similarity evaluation criteria, the difficult-to-load sample screening device normalizes the morpheme similarity of the first candidate sample and the semantic similarity of the second candidate sample after obtaining them, thereby obtaining a uniformly quantified sample similarity. Optionally, the formula for this normalization process is:
[0090] Where x′ represents the sample similarity, min(x) represents the minimum morpheme similarity or the minimum semantic similarity, and max(x) represents the maximum morpheme similarity or the maximum semantic similarity.
[0091] The difficult-to-bearing sample screening device normalizes the morpheme similarity of the first candidate sample and the morpheme similarity of the second candidate sample to obtain the sample similarity. Then, it compares this sample similarity with a preset sample similarity threshold to obtain the comparison result. Based on this comparison result, the device screens the first and second candidate samples. Optionally, this sample similarity threshold is a threshold set according to the actual scenario requirements.
[0092] Optionally, if the comparison result is that the sample similarity is higher than the preset sample similarity threshold, then the first candidate sample and the corresponding standard problem or the standard problem corresponding to the second candidate sample are obtained, and the first candidate sample and the corresponding standard problem or the standard problem corresponding to the second candidate sample are marked as the target problem sample and the hard negative sample of the standard problem of the target problem sample.
[0093] Specifically, after obtaining a difficult-to-responder sample from the target question sample, the difficult-to-responder sample screening device trains a preset model using the target question sample and the corresponding difficult-to-responder sample. The device inputs the target question sample and its standard question as positive samples, and the difficult-to-responder sample and its standard question as negative samples into the preset model for iterative training to obtain the model loss value. This model loss value is then used to correct the model parameters in the preset model, thereby iteratively generating the corresponding question-answering model. This question-answering model is configured to generate a standard question sample corresponding to the target question sample and output the preset answer text for the standard question sample.
[0094] Specifically, after generating the question model, the difficult sample screening device further receives the question-and-answer request sent by the user, obtains the initial question text in the question-and-answer request, inputs the initial question text into the question-and-answer model, and outputs the preset answer text corresponding to the initial question text through the question-and-answer model.
[0095] Specifically, after receiving the initial question text, the question-answering model obtains the initial question sample corresponding to the initial question text, identifies the difficult-to-bear sample for each preset initial question text, determines the difficult-to-bear question sample corresponding to the initial question sample, filters out the difficult-to-bear question text in the initial question sample, thereby obtaining the filtered question sample corresponding to the initial question sample, obtains the target standard question corresponding to the filtered question sample, and the preset answer text of the target standard question, and outputs the preset answer text.
[0096] In this embodiment, the difficult-to-bear sample screening device acquires a target question sample, along with its morphemes and those of preset candidate samples. It then performs a statistical retrieval based on these morphemes to recall a first candidate sample of the target question sample. Next, it acquires the semantic features of the target question sample and the preset candidate samples, and recalls a second candidate sample based on these features. Finally, it further filters the first and second candidate samples to obtain the difficult-to-bear samples of the target question sample. This achieves the identification of difficult-to-bear samples in the sample database through a combination of statistical and semantic matching, thereby improving sample screening quality and reducing the probability of false recalls due to semantic or statistical matching.
[0097] like Figure 3 As shown, Figure 3 This is a flowchart illustrating an embodiment of the difficult-to-bear sample screening method provided in this application, which recalls the first candidate sample of the target problem sample based on the similarity.
[0098] Based on the above embodiments, in this embodiment, by calculating the similarity between the morphemes of the target question sample and the morphemes of the preset candidate sample, a morpheme candidate sample is obtained, and a standard question of the morpheme candidate sample is obtained. A first candidate sample is then recalled based on the standard question. Specifically, this includes steps 301 to 302:
[0099] 301. Calculate the morpheme similarity between the morphemes of the target problem sample and the morphemes of each preset candidate sample, and obtain morpheme candidate samples with morpheme similarity higher than the preset morpheme threshold.
[0100] 302. Calculate the question similarity between the standard question of the target question sample and the standard question of the morpheme candidate sample, and recall the first candidate sample whose question similarity is lower than a preset question similarity threshold.
[0101] In this embodiment, after obtaining the morphemes of the target question sample and the morphemes of the preset candidate samples, the difficult sample screening device calculates the morpheme similarity between the morphemes of the target question sample and the morphemes of the preset candidate samples, and obtains morpheme candidate samples similar to the target question sample through the morpheme similarity. The morpheme candidate samples are the preset candidate samples of the target question sample obtained through statistical retrieval.
[0102] Specifically, the difficult sample screening device has a preset threshold for morphemes to screen each preset candidate sample. After obtaining the morpheme similarity, the difficult sample screening device compares the morpheme similarity with the preset morpheme threshold to obtain morpheme candidate samples with morpheme similarity higher than the preset morpheme threshold.
[0103] After obtaining candidate morphemes based on morpheme similarity, the difficult-to-bear sample screening device further determines whether the candidate morpheme is a negative sample of the target question sample, that is, whether the standard question of the candidate morpheme is the same as the standard question of the target question sample. Specifically, the difficult-to-bear sample screening device calculates the question similarity between the standard question of the target question sample and the standard question of the candidate morpheme, and compares the morpheme similarity with a question similarity threshold, where the question similarity threshold is a morpheme similarity threshold that characterizes whether the standard questions are consistent.
[0104] Optionally, if the question similarity between the standard question of the target question sample and the standard question of the morpheme candidate sample is lower than the question similarity threshold, then it is determined that the standard question of the target question sample is different from the standard question of the morpheme candidate sample, the target question sample and the morpheme candidate sample are negative samples of each other, the morpheme candidate sample is determined as the first candidate sample of the target question sample and recalled.
[0105] In this embodiment, the difficult-to-reject sample screening device calculates the morpheme similarity between the morphemes of the target question sample and the morphemes of each preset candidate sample, and obtains morpheme candidate samples with morpheme similarity higher than a preset morpheme threshold; it then calculates the question similarity between the standard question of the target question sample and the standard question of the morpheme candidate samples, and recalls the first candidate sample with a question similarity lower than a preset question similarity threshold. This achieves accurate acquisition of negative samples of the target question sample and reduces the probability of false recall at the statistical level.
[0106] like Figure 4 As shown, Figure 4 This is a flowchart illustrating an embodiment of the difficult-to-bear sample screening method provided in this application for recalling a second candidate sample of a target problem sample.
[0107] Based on the above embodiments, in this embodiment of the application, the difficult-to-bear sample screening device calculates the semantic similarity between the high-dimensional semantic features of the target problem sample and the high-dimensional semantic features of the preset candidate sample, and recalls a second candidate sample of the target problem sample through the semantic similarity and the standard question. Specifically, it includes steps 401 to 402:
[0108] 401. Calculate the semantic similarity between the high-dimensional semantic features of the target problem sample and the high-dimensional semantic features of the preset candidate sample, and obtain semantic candidate samples with semantic similarity higher than the preset semantic threshold;
[0109] 402. Calculate the semantic similarity between the standard question of the target question sample and the standard question of the semantic candidate sample, and recall the second candidate sample whose question similarity is lower than the preset question similarity threshold.
[0110] In this embodiment, after obtaining the high-dimensional semantic features of the target problem sample and the high-dimensional semantic features of the preset candidate samples, the difficult-to-bear sample screening device calculates the semantic similarity between the high-dimensional semantic features of the target problem sample and the high-dimensional semantic features of the preset candidate samples, and obtains semantic candidate samples similar to the target problem sample through the semantic similarity. The semantic candidate samples are the preset candidate samples of the target problem sample obtained through semantic matching.
[0111] Specifically, after obtaining the semantic similarity, the difficult-to-handle sample screening device compares the semantic similarity with a preset semantic threshold, where the preset semantic threshold is the semantic similarity threshold for screening similar samples to the target problem sample. Optionally, if the semantic similarity is higher than the preset semantic threshold, then the preset candidate sample corresponding to the semantic similarity is determined as a semantic candidate sample of the target problem sample.
[0112] After acquiring each semantic candidate sample, the difficult-to-bear sample screening device obtains the standard question of that semantic candidate sample, calculates the question similarity between the standard question of the target question sample and the standard question of the semantic candidate sample, and compares the question similarity with a preset question similarity threshold to obtain a second candidate sample of the target question sample. The question similarity threshold is a semantic similarity threshold that characterizes whether the semantic candidate sample is a negative sample of the target question sample.
[0113] Optionally, if the question similarity between the standard question of the target question sample and the standard question of the semantic candidate sample is lower than the question similarity threshold, then it is determined that the standard question of the target question sample and the standard question of the semantic candidate sample are different, the target question sample and the semantic candidate sample are negative samples of each other, the semantic candidate sample is determined as the second candidate sample of the target question sample and recalled.
[0114] In this embodiment, the difficult-to-reject sample screening device calculates the semantic similarity between the high-dimensional semantic features of the target question sample and the high-dimensional semantic features of the preset candidate samples to obtain semantic candidate samples with a semantic similarity higher than a preset semantic threshold; it then calculates the semantic similarity between the standard question of the target question sample and the standard question of the semantic candidate samples to recall a second candidate sample with a question similarity lower than a preset question similarity threshold. This achieves accurate acquisition of negative samples of the target question sample and reduces the probability of false recall at the semantic matching level.
[0115] To better implement the difficult-to-bearing sample screening method in the embodiments of this application, a difficult-to-bearing sample screening device is also provided in the embodiments of this application, such as... Figure 5 As shown, Figure 5 This is a schematic diagram of one embodiment of the difficult-to-bear sample screening device in this application. The difficult-to-bear sample screening device 500 includes:
[0116] The sample acquisition module 501 is configured to acquire samples of the target problem.
[0117] The statistical matching module 502 is configured to recall the first candidate sample of the target question sample based on the morphemes of the target question sample and the morphemes of the preset candidate samples.
[0118] The semantic matching module 503 is configured to recall a second candidate sample of the target question sample based on the semantic features of the target question sample and the semantic features of a preset candidate sample.
[0119] The secondary screening module 504 is configured to screen the first candidate sample and the second candidate sample to obtain the hard negative sample of the target problem sample.
[0120] In some embodiments of this application, the difficult-to-handle sample screening device recalls a first candidate sample of the target question sample based on the morphemes of the target question sample and the morphemes of preset candidate samples, including:
[0121] The target question sample is parsed to obtain the morphemes of the target question sample;
[0122] Access the sample database and obtain the morphemes of each preset candidate sample in the sample database;
[0123] Calculate the morpheme similarity between the morphemes of the target question sample and the morphemes of each of the preset candidate samples, and recall the first candidate sample of the target question sample based on the similarity.
[0124] In some embodiments of this application, the difficult-to-handle sample screening device calculates the morpheme similarity between the morphemes of the target question sample and the morphemes of each of the preset candidate samples, and recalls a first candidate sample of the target question sample based on the similarity, including:
[0125] Calculate the morpheme similarity between the morphemes of the target question sample and the morphemes of each preset candidate sample, and obtain morpheme candidate samples whose morpheme similarity is higher than the preset morpheme threshold.
[0126] Calculate the question similarity between the standard question of the target question sample and the standard question of the morpheme candidate sample, and recall the first candidate sample whose question similarity is lower than a preset question similarity threshold.
[0127] In some embodiments of this application, the difficult-to-handle sample screening device recalls a second candidate sample of the target problem sample based on the semantic features of the target problem sample and the semantic features of preset candidate samples, including:
[0128] The target question sample is encoded to obtain its semantic features;
[0129] The semantic features are subjected to high-dimensional mapping processing to obtain the high-dimensional semantic features of the target problem sample;
[0130] Based on the high-dimensional semantic features of the target question sample and the high-dimensional semantic features of the preset candidate samples, a second candidate sample of the target question sample is recalled.
[0131] In some embodiments of this application, the difficult-to-bear sample screening device recalls a second candidate sample of the target problem sample based on the high-dimensional semantic features of the target problem sample and the high-dimensional semantic features of the preset candidate samples, including:
[0132] Calculate the semantic similarity between the high-dimensional semantic features of the target problem sample and the high-dimensional semantic features of the preset candidate sample, and obtain semantic candidate samples whose semantic similarity is higher than the preset semantic threshold;
[0133] Calculate the semantic similarity between the standard question of the target question sample and the standard question of the semantic candidate sample, and recall a second candidate sample whose question similarity is lower than a preset question similarity threshold.
[0134] In some embodiments of this application, the difficult-to-bear sample screening device filters the first candidate sample and the second candidate sample to obtain difficult-to-bear samples of the target problem sample, including:
[0135] The morpheme similarity of the first candidate sample and the semantic similarity of the second candidate sample are normalized to obtain the sample similarity.
[0136] By comparing the sample similarity with a preset sample similarity threshold, a first candidate sample and / or a second candidate sample whose sample similarity is higher than the sample similarity threshold are obtained, thus obtaining the hard negative sample of the target problem sample.
[0137] In some embodiments of this application, after the difficult-to-bear sample screening device screens the first candidate sample and the second candidate sample to obtain the difficult-to-bear sample of the target problem sample, it further includes:
[0138] Obtain the first standard problem of the target problem sample and the second standard problem of the difficult-to-bear sample;
[0139] The target problem sample and the first standard problem are used as positive samples, and the hard negative sample and the second standard problem are used as negative samples. They are input into a preset model for iterative training to obtain the model loss value of the preset model.
[0140] The model parameters in the preset model are corrected based on the model loss value to generate a question-answering model.
[0141] In some embodiments of this application, after the difficult-to-handle sample screening device generates the question-answering model, it further includes:
[0142] Receive a question-and-answer request and obtain the initial question text from the request.
[0143] Input the initial question text into the question-answering model to obtain the initial question sample corresponding to the initial question text;
[0144] The initial problem samples are subjected to hard-to-bear sample identification, and hard-to-bear problem samples are filtered out to obtain the filtered problem samples;
[0145] Obtain the target standard question corresponding to the filtered question sample, and output the preset answer text of the target standard question.
[0146] In this embodiment, the difficult-to-bear sample screening device acquires a target question sample, along with its morphemes and those of preset candidate samples. It then performs a statistical retrieval based on these morphemes to recall a first candidate sample of the target question sample. Next, it acquires the semantic features of the target question sample and the preset candidate samples, and recalls a second candidate sample based on these features. Finally, it further filters the first and second candidate samples to obtain the difficult-to-bear samples of the target question sample. This achieves the identification of difficult-to-bear samples in the sample database through a combination of statistical and semantic matching, thereby improving sample screening quality and reducing the probability of false recalls due to semantic or statistical matching.
[0147] This invention also provides a difficult-to-bear sample screening device, such as... Figure 6 As shown, Figure 6 This is a schematic diagram of one embodiment of the difficult-to-bear sample screening device provided in this application.
[0148] The difficult-to-bearing sample screening device integrates any of the difficult-to-bearing sample screening devices provided in the embodiments of the present invention, and the difficult-to-bearing sample screening device includes:
[0149] One or more processors;
[0150] Memory; and
[0151] One or more applications, wherein the one or more applications are stored in the memory and configured by the processor to perform the steps of the difficult sample screening method described in any of the embodiments of the above-described difficult sample screening method.
[0152] Specifically, the difficult-to-handle sample screening device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will understand that... Figure 6 The structure of the difficult-to-bear sample screening device shown does not constitute a limitation on the difficult-to-bear sample screening device. It may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:
[0153] The processor 601 is the control center of the difficult-to-bearing sample screening device. It connects to various parts of the device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 602, and by calling data stored in the memory 602, it performs various functions and processes data, thereby providing overall monitoring of the difficult-to-bearing sample screening device. Optionally, the processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 601.
[0154] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the difficult-to-manage sample screening device, etc. In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.
[0155] The difficult-to-handle sample screening device also includes a power supply 603 that supplies power to the various components. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 603 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.
[0156] The difficult sample screening device may also include an input unit 604, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
[0157] Although not shown, the difficult-to-bear sample screening device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 601 in the difficult-to-bear sample screening device loads the executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 601 runs the application programs stored in the memory 602 to realize various functions, as follows:
[0158] Obtain a sample of the target problem;
[0159] Based on the morphemes of the target question sample and the morphemes of the preset candidate samples, recall the first candidate sample of the target question sample;
[0160] Based on the semantic features of the target question sample and the semantic features of the preset candidate samples, a second candidate sample of the target question sample is recalled;
[0161] The first and second candidate samples are filtered to obtain the hard negative samples of the target problem sample.
[0162] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.
[0163] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.
[0164] Therefore, embodiments of the present invention provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a disk, or an optical disk, etc. A computer program is stored thereon, which is loaded by a processor to execute the steps in any of the difficult sample screening methods provided in the embodiments of the present invention. For example, the computer program loaded by the processor can execute the following steps:
[0165] Obtain a sample of the target problem;
[0166] Based on the morphemes of the target question sample and the morphemes of the preset candidate samples, recall the first candidate sample of the target question sample;
[0167] Based on the semantic features of the target question sample and the semantic features of the preset candidate samples, a second candidate sample of the target question sample is recalled;
[0168] The first and second candidate samples are filtered to obtain the hard negative samples of the target problem sample.
[0169] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.
[0170] The above provides a detailed description of a difficult-to-bear sample screening method provided by the embodiments of this application. Specific embodiments have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for screening difficult-to-bear samples, characterized in that, The method for screening difficult-to-bear samples includes: Obtain a sample of the target problem; Based on the morphemes of the target question sample and the morphemes of preset candidate samples, a first candidate sample of the target question sample is recalled; wherein, the step of recalling the first candidate sample of the target question sample based on the morphemes of the target question sample and the morphemes of preset candidate samples includes: parsing the target question sample to obtain the morphemes of the target question sample; accessing a sample database to obtain the morphemes of each preset candidate sample in the sample database; calculating the morpheme similarity between the morphemes of the target question sample and the morphemes of each preset candidate sample, and recalling the first candidate sample of the target question sample based on the similarity; Based on the semantic features of the target question sample and the semantic features of the preset candidate samples, a second candidate sample of the target question sample is recalled; The first candidate sample and the second candidate sample are screened to obtain the hard-to-bear samples of the target problem sample; wherein, the screening of the first candidate sample and the second candidate sample to obtain the hard-to-bear samples of the target problem sample includes: normalizing the morpheme similarity of the first candidate sample and the semantic similarity of the second candidate sample to obtain the sample similarity; comparing the sample similarity with a preset sample similarity threshold, and obtaining the first candidate sample and / or the second candidate sample whose sample similarity is higher than the sample similarity threshold to obtain the hard-to-bear samples of the target problem sample.
2. The method for screening difficult-to-bear samples as described in claim 1, characterized in that, The step of calculating the morpheme similarity between the morphemes of the target question sample and the morphemes of each of the preset candidate samples, and recalling the first candidate sample of the target question sample based on the similarity, includes: Calculate the morpheme similarity between the morphemes of the target question sample and the morphemes of each preset candidate sample, and obtain morpheme candidate samples whose morpheme similarity is higher than the preset morpheme threshold. Calculate the question similarity between the standard question of the target question sample and the standard question of the morpheme candidate sample, and recall the first candidate sample whose question similarity is lower than a preset question similarity threshold.
3. The method for screening difficult-to-bear samples as described in claim 1, characterized in that, The step of recalling a second candidate sample of the target question sample based on the semantic features of the target question sample and the semantic features of preset candidate samples includes: The target question sample is encoded to obtain its semantic features; The semantic features are subjected to high-dimensional mapping processing to obtain the high-dimensional semantic features of the target problem sample; Based on the high-dimensional semantic features of the target question sample and the high-dimensional semantic features of the preset candidate samples, a second candidate sample of the target question sample is recalled.
4. The method for screening difficult-to-bear samples as described in claim 3, characterized in that, The step of recalling a second candidate sample of the target question sample based on the high-dimensional semantic features of the target question sample and the high-dimensional semantic features of the preset candidate samples includes: Calculate the semantic similarity between the high-dimensional semantic features of the target problem sample and the high-dimensional semantic features of the preset candidate sample, and obtain semantic candidate samples whose semantic similarity is higher than the preset semantic threshold; Calculate the question similarity between the standard question of the target question sample and the standard question of the semantic candidate sample, and recall a second candidate sample whose question similarity is lower than a preset question similarity threshold.
5. The method for screening difficult-to-bear samples as described in any one of claims 1-4, characterized in that, After filtering the first and second candidate samples to obtain the hard-to-bear samples of the target problem sample, the method further includes: Obtain the standard problem of the target problem sample and the standard problem of the hard-to-bear sample; The target problem sample and the standard problem of the target problem sample are used as positive samples, and the hard negative sample and the standard problem of the hard negative sample are used as negative samples. They are input into a preset model for iterative training to obtain the model loss value of the preset model. The model parameters in the preset model are corrected based on the model loss value to generate a question-answering model.
6. The method for screening difficult-to-bear samples as described in claim 5, characterized in that, Following the generation of the question-answering model, the following is also included: Receive a question-and-answer request and obtain the initial question text from the request. Input the initial question text into the question-answering model to obtain the initial question sample corresponding to the initial question text; The initial problem samples are subjected to hard-to-bear sample identification, and hard-to-bear problem samples are filtered out to obtain the filtered problem samples; Obtain the target standard question corresponding to the filtered question sample, and output the preset answer text of the target standard question.
7. A sample screening device for difficult-to-bear samples, characterized in that, The difficult-to-bear sample screening device includes: The sample acquisition module is configured to acquire samples of the target problem. A statistical matching module is configured to recall a first candidate sample of the target question sample based on the morphemes of the target question sample and the morphemes of preset candidate samples; wherein, recalling the first candidate sample of the target question sample based on the morphemes of the target question sample and the morphemes of preset candidate samples includes: parsing the target question sample to obtain the morphemes of the target question sample; accessing a sample database to obtain the morphemes of each preset candidate sample in the sample database; calculating the morpheme similarity between the morphemes of the target question sample and the morphemes of each preset candidate sample; and recalling the first candidate sample of the target question sample based on the similarity. The semantic matching module is configured to recall a second candidate sample of the target question sample based on the semantic features of the target question sample and the semantic features of a preset candidate sample; The secondary screening module is configured to screen the first candidate sample and the second candidate sample to obtain the hard-to-bear samples of the target problem sample; wherein, the screening of the first candidate sample and the second candidate sample to obtain the hard-to-bear samples of the target problem sample includes: normalizing the morpheme similarity of the first candidate sample and the semantic similarity of the second candidate sample to obtain the sample similarity; comparing the sample similarity with a preset sample similarity threshold, and obtaining the first candidate sample and / or the second candidate sample whose sample similarity is higher than the sample similarity threshold to obtain the hard-to-bear samples of the target problem sample.
8. A screening device for difficult-to-bear samples, characterized in that, The difficult-to-bear sample screening device includes: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the difficult sample screening method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to perform the steps of the difficult-to-bear sample screening method according to any one of claims 1 to 6.