Positive and negative sample generation method, system and equipment based on large language model and medium

Abstract

The invention discloses a positive and negative sample generation method, system and device based on a large language model, and a medium. The method comprises the following steps: obtaining user behavior data, and calling a data analysis technology to generate a preliminary positive sample and a preliminary negative sample according to the user behavior data; based on a word list generation technology and the preliminary positive sample, generating a difficult negative sample which is not related to the semantic related key meaning of the preliminary positive sample; inputting the difficult negative sample into a large language model for reasoning optimization; according to the method, by combining user behavior data analysis and word list technologies, the construction quality of positive and negative samples in semantic similarity retrieval, especially generation of difficult negative samples, is effectively improved, the retrieval performance and accuracy of the model are remarkably enhanced, the cost is reduced, more flexible, diversified, accurate and controllable contrast learning data is generated, and the method is suitable for popularization and application. The generalization ability of the model is enhanced, and the search accuracy and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and particularly to a method, system, device, and medium for generating positive and negative samples based on a large language model. Background Art

[0002] In the field of natural language processing (NLP), semantic similarity search is a key task and is crucial for various applications such as information retrieval, recommendation systems, question answering systems, and machine translation. With the rapid growth of Internet content, it has become particularly important to develop technologies that can accurately understand and match the semantic relationship between user queries and documents. As a self-supervised learning method, contrastive learning trains a model by constructing positive and negative sample pairs to improve the model's ability to recognize semantic similarity. Its advantage lies in being able to utilize a large amount of unlabeled data, reducing the dependence on manual annotation, and enhancing the generalization ability of the model. However, the effectiveness of contrastive learning highly depends on the construction quality of the positive and negative sample pairs, especially the selection of negative samples, which has a decisive impact on the training effect and final performance of the model.

[0003] In practical applications, generating high-quality negative samples is challenging because negative samples not only need to be semantically unrelated to the query but also have a certain degree of confusion. Existing sample generation methods, including rule-based methods, manually annotated methods, and deep learning-based methods, all have certain limitations. Rule-based methods may be too rigid to capture the complexity and diversity of data; manually annotated methods are costly, time-consuming, and may be affected by the subjectivity of annotators; deep learning-based methods can generate diverse and high-dimensional data, but may require a large amount of computing resources, and the quality of the generated samples needs additional verification.

[0004] Specifically, the method of generating samples based on rules lacks flexibility and adaptability and is difficult to capture the complexity and diversity in real-world data; manually annotated methods face cost and efficiency issues, and may lead to inconsistencies and biases in the dataset due to individual differences among annotators; deep learning-based data generation methods are powerful but also require a large amount of computing resources, may contain human biases, and the generation process lacks transparency.

[0005] Therefore, exploring a new and effective method for generating contrastive learning samples, especially a method that can generate high-quality negative samples, has important research significance and application value for improving the performance of semantic similarity search technology. Summary of the Invention

[0006] The purpose of the present invention is to provide a method, system, device, and medium for generating positive and negative samples based on a large language model, thereby solving all or one of the above problems existing in the prior art.

[0007] To solve the above technical problems, the specific technical solution of the present invention is as follows: On the one hand, the present invention provides a method for generating positive and negative samples based on a large language model, including the following steps: Positive and negative sample configuration step: Obtain user behavior data, and call data analysis technology to generate preliminary positive samples and preliminary negative samples according to the user behavior data; Negative sample hardening step: Based on the vocabulary generation technology and the preliminary positive samples, generate difficult negative samples that are semantically related to the preliminary positive samples and have non-related key meanings; Sample optimization step: Input the difficult negative samples into the large language model for inference optimization.

[0008] As an improved solution, the calling of data analysis technology to generate preliminary positive samples and preliminary negative samples according to the user behavior data includes: Determine user-related documents according to the user behavior data; screen the user-related documents based on the click-through rate index, and determine the user-related documents highly relevant to the user query as the preliminary positive samples.

[0009] As an improved solution, the calling of data analysis technology to generate preliminary positive samples and preliminary negative samples according to the user behavior data further includes: Adopt a random sampling strategy to select combinations of articles that do not match the user query in the document library as the preliminary negative samples.

[0010] As an improved solution, the screening of the user-related documents based on the click-through rate index to determine the user-related documents highly relevant to the user query as the preliminary positive samples further includes: Calculate the click-through rate corresponding to the user-related documents and the average click-through rate of the user-related documents; Use the user-related documents with a click-through rate greater than the average click-through rate as the preliminary positive samples.

[0011] As an improved solution, the vocabulary generation technology includes: According to the preliminary positive samples, use an algorithm for replacing based on core words, an algorithm for direct comparison based on core words, and an algorithm for filtering unclicked exposures to generate the difficult negative samples.

[0012] As an improved solution, the replacement algorithm based on core words includes: identifying the positive sample core words in the preliminary positive samples, replacing the positive sample core words with semantically related non-positive sample words, and using the semantically related non-positive sample words as the difficult negative samples; The direct comparison algorithm based on core words includes: generating words that are lexically similar but semantically dissimilar to the positive sample core words as the difficult negative samples; The algorithm based on filtering unclicked exposures includes: determining the items that the user has not interacted with in the content displayed on the client side, and using the items that the user has not interacted with as the difficult negative samples.

[0013] As an improved solution, inputting the difficult negative samples into the large language model for inference optimization includes: Creating a training query, putting the training query into the large language model for difficult negative sample inference, training the discrimination ability of the large language model, and obtaining the output result of the large language model; Using the proportional data of the output result as input data; Inputting both the input data and the difficult negative samples into the large language model for judgment, and calling the large language model to evaluate the effectiveness of the difficult negative samples according to semantic similarity and difference.

[0014] On the other hand, the present invention also provides a positive and negative sample generation system based on a large language model, including: A positive and negative sample configuration module, configured to: obtain user behavior data, and call data analysis techniques to generate preliminary positive samples and preliminary negative samples according to the user behavior data; A negative sample hardening module, configured to: based on the word list generation technology and the preliminary positive samples, generate difficult negative samples that are semantically related to the preliminary positive samples and have non-related key meanings; A sample optimization module, configured to: input the difficult negative samples into the large language model for inference optimization.

[0015] On the other hand, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the positive and negative sample generation method based on the large language model are implemented.

[0016] On the other hand, the present invention also provides a computer device, the computer device includes a processor, a communication interface, a memory, and a communication bus, wherein, the processor, the communication interface, and the memory complete communication with each other through the communication bus; wherein: The memory is used to store a computer program; The processor is used to execute the steps of the positive and negative sample generation method based on the large language model by running the program stored in the memory.

[0017] The beneficial effects of the technical solution of the present invention are as follows: 1. The positive and negative sample generation method based on the large language model of the present invention can effectively improve the construction quality of positive and negative samples in semantic similarity retrieval, especially the generation of difficult negative samples, by combining user behavior data analysis and vocabulary technology, significantly enhancing the retrieval performance and accuracy of the model; compared with traditional methods, it not only reduces costs and improves efficiency, but also generates more flexible, diverse, accurate and controllable contrast learning data; using the large language model to optimize the samples further improves the semantic quality and diversity of the samples, enabling the model to more accurately predict the relevance between documents and queries, enhancing the generalization ability of the model, and thus improving the accuracy and efficiency of search as a whole.

[0018] 2. The positive and negative sample generation system based on the large language model of the present invention can realize the positive and negative sample generation method based on the large language model of the present invention through the mutual cooperation of system modules.

[0019] 3. The computer-readable storage medium of the present invention can realize guiding the system modules to cooperate, and then realize the positive and negative sample generation method based on the large language model of the present invention, and the computer-readable storage medium of the present invention also effectively improves the operability of the positive and negative sample generation method based on the large language model.

[0020] 4. The computer device of the present invention can realize storing and executing the computer-readable storage medium, and then realize the positive and negative sample generation method based on the large language model of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0022] Figure 1 is a schematic flowchart of the positive and negative sample generation method based on the large language model according to Embodiment 1 of the present invention; Figure 2 is a detailed schematic flowchart of the positive and negative sample generation method based on the large language model according to Embodiment 1 of the present invention; Figure 3It is a schematic structural diagram of the positive and negative sample generation system based on the large language model described in Embodiment 2 of the present invention; Figure 4 It is a schematic structural diagram of the computer device described in Embodiment 4 of the present invention; The reference numerals in the drawings are explained as follows: 1501, processor; 1502, communication interface; 1503, memory; 1504, communication bus. Specific embodiments

[0023] The following elaborates on the preferred embodiments of the present invention in conjunction with the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0024] In the description of the present invention, it should be noted that the embodiments described in the present invention are part of the embodiments of the present invention, rather than all the embodiments; all other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts belong to the protection scope of the present invention.

[0025] The terms "first", "second", etc. in the specification, claims and above-mentioned drawings of this article are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances, so that the embodiments described in this article can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or equipment that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or equipment. Embodiment 1

[0026] This embodiment provides a method for generating positive and negative samples based on a large language model, as Figure 1 and Figure 2 shown, including the following steps: S100. Positive and negative sample configuration step, including: In this step, user behavior data is obtained, and preliminary high-quality positive and negative samples for contrast learning are generated based on user behavior data analysis technology, specifically as follows: S101. According to the user behavior data, determine the documents related to the user, screen these documents according to the click-through rate, and finally determine the documents highly relevant to the user query as positive samples; Among them, a big data platform is used to collect user behavior data, including user queries and article data, specifically: users' historical behaviors (such as exposure and click behavior data of users in the recent 30 days), preferences, query IDs, content (and category information related to the content), as well as article IDs and content.

[0027] Among them, after the above data is collected, data preprocessing and cleaning are performed on the collected data, including but not limited to: removing noise, handling missing values and outliers, so as to ensure data quality.

[0028] Among them, since the construction of positive samples is based on users' actual click behaviors, all clicked documents (docs) corresponding to the same query are determined, and the click-through rates of these documents are calculated respectively; the documents with click-through rates greater than the average click-through rate are used as positive samples; these positive samples represent the content that users are interested in and are highly relevant to users' queries, and are used as key data for training the model to identify relevant documents.

[0029] S102. Additionally, through random sampling, a combination of articles that do not match the user's query and the query is randomly selected from the document library, and this combination is used as a preliminary simple negative sample pair; these preliminary negative sample pairs are used to preliminarily train the model to help the model learn to distinguish documents that are not relevant to the query.

[0030] S200. Negative sample hardening step, including: In this step, in order to generate more challenging hard negative samples, based on the positive samples generated in the previous step and the vocabulary generation technology, hard negative samples that are semantically similar to the positive samples but different in keyword vocabulary are generated, thereby increasing the difficulty for the model to distinguish relevant and irrelevant documents. The vocabulary generation technology adopted in this step is specifically as follows: S201. Use the Core Term Replacement algorithm to generate hard negative samples: Identify the vocabulary that best represents the sample characteristics in the positive sample as the positive sample core term, replace the positive sample core term with a vocabulary that is semantically similar but does not belong to the positive sample, and use the replaced vocabulary as the hard negative sample (for example, replace the core term "ninja400" in "Kawasaki ninja400" with "z440ltd" to generate "Kawasaki z440ltd minor modification" as the hard negative sample).

[0031] S202. Generate hard negative samples using the Direct Contrast of Core Terms algorithm: Generate words that are similar to the core words of the positive samples but semantically directly opposite or inconsistent as hard negative samples (e.g., replace the core word "wl125t-27" in "Wanglong wl125t-27" with "wl125t-19" to generate "Wanglong wl125t-19" as a hard negative sample).

[0032] S203. Generate hard negative samples using the Filtered Exposure Non-Clicks algorithm: Based on user behavior data, determine the items that the user did not click or interact with among the content shown to the user, and use these items as hard negative samples (e.g., after the user queries for information about their favorite motorcycle, Ducati Panigale V4, they may see a recommendation for "BMW S1000RR", but the user did not click. Although "BMW S1000RR" is similar to "Ducati Panigale V4", the user may not have clicked due to price, brand preference, or other factors. Therefore, "BMW S1000RR" is also used as a hard negative sample).

[0033] S300. Sample optimization step, including: In this step, optimize the hard negative samples generated in the previous steps based on the text generation and understanding capabilities of the large language model (LLM) as follows: S301. Use the LLM to determine whether the generated hard negative samples are truly hard negative samples: Use a batch of queries (i.e., training queries) to perform hard negative sample inference in the LLM, and use the obtained top 50 (i.e., proportional data) as input data; Input the input data and the hard negative samples obtained in step S200 into the LLM for judgment, and call the LLM to evaluate these samples based on semantic similarity and difference to determine whether the input data and the hard negative samples obtained in step S200 are valid hard negative samples.

[0034] S302. Determine the final valid hard negative samples according to the output of the LLM, which can ensure that these hard negative samples are more challenging in contrastive learning and are easy to improve the discrimination ability of the model.

[0035] It should be noted that the LLM can generate samples that are more in line with the actual context based on the context information, thereby improving the semantic quality and diversity of the samples; Optimization based on the LLM can further screen and optimize the effectiveness of the hard negative samples, ensure the quality of the negative samples, and improve the discrimination ability and generalization ability of the model.

[0036] In one implementation, the prompt and inference process for the LLM are as follows: Input: Assume you are an expert in motorcycles, knowing many motorcycle brands, model names, series names, abbreviations, etc. Now I will give you several inputs. You need to determine their brands, model names, and other similar but different model names in the same series. If they are similar, it is a positive sample; if not, it is a negative sample. You should judge based on the core words in the sentence. The core words are the brand and model name of the motorcycle. If the model name is not mentioned, judge according to the semantics. The numbers in the motorcycle model are very important. Generally, if the numbers are different, they are not relevant. For example, Fuyi ys150 and Fuyi ys250, but Fuyi ys150 and Feizhi 150 are marked as uncertain. Judgment criteria: 1. If the "brand" is the same and the "numbers in the model" are the same, it is determined as a positive sample. 2. If the "brand" is different or the "numbers in the model" are different, it is determined as a difficult negative sample. Thought process: 1. Analyze one by one first, and then output the analysis results. 2. Then output the negative samples in jsonl format. Thought pattern: (The thought process does not need to be output) "Yamaha TMAX": ["Small-displacement sports scooter, Yamaha X-Force155, three-eye appearance is alternative and individual", query: "Yamaha TMAX" The brand name mentioned in the query query_brand: "Yamaha" The model name mentioned in the query query_moto: "TMAX" title: "Small-displacement sports scooter, Yamaha X-Force155, three-eye appearance is alternative and individual" The brand name mentioned in the title title_brand: "Yamaha" The model name mentioned in the title title_moto: "X-Force155" Analysis: The title mentions "Yamaha X-Force155", which does not match "Yamaha TMAX", so it is a negative sample. Conclusion: Negative sample (label: 0) The final output in jsonl format is as follows: {"query": "Yamaha TMAX", "query_brand": "Yamaha", "query_moto": "TMAX", "title": "Small-displacement sports scooter, Yamaha X-Force155, three-eye appearance is alternative and individual", "title_brand": "Yamaha", "title_moto": "X-Force155", "label": 0}. It should be noted that the above examples are only for explaining the present invention and should not limit the protection scope of the present invention accordingly. Example 2

[0037] Based on the same inventive concept as the positive and negative sample generation method based on a large language model described in Example 1, this example provides a positive and negative sample generation system based on a large language model, as Figure 3 shown, including: A positive and negative sample configuration module, configured to: obtain user behavior data, and call data analysis technology to generate preliminary positive samples and preliminary negative samples according to the user behavior data; A negative sample hardening module, configured to: generate difficult negative samples that are semantically related to the preliminary positive samples and have non-related key meanings based on a vocabulary generation technology and the preliminary positive samples; A sample optimization module, configured to: input the difficult negative samples into a large language model for inference optimization. Example 3

[0038] This example provides a computer-readable storage medium, including: The storage medium is used to store computer software instructions for implementing the positive and negative sample generation method based on a large language model described in Example 1 above, and it contains a program set for executing the positive and negative sample generation method based on a large language model described above; specifically, this executable program can be built into the positive and negative sample generation system based on a large language model described in Example 2. In this way, the positive and negative sample generation system based on a large language model can implement the positive and negative sample generation method based on a large language model described in Example 1 by executing the built-in executable program.

[0039] In addition, the computer-readable storage medium of this example can adopt any combination of one or more readable storage media, where the readable storage media include electrical, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. Example 4

[0040] This example provides an electronic device, as Figure 4 shown, the electronic device may include: a processor 1501, a communication interface 1502, a memory 1503, and a communication bus 1504, where the processor 1501, the communication interface 1502, and the memory 1503 complete mutual communication through the communication bus 1504.

[0041] The memory 1503 is used to store a computer program; The processor 1501, when executing the computer program stored in the memory 1503, implements the steps of the method for generating positive and negative samples based on the large language model described in Embodiment 1 above.

[0042] As an implementation manner of the present invention, the communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, Figure 4 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0043] As an implementation manner of the present invention, the communication interface is used for communication between the above terminal and other devices.

[0044] As an implementation manner of the present invention, the memory may include a Random Access Memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0045] As an implementation manner of the present invention, the above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0046] Different from the prior art, by adopting a positive and negative sample generation method, system, device and medium based on a large language model in this application, the construction quality of positive and negative samples in semantic similarity retrieval can be effectively improved by combining user behavior data analysis and vocabulary technology, especially the generation of difficult negative samples, significantly enhancing the retrieval performance and accuracy of the model; compared with traditional methods, it not only reduces costs and improves efficiency, but also generates more flexible, diverse, accurate and controllable contrast learning data; using the large language model to optimize the samples further improves the semantic quality and diversity of the samples, enabling the model to more accurately predict the relevance between documents and queries, enhancing the generalization ability of the model, and thus improving the accuracy and efficiency of search as a whole.

[0047] It should be understood that in various embodiments of this article, the sequence numbers of the above processes do not mean the order of execution is prior or posterior. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this article.

[0048] It should also be understood that in the embodiments of this article, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after.

[0049] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but this implementation should not be considered to exceed the scope of this article.

[0050] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described system, device, and unit can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0051] In several embodiments provided in this document, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Additionally, the displayed or discussed couplings or direct couplings or communication connections between each other can be indirect couplings or communication connections through some interfaces, devices, or units, and can also be electrical, mechanical, or other forms of connection.

[0052] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the objectives of the solution of the embodiments in this document.

[0053] In addition, in each embodiment of this document, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0054] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solution in this document, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of this document. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0055] The above are only the embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. All equivalent structural or equivalent process transformations made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, are similarly included in the patent protection scope of the present invention.

Claims

1. A method for generating positive and negative samples based on a large language model, characterized in that: The following steps are involved: Positive and negative sample configuration steps: Acquire user behavior data, and use data analysis technology to generate preliminary positive samples and preliminary negative samples based on the user behavior data; Negative sample difficulty steps: Based on the vocabulary generation technology and the preliminary positive samples, generate difficult negative samples that are semantically related to the preliminary positive samples and have irrelevant key meanings; Sample optimization steps: The difficult negative samples are input into a large language model for inference optimization.

2. The method for generating positive and negative samples based on a large language model according to claim 1, characterized in that: The call data analysis technology generates preliminary positive samples and preliminary negative samples according to the user behavior data, including: Determine user-related documents according to the user behavior data; screen the user-related documents based on a click-through rate indicator, and determine user-related documents that are highly relevant to the user query as the preliminary positive samples.

3. The method for generating positive and negative samples based on a large language model according to claim 1, characterized in that: The call data analysis technology generates preliminary positive samples and preliminary negative samples according to the user behavior data, and further includes: A random sampling strategy is adopted to select a combination of articles and queries that do not match the user query from the document library as the preliminary negative samples.

4. The method for generating positive and negative samples based on a large language model according to claim 2, characterized in that: The step of screening the user-related documents based on the click rate index and determining the user-related documents that are highly relevant to the user query as the preliminary positive samples further includes: Calculate the click-through rate of the user-related documents and the average click-through rate of the user-related documents; The user-related documents whose click-through rates are greater than the average click-through rate are taken as the preliminary positive samples.

5. The method for generating positive and negative samples based on a large language model according to claim 1, characterized in that: The vocabulary generation technology includes: According to the preliminary positive samples, the difficult negative samples are generated by adopting a core word-based replacement algorithm, a core word-based direct comparison algorithm, and a filtered exposure-but-not-click algorithm.

6. The method for generating positive and negative samples based on a large language model according to claim 5, characterized in that: The core word-based replacement algorithm includes: identifying positive sample core words in the preliminary positive samples, replacing the positive sample core words with semantically related non-positive sample words, and using the semantically related non-positive sample words as the difficult negative samples; The core word-based direct comparison algorithm includes: generating words that are similar in vocabulary to the core words of the positive sample but not in semantics as the difficult negative samples; The algorithm based on filtering exposure without clicking includes: determining the items in the content displayed on the user side without user interaction, and using the items without user interaction as the difficult negative samples.

7. The method for generating positive and negative samples based on a large language model according to claim 1, characterized in that: The step of inputting the difficult negative samples into a large language model for inference optimization includes: Creating a training query, putting the training query into the large language model for difficult negative sample reasoning, training the discrimination ability of the large language model, and obtaining an output result of the large language model; Using the proportional data of the output result as input data; The input data and the difficult negative samples are both input into the large language model for judgment, and the large language model is called to evaluate the validity of the difficult negative samples according to semantic similarity and difference.

8. A positive and negative sample generation system based on a large language model, characterized in that: include: A positive and negative sample configuration module is used to: obtain user behavior data, and call data analysis technology to generate preliminary positive samples and preliminary negative samples according to the user behavior data; A negative sample difficulty module, used to: generate a difficult negative sample that is semantically relevant to the preliminary positive sample and has no key meaning related to the preliminary positive sample based on the vocabulary generation technology and the preliminary positive sample; The sample optimization module is used to: input the difficult negative samples into the large language model for inference optimization.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the method for generating positive and negative samples based on a large language model according to any one of claims 1 to 7 are implemented.

10. A computer device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; wherein: The memory is used to store computer programs; The processor is used to execute the steps of the method for generating positive and negative samples based on a large language model according to any one of claims 1 to 7 by running the program stored in the memory.

Images