Method and apparatus for generating sample data

Abstract

This disclosure provides a method and apparatus for generating sample data, relating to the fields of data processing and artificial intelligence, specifically big data, intelligent search, and deep learning. The specific implementation involves: obtaining a list of search result records corresponding to a search request; determining the search parameters corresponding to each search result record in the list based on the obtained historical search records; and generating sample data based on the search parameters, the search request, and the list of search result records for each search result record. This method can generate sample data with a relatively high degree of objectivity, thereby improving the efficiency and reliability of sample data generation.

Description

Technical Field

[0001] This disclosure relates to the fields of data processing and artificial intelligence, specifically to the fields of big data, intelligent search, deep learning, etc., and in particular to a method and apparatus for generating sample data. Background Technology

[0002] Sample data can be used for model training and optimization. For example, in search scenarios, sample data can be used for training and optimizing search models.

[0003] In some embodiments, sample data is mainly obtained through manual annotation; correspondingly, sample data can also be referred to as labeled data. Summary of the Invention

[0004] This disclosure provides a method and apparatus for generating sample data to improve the validity of sample data.

[0005] According to a first aspect of this disclosure, a method for generating sample data is provided, comprising:

[0006] Retrieve the list of search results records corresponding to the search request;

[0007] Based on the obtained historical search records, determine the search parameters corresponding to each search result record in the search result record list;

[0008] Sample data is generated based on the search parameters corresponding to each search result record, the search request, and the list of search result records.

[0009] According to a second aspect of this disclosure, an apparatus for generating sample data is provided, comprising:

[0010] The acquisition unit is used to retrieve a list of search result records corresponding to the search request.

[0011] The first determining unit is used to determine the search parameters corresponding to each search result record in the search result record list based on the acquired historical search records.

[0012] The generation unit is used to generate sample data based on the search parameters corresponding to each search result record, the search request, and the list of search result records.

[0013] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0017] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method according to the first aspect.

[0018] According to a fifth aspect of this disclosure, a computer program product is provided, the computer program product comprising: a computer program stored in a readable storage medium, wherein at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the method described in the first aspect.

[0019] This disclosure provides a method and apparatus for generating sample data, comprising: obtaining a list of search result records corresponding to a search request; determining the search parameters corresponding to each search result record in the list of search result records based on the obtained historical search records; and generating sample data based on the search parameters corresponding to each search result record, the search request, and the list of search result records. By determining the search parameters corresponding to each search result record based on historical search records and combining the search parameters to generate sample data, the technical feature of generating sample data from a more objective perspective can be achieved, thereby improving the efficiency and reliability of sample data generation.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0021] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0022] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0023] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure;

[0024] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure;

[0025] Figure 4 This is a schematic diagram illustrating the principle of the editing process in an embodiment of this disclosure;

[0026] Figure 5This is a schematic diagram according to the fourth embodiment of the present disclosure;

[0027] Figure 6 This is a schematic diagram according to the fifth embodiment of the present disclosure;

[0028] Figure 7 This is a schematic diagram according to the sixth embodiment of the present disclosure;

[0029] Figure 8 This is a block diagram of an electronic device used to implement the sample data generation method of the embodiments of this disclosure. Detailed Implementation

[0030] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0031] Search engine technology is a widely used information technology, with applications in areas such as mobile internet and enterprise services.

[0032] When search engine technology is applied to mobile internet scenarios, it can index web page content and provide refined services such as retrieval and ranking based on user search requests.

[0033] For example, a search system is a system implemented based on search engine technology. A search system can receive search requests initiated by users, obtain and return search results corresponding to the search requests based on search engine technology.

[0034] When search engine technology is applied to enterprise service scenarios, it is mainly used in two aspects: one is the search of internal information within the enterprise, and the other is to provide search capabilities for external information applications.

[0035] Especially when search engine technology is applied to enterprise service scenarios, the architectural design requirements for search systems are lightweight, easy to maintain, and scalable. At the same time, as a product function related to specific data, the search effectiveness supported by the search system must be considered as user habits evolve and databases are constantly updated.

[0036] Therefore, both the construction of search systems and the evaluation of their search performance are important aspects of search engine technology.

[0037] With the development of artificial intelligence technology, in some embodiments, a search model (also known as a query model, etc.) can be constructed to achieve the retrieval and ranking output of search requests based on the search model.

[0038] For example, sample data can be obtained to train a basic network model, thereby obtaining a search model. A search system can be built based on this search model to recall search requests. Furthermore, the search performance of the search model can be evaluated based on the sample data to optimize the search model.

[0039] In other words, both the training and optimization of the search model heavily rely on the sample data. The quality of the sample data determines the search performance of the search model to a certain extent, and also determines the optimization performance of the search model to a certain extent.

[0040] In some embodiments, sample data is primarily obtained through annotation; therefore, sample data can also be referred to as labeled data. Specifically, sample data can be obtained through manual annotation.

[0041] For example, based on the search request, a list of search results is obtained, which includes search result records. These records are then output one by one, allowing annotators to rate the currently output search result record.

[0042] Accordingly, in response to the ratings assigned to each search result record by the annotators, the search system processes each search result record by rating, such as recording and retaining the ratings assigned to each search result record and exporting the sample data.

[0043] The sample data includes multiple data entries. Each search result record corresponds to one data entry in the sample data, and each data entry in the sample data includes the search request, the search result record, and the rating.

[0044] However, when using the above methods to obtain sample data, it is difficult for annotators to accurately quantify the score of each search record result. Especially when scoring the search results of massive search requests, the resources and costs are high, making it difficult to implement on a large scale. Moreover, due to the influence of human subjective factors, the accuracy and reliability of the sample data are relatively low.

[0045] To avoid the aforementioned technical problems, this disclosure provides a creative technical concept: generating sample data by combining historical search records, search requests, and a list of search results records.

[0046] Based on the above technical concept, this disclosure provides a method and apparatus for generating sample data, which are applied in the fields of data processing and artificial intelligence, specifically involving big data, intelligent search, deep learning and other technical fields, so as to improve the effectiveness and reliability of sample data generation.

[0047] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure, as shown below. Figure 1 As shown, the method for generating sample data in this embodiment of the disclosure includes:

[0048] S101: Get the list of search results records corresponding to the search request.

[0049] For example, the execution subject of this embodiment can be a sample data generation device (hereinafter referred to as the generation device). The generation device can be a server, a terminal device, a processor, a chip, etc., which will not be listed here.

[0050] If the generating device is a server, it can be a local server, a cloud server, a server cluster, or a standalone server; this embodiment does not impose any limitations.

[0051] The search results record column includes one or more search results records. The content of the search results record may differ depending on the search scenario. For example, the search results record can be a title record (also known as a name record, etc.).

[0052] For example, a search system may retrieve one or more search result records for a single search request. The content of these search result records may differ depending on the search scenario; for instance, a search result record might be a title record. Accordingly, the list of search result records may include one or more title records.

[0053] For example, in response to a search request, the search system may recall a single search result record, which will then be included in the list of search result records; the search system may also recall multiple search result records, which will then be included in the list of search result records, and a single search result record can be a title record.

[0054] The search results records in the search results record list can be all search results records or only a portion of the search results records.

[0055] For example, if the number of search result records is N (N is a positive integer greater than or equal to 1), the search result record list can include N search result records, or it can include a subset of search result records obtained from the N search result records.

[0056] S102: Based on the obtained historical search records, determine the search parameters corresponding to each search result record in the search result record list.

[0057] For example, historical search records can be understood as historical search records obtained based on the search system performing search tasks. Historical search records include one or more of the following: historical search requests, historical recall results corresponding to historical search requests, and browsing records of historical recall results.

[0058] The search task may include a search task in response to a search request.

[0059] This embodiment does not limit the method of obtaining historical search records. For example, the generating device can be communicatively connected to a search system, the search system can store historical search records and transmit them to the generating device, and the generating device can then obtain the historical search records.

[0060] Search parameters can be understood as browsing-related parameters recorded in the search results, such as the click-through rate recorded in the search results.

[0061] S103: Generate sample data based on the search parameters, search requests, and search result record list corresponding to each search result record.

[0062] For example, the search parameters are determined based on historical search records. Relatively speaking, the search parameters can objectively and accurately represent the search information recorded in the search results. By combining the search parameters to generate sample data, the drawback of low accuracy of sample data caused by human methods in the above embodiments can be avoided, thereby improving the effectiveness and reliability of the sample data.

[0063] Based on the above analysis, this disclosure provides a method for generating sample data, including: obtaining a list of search result records corresponding to a search request; determining the search parameters corresponding to each search result record in the list of search result records based on the obtained historical search records; and generating sample data based on the search parameters corresponding to each search result record, the search request, and the list of search result records. In this embodiment, by determining the search parameters corresponding to each search result record based on historical search records and combining the technical features of generating sample data with each search parameter, it is possible to generate sample data from a more objective dimension, thereby improving the efficiency and reliability of sample data generation.

[0064] As can be seen from the above examples, the search parameter can be the click-through rate. To facilitate readers' deep understanding of the implementation principle of this disclosure, the following is a combination of... Figure 2 The method for generating sample data in this disclosure will be described in more detail from the perspective of click-through rate. Among other things, Figure 2This is a schematic diagram based on the second embodiment of the present disclosure, as shown below. Figure 2 As shown, the method for generating sample data in this embodiment of the disclosure includes:

[0065] S201: Get the list of search results records corresponding to the search request.

[0066] It should be understood that, in order to avoid tedious descriptions, the same technical features as those in the above embodiments will not be repeated in this embodiment.

[0067] For example, the implementation principle of S201 can be found in the description of S101, which will not be repeated here.

[0068] S202: Based on the obtained historical search records, determine the search parameters corresponding to each search result record in the search result record list.

[0069] For example, the implementation principle of S202 can be found in the description of S102, which will not be repeated here.

[0070] S203: Calculate the score corresponding to each search result record based on the search parameters corresponding to each search result record.

[0071] Based on the above analysis, it can be seen that in some embodiments, the rating of each search result record can be determined based on the rating operation of the annotator. However, this method of determining the rating is inefficient and inaccurate.

[0072] In view of this, in this embodiment, the score is determined by combining search parameters. Since search parameters can objectively represent the search information recorded in the search results, determining the score based on search parameters can avoid the influence of human factors on the accuracy of the score, thereby improving the effectiveness and reliability of the score.

[0073] In some embodiments, the search parameter is the click-through rate; S203 may include the following steps:

[0074] First step: Obtain the click-through rate of the first search result record in the search results record list.

[0075] Specifically, click-through rate (CTR) can be understood as the click-through rate of the actual search result display location, and the CTR can be a normalized CTR. For example, when the search system retrieves search result records, it can output the search result records through a display device connected to the search system; the location where the search result records are output can be called the actual search result display location.

[0076] The second step: For each search result record, determine the score of that search result record based on its click-through rate and the click-through rate of the first search result record.

[0077] For example, the search result record list includes n (n is a positive integer greater than or equal to 1) search result records, which are respectively called search result record 1, search result record 2, and so on up to search result record n.

[0078] Accordingly, for any search result record i (1≤i≤n) among the n search result records, the score of search result record i can be calculated based on the click-through rate of search result record 1 and the click-through rate of search result record i.

[0079] In this embodiment, by combining the click-through rate of the first search result record with the click-through rates of other search result records, the rating of other search result records can be determined. This can achieve a high correlation between the rating and the click-through rate, thereby making the rating more objective and improving the effectiveness and reliability of the rating.

[0080] In some embodiments, the second step may include: for each search result record, calculating the ratio of the click-through rate of that search result record to the click-through rate of the first search result record, and determining the ratio as the score of that search result record.

[0081] For example, combining the above examples, the score rel of search result record i can be calculated based on Equation 1. i Formula 1:

[0082]

[0083] Among them, C i Let C1 be the click-through rate of search result record i, and C1 be the click-through rate of the first search result record.

[0084] S204: Generate sample data based on the corresponding rating, search request, and search result record list for each search result record.

[0085] Correspondingly, because the scoring has high objectivity, effectiveness, and reliability, the sample data generated based on the scoring has high technical effectiveness and reliability.

[0086] In other embodiments, sample data can also be generated in conjunction with editing operations. Now, in conjunction with... Figure 3 The method for generating sample data in this disclosure embodiment is described in detail from the perspective of editing operations. Among them, Figure 3 This is a schematic diagram based on the third embodiment of the present disclosure, as shown below. Figure 3 As shown, the method for generating sample data in this embodiment of the disclosure includes:

[0087] S301: Get the list of search results records corresponding to the search request.

[0088] Similarly, to avoid tedious descriptions, the same technical features as those in the above embodiments will not be repeated in this embodiment.

[0089] For example, the implementation principle of S301 can be found in the description of S101, which will not be repeated here.

[0090] S302: Output a list of search request and search result records.

[0091] For example, the generating device includes a display, which can output and display a list of search requests and search results records.

[0092] Correspondingly, the annotators can see a list of search requests and search results on the monitor.

[0093] S303: In response to an edit operation on the search results record list, edit the search results record list according to the edit operation to obtain an edited search results record list.

[0094] For example, annotators can edit the list of search results records using a monitor and editing devices (such as a mouse and / or keyboard). Correspondingly, the generating device can perform editing processing on the search results records based on the annotators' editing operations, thereby obtaining the edited list of search results records.

[0095] In this embodiment, by combining editing operations to edit the search results record list and then scoring the edited search results record list, the generation of sample data can take into account both click-through rate and editing processing, thereby further improving the effectiveness and reliability of the sample data.

[0096] In some embodiments, the editing process includes at least one of adding, deleting, and reordering processes.

[0097] For example, editing processes may include adding, deleting, or rearranging elements. Of course, editing processes may also include adding and deleting elements, adding and rearranging elements, or deleting and rearranging elements. Alternatively, editing processes may include adding, deleting, and rearranging elements.

[0098] For example, taking the addition of a new process as an example, such as Figure 4 As shown, a search result record n+1 can be added after the search result record n.

[0099] For example, taking deletion as an example, such as Figure 4As shown, search result record 2 can be deleted from the search result record list.

[0100] For example, taking the process of adjusting the order as an example, such as Figure 4 As shown, search result record 2 can be moved to the position of the first search result record in the search result record list.

[0101] It should be understood that, Figure 4 This is intended only to illustrate editing processes and should not be construed as a limitation on editing processes.

[0102] For example, this embodiment does not limit the number of times the addition process, deletion process, and order adjustment process are performed, nor does it limit the position of the addition process, deletion process, and order adjustment process.

[0103] For example, if the editing process includes multiple methods, such as two or three, then there is no restriction on the order in which the editing processes are performed.

[0104] In some embodiments, if the editing process includes multiple methods, and the multiple methods include a sequence adjustment process, then the editing process of other methods can be performed first, and then the sequence adjustment process can be performed.

[0105] For example, if the editing process includes adding a new element and adjusting the order, the generating device can perform the adding a new element first and then the adjusting the order.

[0106] For example, if the editing process includes deletion processing and order adjustment processing, the generating device can perform the deletion processing first and then perform the order adjustment processing.

[0107] For example, if the editing process includes adding, deleting, and adjusting the order, the generating device can first perform the adding and deleting processes (either by performing the adding process first and then the deleting process, or by performing the deleting process first and then the adding process), and then perform the adjusting the order process.

[0108] In this embodiment, the editing process of the search results record list can be achieved by adding, deleting, or rearranging the order, thus enabling flexibility and diversity in the editing process.

[0109] S304: Search result records that have been deleted from the search result record list are identified as negative example data. The sample data includes negative example data.

[0110] For example, if a search result record is deleted, it indicates that the correlation between that search result record and the search request was relatively low. In other words, users are unlikely to click on or view the search result record retrieved by the search system based on the search request. Therefore, identifying this search result record as negative example data can increase the amount of sample data and, more importantly, improve the effectiveness and reliability of the negative example data.

[0111] In some embodiments, a negative example data includes: a search request, a search result record that has been deleted, and may also include the rating corresponding to the deleted search result record. The rating may be implemented based on the above example or in other ways, and this embodiment does not limit it.

[0112] S305: Based on the obtained historical search records, determine the search parameters corresponding to each search result record in the edited search result record list.

[0113] For example, the implementation principle of S305 can be found in the description of S102, which will not be repeated here.

[0114] S306: Generate positive example data based on the search parameters, search requests, and the edited list of search result records for each search result record. The sample data includes positive example data.

[0115] For example, referring to the above example, this step can be understood as: generating positive example data based on the remaining search result records. Here, the remaining search result records are the search result records other than the search structure records that have undergone deletion processing.

[0116] In this embodiment, by combining the remaining search results records to generate positive example data, the positive example data can be relatively highly matched with the search request, thereby making the positive example data more reliable and effective.

[0117] In some embodiments, a positive example data includes: a search request, a record of remaining search results, and search parameters corresponding to the record of remaining search results (specifically, a score determined based on each search parameter as described in the example above).

[0118] In some embodiments, the Normalized Discounted Cumulative Gain (NDCG) of the edited search result record list can be calculated based on the scores corresponding to each search result record, in order to evaluate the edited search result record list.

[0119] For example, the normalized cumulative loss gain NDCG can be calculated based on Equation 2: Equation 2:

[0120]

[0121] in, or, IDCG n DCG for the best pre-set arrangement n rel1 is the rating corresponding to the first search result record in the edited search result record list, n is the number of search result records in the edited search result record list, and i is the i-th search result record in the edited search result record list.

[0122] S307: Train and / or optimize the search model based on the sample data.

[0123] For example, in the training scenario of the search model, the search model can be trained based on sample data, and the search model can be used to recall and return a list of search results records for the received search requests.

[0124] Because the sample data has high validity and reliability, and includes both positive and negative examples, the search model trained on the sample data has strong recall capability, thereby improving the validity and reliability of the search model.

[0125] For optimization scenarios, such as when a search model has already been trained, it can be optimized based on sample data to obtain an optimized search model, thereby improving the recall effectiveness and reliability of the optimized search model.

[0126] Figure 5 This is a schematic diagram based on the fourth embodiment of the present disclosure, as shown below. Figure 5 As shown, the sample data generation apparatus 500 of this embodiment includes:

[0127] The acquisition unit 501 is used to acquire the list of search result records corresponding to the search request.

[0128] The first determining unit 502 is used to determine the search parameters corresponding to each search result record in the search result record list based on the acquired historical search records.

[0129] The generation unit 503 is used to generate sample data based on the search parameters, search requests, and search result record list corresponding to each search result record.

[0130] Figure 6 This is a schematic diagram based on the fifth embodiment of the present disclosure, as shown below. Figure 6 As shown, the sample data generation apparatus 600 of this embodiment includes:

[0131] The acquisition unit 601 is used to acquire the list of search result records corresponding to the search request.

[0132] The first determining unit 602 is used to determine the search parameters corresponding to each search result record in the search result record list based on the acquired historical search records.

[0133] The generation unit 603 is used to generate sample data based on the search parameters, search requests, and search result record list corresponding to each search result record.

[0134] Combination Figure 6 It is understood that, in some embodiments, the generation unit 603 includes:

[0135] The calculation subunit 6031 is used to calculate the score corresponding to each search result record based on the search parameters corresponding to each search result record.

[0136] In some embodiments, the search parameter is the click-through rate; the calculation subunit 6031 includes:

[0137] The acquisition module is used to retrieve the click-through rate of the first search result record in the list of search results records.

[0138] The determination module is used to determine the score of each search result record based on its click-through rate and the click-through rate of the first search result record.

[0139] In some embodiments, the determining module is configured to, for each search result record, calculate the ratio of the click-through rate of that search result record to the click-through rate of the first search result record, and determine the ratio as the score of that search result record.

[0140] The generation subunit 6032 is used to generate sample data based on the corresponding rating, search request, and search result record list of each search result record.

[0141] The output unit 604 is used to output the search request and the list of search results records.

[0142] The editing unit 605 is configured to respond to an editing operation on the search result record list, and edit the search result record list according to the editing operation to obtain an edited search result record list.

[0143] In some embodiments, the editing process includes at least one of adding, deleting, and reordering processes.

[0144] Editing processes include deletion processes; the sample data generation apparatus also includes:

[0145] The second determining unit 606 is used to determine the search result records that have been deleted from the search result record list as negative example data, wherein the sample data includes negative example data.

[0146] In some embodiments, the generation unit 603 is configured to generate positive example data based on the search parameters corresponding to each of the remaining search result records, the search request, and the remaining search result records.

[0147] In the search results record list, the search results records other than those that have been deleted are the remaining search results records; the sample data includes positive example data.

[0148] In some embodiments, sample data is used to train and / or optimize the search model.

[0149] Figure 7 This is a schematic diagram based on the sixth embodiment of the present disclosure, as shown below. Figure 7 As shown, the electronic device 700 in this disclosure may include a processor 701 and a memory 702.

[0150] Memory 702 is used to store programs. Memory 702 may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; memory may also include non-volatile memory, such as flash memory. Memory 702 is used to store computer programs (such as application programs, functional modules, etc. that implement the above methods), computer instructions, etc. The computer programs, computer instructions, etc., can be partitioned and stored in one or more memories 702. Furthermore, the computer programs, computer instructions, data, etc., can be accessed by processor 701.

[0151] The aforementioned computer programs and instructions can be stored in one or more partitions of memory 702. Furthermore, the aforementioned computer programs and instructions can be invoked by processor 701.

[0152] The processor 701 is configured to execute the computer program stored in the memory 702 to implement the various steps in the methods described in the above embodiments.

[0153] For details, please refer to the relevant descriptions in the preceding method embodiments.

[0154] The processor 701 and the memory 702 can be independent structures or integrated structures. When the processor 701 and the memory 702 are independent structures, the memory 702 and the processor 701 can be coupled together via bus 703.

[0155] The electronic device in this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principle are the same, and will not be repeated here.

[0156] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0157] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0158] According to embodiments of this disclosure, this disclosure also provides a computer program product comprising: a computer program stored in a readable storage medium, at least one processor of an electronic device being able to read the computer program from the readable storage medium, and the at least one processor executing the computer program causing the electronic device to perform the scheme provided in any of the above embodiments.

[0159] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0160] like Figure 8As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0161] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0162] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the sample data generation method. For example, in some embodiments, the sample data generation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the sample data generation method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the sample data generation method by any other suitable means (e.g., by means of firmware).

[0163] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0164] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0165] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0166] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0167] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0168] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0169] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0170] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for generating sample data, comprising: Retrieve the list of search results records corresponding to the search request; Based on the obtained historical search records, determine the search parameters corresponding to each search result record in the search result record list; The search parameter is the click-through rate; Get the click-through rate of the first search result record in the list of search results records; For each search result record, calculate the ratio of the click-through rate of that search result record to the click-through rate of the first search result record, and determine the ratio as the score of that search result record; Sample data is generated based on the corresponding ratings for each search result record, the search request, and the list of search result records.

2. The method according to claim 1, after obtaining the list of search result records corresponding to the search query request, the method further includes: Output the search request and the list of search results records; In response to an edit operation on the search results record list, the search results record list is edited according to the edit operation to obtain an edited search results record list.

3. The method according to claim 2, wherein, The editing process includes at least one of the following: adding, deleting, and reordering.

4. The method according to claim 3, wherein the editing process includes the deletion process; the method further includes: The search results records that have undergone the deletion process in the search results record list are identified as negative example data, wherein the sample data includes the negative example data.

5. The method according to claim 4, wherein, Based on the search parameters corresponding to each search result record, the search request, and the list of search result records, sample data is generated, including: Based on the search parameters corresponding to each of the remaining search results records, the search request, and the remaining search results records, positive example data is generated; In the search results record list, the search results records other than those that have been deleted are the remaining search results records; the sample data includes the positive example data.

6. The method according to any one of claims 1-5, wherein, The sample data is used to train and / or optimize the search model.

7. A sample data generation apparatus, comprising: The acquisition unit is used to retrieve a list of search result records corresponding to the search request. The first determining unit is used to determine the search parameters corresponding to each search result record in the search result record list based on the acquired historical search records. The search parameter is the click-through rate; A generation unit is used to obtain the click-through rate of the first search result record in the search result record list; For each search result record, calculate the ratio of the click-through rate of that search result record to the click-through rate of the first search result record, and determine the ratio as the score of that search result record; Sample data is generated based on the corresponding ratings for each search result record, the search request, and the list of search result records.

8. The apparatus according to claim 7, further comprising: The output unit is used to output the search request and the list of search results records; An editing unit is configured to respond to an editing operation on the search result record list, and edit the search result record list according to the editing operation to obtain an edited search result record list.

9. The apparatus according to claim 8, wherein, The editing process includes at least one of the following: adding, deleting, and reordering.

10. The apparatus of claim 9, wherein the editing process includes the deletion process; the apparatus further comprises: The second determining unit is used to determine the search result records that have undergone the deletion process in the search result record list as negative example data, wherein the sample data includes the negative example data.

11. The apparatus according to claim 10, wherein, The generation unit is used to generate positive example data based on the search parameters corresponding to each of the remaining search result records, the search request, and the remaining search result records. In the search results record list, the search results records other than those that have been deleted are the remaining search results records; the sample data includes the positive example data.

12. The apparatus according to any one of claims 7-11, wherein, The sample data is used to train and / or optimize the search model.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

15. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-6.

Images