Training method of search recall model and search recall method

Abstract

The invention provides a training method and device of a search recall model, electronic equipment and a medium, and relates to the technical field of computers, in particular to the technical field of large language models, model training and search recall. According to the implementation scheme, graph structure data is obtained, the graph structure data comprises a plurality of nodes connected based on the incidence relation, and the nodes comprise at least one node indicating a sample search word and at least one node indicating a sample search result; at least one time of text sequence sampling is conducted on the basis of the graph structure data, at least one corresponding text sequence is obtained, and each text sequence in the at least one text sequence is associated with at least one of sample search terms and sample search results indicated by one or more nodes in the multiple nodes; the initial large language model is trained through the at least one text sequence, a target search recall model is obtained, and the target search recall model is used for recalling a corresponding target search result according to the target search word input by the user.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to the fields of large language models, model training and search and recall technology, and specifically to a search and recall model training method and a search and recall method, device, electronic device, computer-readable storage medium and computer program product. Background Art

[0002] A large language model (LLM, also known as a large language model or large model) is a deep learning model trained using a large amount of text data, which can realize the understanding and generation of natural language text.

[0003] At present, a large language model can be used as a search recall model to enhance the knowledge of search terms through the understanding generation ability of the large language model, so as to recall the corresponding search results. Specifically, the search term samples used for model training can be enhanced with knowledge, and the original search term samples, the search term samples after knowledge enhancement, and the search results associated with the search term samples can be used as training data to train the large language model to obtain the above-mentioned search recall model.

[0004] The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention

[0005] The present disclosure provides a search-recall model training method and a search-recall method, apparatus, electronic device, computer-readable storage medium, and computer program product.

[0006] According to one aspect of the present disclosure, a method for training a search recall model is provided, comprising: acquiring graph structure data, wherein the graph structure data comprises a plurality of nodes connected based on an association relationship, the plurality of nodes comprising at least one node indicating a sample search term and at least one node indicating a sample search result; performing at least one text sequence sampling based on the graph structure data to obtain at least one corresponding text sequence, wherein each text sequence in the at least one text sequence is associated with at least one of the sample search terms and the sample search results indicated by one or more nodes in the plurality of nodes; and training an initial large language model using the at least one text sequence to obtain a target search recall model, wherein the target search recall model is used to recall the corresponding target search result according to the target search term input by a user.

[0007] According to another aspect of the present disclosure, a search recall method is provided, comprising: obtaining a target search term input by a target user; processing the target search term using a target search recall model to recall a target search result corresponding to the target search term and display it to the target user, wherein the target search recall model is trained according to the search recall model training method as described above.

[0008] According to another aspect of the present disclosure, a training device for a search recall model is provided, comprising: a first acquisition module, configured to acquire graph structure data, wherein the graph structure data comprises a plurality of nodes connected based on an association relationship, the plurality of nodes comprising at least one node indicating a sample search term and at least one node indicating a sample search result; a sampling module, configured to perform at least one text sequence sampling based on the graph structure data to obtain at least one corresponding text sequence, wherein each text sequence in the at least one text sequence is associated with at least one of the sample search terms and the sample search results indicated by one or more nodes in the plurality of nodes; and a first training module, configured to train an initial large language model using the at least one text sequence to obtain a target search recall model, wherein the target search recall model is used to recall the corresponding target search result according to the target search term input by a user.

[0009] According to another aspect of the present disclosure, a search recall device is provided, including: a fourth acquisition module, configured to acquire a target search term input by a target user; a processing module, configured to process the target search term using a target search recall model to recall the target search results corresponding to the target search term and display them to the target user, wherein the target search recall model is trained according to the training method of the search recall model as described above.

[0010] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above method.

[0011] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above method.

[0012] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the above method when executed by a processor.

[0013] According to one or more embodiments of the present disclosure, a method for training a search recall model is provided, which introduces graph structure data of sample search terms and sample search results, and generates training data for a large language model by performing meta-path sampling on the graph data structure. As a result, during training, the large language model can not only understand the sample search terms themselves and their associated search results from a semantic level, but can also use the associated search results of the sample search terms as an intermediate medium to further learn the association relationship between the current sample search terms and other sample search terms and their associated search results, so that the large language model can obtain richer semantic information during the training process, and understand and learn more comprehensively, thereby effectively improving the recall accuracy of the trained search recall model.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0016] Figure 1 is a schematic diagram illustrating an example system in which the various methods described herein may be implemented according to an exemplary embodiment;

[0017] Figure 2 A flow chart of a method for training a search recall model according to an embodiment of the present disclosure is shown;

[0018] Figure 3 A partial schematic diagram of exemplary graph structure data according to an embodiment of the present disclosure is shown;

[0019] Figure 4 A partial flow chart of another method for training a search-recall model according to an embodiment of the present disclosure is shown;

[0020] Figure 5 A partial flow chart of another method for training a search-recall model according to an embodiment of the present disclosure is shown;

[0021] Figure 6 A partial flow chart of another method for training a search-recall model according to an embodiment of the present disclosure is shown;

[0022] Figure 7 A partial flow chart of another method for training a search-recall model according to an embodiment of the present disclosure is shown;

[0023] Figure 8 A partial flow chart of another method for training a search-recall model according to an embodiment of the present disclosure is shown;

[0024] Fig. 9 A flow chart of a search and recall method according to an embodiment of the present disclosure is shown;

[0025] Fig.10 A structural block diagram of a training device for a search-recall model according to an embodiment of the present disclosure is shown;

[0026] Fig.11 A structural block diagram of a search and recall device according to an embodiment of the present disclosure is shown; and

[0027] Fig.12 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0028] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0029] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0030] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations.

[0031] In the related art, it is proposed that a large language model can be used as a search recall model to enhance the knowledge of search terms through the understanding generation ability of the large language model, so as to recall the corresponding search results. Specifically, the search term samples used for model training can be enhanced with knowledge, and the original search term samples, the search term samples after knowledge enhancement, and the search results associated with the search term samples can be used as training data to train the large language model to obtain the above-mentioned search recall model.

[0032] However, the search recall model trained in this way can only understand and recall based on the search terms themselves, resulting in low accuracy of the recall results.

[0033] To solve the above problems, the present disclosure provides a training method for a search recall model, which introduces graph structure data of sample search terms and sample search results, and generates training data for a large language model by performing meta-path sampling on the graph data structure. As a result, during training, the large language model can not only understand the sample search terms themselves and their associated search results from a semantic level, but also use the associated search results of the sample search terms as an intermediate medium to further learn the association relationship between the current sample search terms and other sample search terms and their associated search results, so that the large language model can obtain richer semantic information during the training process, and understand and learn more comprehensively, thereby effectively improving the recall accuracy of the trained search recall model.

[0034] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0035] Figure 1 FIG. 1 is a schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.

[0036] In an embodiment of the present disclosure, the server 120 may run one or more services or software applications that enable execution of the training method of the search-recall model and the search-recall method.

[0037] In some embodiments, server 120 may also provide other services or software applications that may include non-virtualized environments and virtualized environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to users of client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.

[0038] exist Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 may in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is an example of a system for implementing the search-recall model training method and search-recall method described herein and is not intended to be limiting.

[0039] The user may use client devices 101, 102, 103, 104, 105 and / or 106 to perform the training method of the search recall model and the search recall method. The client device may provide an interface that enables the user of the client device to interact with the client device. The client device may also output information to the user via the interface. Figure 1 Only six client devices are depicted, but one skilled in the art will appreciate that the present disclosure may support any number of client devices.

[0040] Client devices 101, 102, 103, 104, 105 and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, game systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as GOOGLE Chrome OS); or include various mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular phones, smart phones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Game systems may include various handheld game devices, Internet-enabled game devices, etc. Client devices are capable of executing various different applications, such as various Internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and may use various communication protocols.

[0041] The network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0042] Server 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0043] The computing units in the server 120 may run one or more operating systems including any of the above operating systems and any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0044] In some implementations, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106.

[0045] In some embodiments, the server 120 may be a server of a distributed system, or a server combined with a blockchain. The server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in a cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0046] The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store training data and other information for large language models. For example, one or more of the databases 130 may be used to store graph structure data such as for model training. The databases 130 may reside in various locations. For example, the database used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The databases 130 may be of different types. In some embodiments, the database used by the server 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve data to and from the database in response to commands.

[0047] In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the applications may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0048] Figure 1The system 100 may be configured and operated in various ways to enable the application of various methods and apparatuses described in the present disclosure.

[0049] Figure 2 A flowchart of a method for training a search recall model according to an embodiment of the present disclosure is shown.

[0050] like Figure 2 As shown, the training method 200 of the search-recall model includes:

[0051] Step 210: Acquire graph structure data, wherein the graph structure data includes a plurality of nodes connected based on an association relationship, and the plurality of nodes include at least one node indicating a sample search term and at least one node indicating a sample search result;

[0052] Step 220: sampling a text sequence at least once based on the graph structure data to obtain at least one corresponding text sequence, wherein each text sequence in the at least one text sequence is associated with at least one of a sample search term and a sample search result indicated by one or more nodes in the plurality of nodes; and

[0053] Step 230: Use at least one text sequence to train an initial large language model to obtain a target search recall model, wherein the target search recall model is used to recall corresponding target search results according to the target search term input by the user.

[0054] The above method introduces graph structure data of sample search terms and sample search results, and generates training data for the large language model by performing meta-path sampling on the graph data structure. As a result, the large language model can not only understand the sample search terms themselves and their associated search results from a semantic level during training, but also use the associated search results of the sample search terms as an intermediate medium to further learn the association relationship between the current sample search terms and other sample search terms and their associated search results, so that the large language model can obtain richer semantic information during the training process, and understand and learn more comprehensively, thereby effectively improving the recall accuracy of the trained search recall model.

[0055] In particular, for some relatively simple search terms, the content that can be enhanced by them is limited.

[0056] Through the above method, it is possible to associate it with more search terms and search results, and enrich and expand the relevant semantic information based on the text content of the associated nodes, thereby effectively improving the recall ability and recall relevance of the trained search recall model for this type of search terms.

[0057] In step 210, for example, graph structure data may be constructed based on the user's historical click information. Specifically, for any search term, in response to a sample user searching based on the search term and clicking on any search result displayed in the recall, two nodes of the graph structure data are constructed based on the search term and the search result, and a connection is established between the two nodes. Similarly, new search terms and search results are continuously added through the sample user's historical click information, and finally a relatively complete structure data is constructed.

[0058] In step 210, illustratively, graph structure data for search-recall model training may also be directly obtained from a database, and the obtained graph structure data may be adaptively adjusted so that it can be used for training a large language model.

[0059] In step 210, illustratively, for graph structure data that only has nodes indicating sample search results, in order to make the training data valid, the graph structure data may be reconstructed so that the graph structure data includes at least one node indicating a sample search term.

[0060] Figure 3 A partial schematic diagram of exemplary graph structure data according to an embodiment of the present disclosure is shown.

[0061] In step 210, if Figure 3 As shown, the graph structure data of this part takes the q node indicating the sample search term as the central node, and the q node is connected to multiple nodes (b1 node, b2 node, t node, Q node and T node) to indicate that there is an association relationship between the q node and other nodes.

[0062] It should be noted that Figure 3 Only a partial structure of the graph structure data in step 210 is shown. Each of the multiple nodes connected to the q node can also be connected to other nodes indicating sample search terms or sample search results to reflect a more complete network structure of the associated search recall data nodes.

[0063] Exemplarily, the search term indicated by the q node may be "Y Province Travel", the T node may indicate a search result webpage titled "Y Province A City Travel Guide", and the T node may be connected to the p node indicating the search term "A City Food", and the p node may be further connected to the x node indicating the search result webpage titled "Simple and Easy to Make Food" based on the search term "Food", and so on, so that a relatively complete association network for the search term "Y Province Travel" can be established based on the text content of multiple nodes.

[0064] Therefore, compared with directly enhancing the knowledge of "Y Province Tourism" itself, first of all, the scope of search terms and search results that can be associated is expanded (there is no identical text content between "Y Province Tourism" and "A City Food", and it is difficult to perform search recall between the two through vector calculation based on knowledge enhancement for "Y Province Tourism" alone). In addition, since a large language model is used to understand and learn the text content of each associated node, more semantic information is obtained, and the complex internal relationship between each node can be deeply understood, making the recall content of the trained target search recall model more accurate (for example, the search results of "A City Food" are recalled first and the search results of "simple and easy-to-make food" are excluded).

[0065] According to some embodiments, the sample search results include at least one of a recommended search term, a title keyword of a search result page, and a summary keyword of a search result page.

[0066] For example, Figure 3 The Q node in the query can indicate a recommended search term, the T node can indicate a title keyword of a search result page, and the t node can indicate a summary keyword of a search result page. Based on this, an association relationship between sample search terms and different types of search result content can be established for information acquisition and understanding learning by the large language model.

[0067] According to some embodiments, the sample search results further include at least one of a title keyword of the advertisement page and a summary keyword of the advertisement page.

[0068] For example, Figure 3 The b1 node in the example may indicate the title keyword of the advertisement page, and the b2 node may indicate the summary keyword of the advertisement page. Based on this, an association relationship between the sample search terms and the corresponding advertisement content can be established, so that the trained search recall model can recall the corresponding advertisement content based on the search terms input by the user for recommendation.

[0069] In one example, the search recall model trained by method 200 can be applied to a web page search scenario. For example, if the search term entered by the user is "Y Province Tourism", the search recall model can specifically recall the search results by understanding "Y Province Tourism" and include recommended search terms associated with "Y Province Tourism" (for example, "Top Ten Tourist Attractions in Y Province", "Y Province Tourism Off-season" and "Y Province Food", etc.) and natural search results associated with "Y Province Tourism" (for example, a web page titled "Y Province Tourism Guide Sharing", or a web page whose summary content includes the keyword "Y Province Tourism") for the user to select.

[0070] In another example, based on the advertisements placed by merchants, the search results may also include advertisement search results associated with "Travel to Province Y" (for example, an advertisement webpage of a travel agency whose title or summary content includes "Travel to Province Y") to recommend advertisements to users.

[0071] In another example, the search recall model trained by method 200 can also be applied to e-commerce platform search scenarios. For example, if the search term entered by the user is "mobile phone", the search recall model recalls the corresponding search results by understanding "mobile phone", and the search results can specifically include recommended search terms associated with "mobile phone" (for example, "mobile phone holder", "mobile phone case" and "brand A mobile phone", etc.) and natural search results associated with "mobile phone" (for example, a product link with the title "brand B mobile phone" or a product link with the summary content including the keyword "mobile phone") for the user to select.

[0072] In another example, based on the advertisements placed by merchants, the search results may also include advertisement search results associated with "mobile phone" (for example, an advertisement link for a C-brand mobile phone whose title or summary content includes "mobile phone") to recommend advertisements to users.

[0073] Figure 4 A partial flow chart of another method for training a search recall model according to an embodiment of the present disclosure is shown.

[0074] According to some embodiments, Figure 4 As shown, step 220 includes:

[0075] Step 410: taking at least one node indicating a sample search term as a search term node, and obtaining at least one corresponding search term node; and

[0076] Step 420: For each search term node in at least one search term node, determine a sampling path for the search term node, and perform text sequence sampling based on the sampling path to obtain a text sequence corresponding to the search term node, thereby obtaining at least one text sequence.

[0077] To achieve search recall, the model needs to associate the corresponding search content based on the search terms. Therefore, by traversing the sample search terms in the graph structure data to sample the text sequence, the large language model can completely learn and understand the associated content of all search terms in the graph structure data, ensuring that the training data is complete enough, thereby improving the overall performance of the search recall model finally trained.

[0078] Figure 5 A partial flow chart of another method for training a search recall model according to an embodiment of the present disclosure is shown.

[0079] According to some embodiments, Figure 5 As shown, step 420 includes:

[0080] Step 510: for each search word node in at least one search word node, use the search word node as a current sampling node, and determine whether the current sampling node meets a target sampling condition;

[0081] Step 520: In response to the current sampling node not meeting the target sampling condition, perform the following operations:

[0082] Step 521: perform text sampling on the current sampling node, and add the sampled text content to the text sequence corresponding to the search term node;

[0083] Step 522: Obtain the user's historical click volume corresponding to each connection node of the current sampling node; and

[0084] Step 523: determine the next sampling node from each connection node according to the user's historical click volume corresponding to each connection node to update the current sampling node, and return to the step of determining whether the current sampling node meets the target sampling condition; and

[0085] Step 530: In response to the current sampling node satisfying the target sampling condition, output the text sequence corresponding to the search term node;

[0086] The target sampling conditions include at least one of the following:

[0087] The number of connected nodes of the current sampling node is equal to 1, the length of the text sequence corresponding to the search term node reaches a first threshold, and the number of nodes currently sampled reaches a second threshold.

[0088] By determining the degree of association between the sample search term and each of the multiple sample search results based on the user's historical click volume, and thereby determining the probability of moving to each node during sampling, it is possible to effectively balance the randomness of the sampling path during wandering sampling and the degree of association between the nodes of the sampling path, thereby providing higher confidence training data for large language model learning and improving the overall performance of the trained search recall model.

[0089] In step 522, the above text sequence sampling is implemented based on the meta-path sampling method. Specifically, the currently targeted search term node is used as the starting point of the sampling path, and the probability of moving to each connection node is calculated based on the user's historical click volume corresponding to each connection node of the current sampling node to determine the specific sampling path.

[0090] See also Figure 3, the numbers on the line (i.e., path) between the q node and the connected nodes in the figure represent the number of times the sample users click on the corresponding search results of each connected node when searching based on the sample search term of the q node, that is, the path click weight between the two nodes.

[0091] During sampling, the calculation formula for the movement probability of selecting any path for movement is as follows: Movement probability = path click weight of the path / sum of click weights of movable paths.

[0092] Depend on Figure 3 It can be seen that the sum of the path click weights from the current sampling node q to all connected nodes is 10+50+15+20+30=125. Figure 3 The specific moving probabilities of each path in are as follows:

[0093] Sampling Path Move Probability q-b1 10 / 125＝8％ q-b2 50 / 125＝40％ qt 15 / 125＝12％ Q 30 / 125＝24％ qQ 20 / 125＝16％

[0094] Therefore, according to the above movement probability, a random walk can be performed on the q node to determine a high-confidence sampling element path of the q node.

[0095] Exemplarily, multiple random walks may be performed for each search term node in the graph structure data to obtain multiple sampling paths for the search term to perform text sequence sampling, thereby better enriching the sampled semantic information.

[0096] It should be noted that due to Figure 3 Only part of the graph structure data centered on node q is shown, so the above sampling path only involves Figure 3 The nodes and their connection relationships shown in FIG. Figure 3 In the unshown part, nodes b1, b2, Q, T and t may also have a connection relationship. Exemplarily, the meta-paths (sampling paths) may be, for example, "qQT", "qQTq-b1" and "qbqQ-b2".

[0097] In addition, nodes b1, b2, Q, T, and t may further have connection relationships with other sample search terms or sample search result nodes, so the sampling path may continue to extend outward until the current sampling node meets the target sampling condition.

[0098] In one example, the target sampling condition may be that the number of connected nodes of the current sampling node is equal to 1. At this time, the current sampling node has no other movable subsequent nodes, so the current text sequence sampling is completed.

[0099] In another example, the target sampling condition may be that the length of the text sequence corresponding to the currently targeted search term node reaches a first threshold, thereby controlling the text length of the training data, reducing the learning difficulty of the large model, and improving efficiency.

[0100] In another example, the target sampling condition may be that the number of currently sampled nodes reaches a second threshold, so that the degree of association of the currently targeted search term's associated content can be controlled within a certain range, thereby avoiding excessive association of the trained search recall model.

[0101] In yet another example, the target sampling condition may be two or three of the above conditions, so as to better control the text length of the training data to reduce the occupied storage space.

[0102] Figure 6 A partial flow chart of another method for training a search recall model according to an embodiment of the present disclosure is shown.

[0103] According to some embodiments, Figure 6 As shown, step 230 includes:

[0104] Step 610: for each text sequence in at least one text sequence, mask a portion of the text included in the text sequence to update the text sequence; and

[0105] Step 620: Train an initial large language model based on the updated at least one text sequence.

[0106] Each text sequence in the at least one text sequence mentioned above includes one or more text contents sampled from one or more nodes. Therefore, by masking one or more of the text contents, the large model can effectively learn the node text on the meta-path and understand and memorize the graph structure data, thereby training the large model's ability to estimate the masked text part, so that it can recall relevant search results based on the search terms entered by the user.

[0107] According to some embodiments, each text sequence includes a search term text and a search result text, and step 610 includes at least one of the following: for each text sequence, masking all or part of the search term text in the text sequence; and for each text sequence, masking all or part of the search result text in the text sequence.

[0108] In step 610, a prompt word of a corresponding node type may be set for each text in each text sequence to construct training data for the large language model to learn. Figure 3 , Figure 3 The Q node indicates the recommended search term, the T node indicates the title keyword of the search result page, the t node may be the summary keyword of the search result page, the b1 node indicates the title keyword of the advertisement page, and the b2 node indicates the summary keyword of the advertisement page.

[0109] In one example, for a sample text sequence whose original path is "qQT", it can be constructed as a text sequence in the form of "search term: q; recommended search term: Q; title keyword of search result page: [mask]".

[0110] The above sampled text sequence includes three texts sampled from node q, node Q and node T. Before the colon of each text is the prompt word of the corresponding node type set, and after the colon is the text content sampled from the node. [mask] indicates that this part of the text has been masked.

[0111] Similarly, in another example, for a sampled text sequence with an original path of "qQT-b1-b2", it can be constructed as a text sequence in the form of "search term: q; recommended search term: Q; title keyword of search result page: [mask]; title keyword of advertisement page: [mask]; summary keyword of advertisement page: b2".

[0112] Therefore, based on the cooperation of multiple sampled text sequences, the large language model can be trained to accurately predict the text content of the [mask] position based on partial text content and its type, so as to achieve search recall.

[0113] Figure 7 A partial flow chart of another method for training a search recall model according to an embodiment of the present disclosure is shown.

[0114] According to some embodiments, Figure 7 As shown, in addition to steps 210 to 230, method 200 further includes:

[0115] Step 710: Acquire a fine-tuning training data set, wherein the fine-tuning training data set includes at least one training data pair, and each training data pair in the at least one training data pair includes a training search term and a training search content associated with the training search term; and

[0116] Step 720: Use the fine-tuning training data set to train the target search recall model to adjust the parameters of the target search recall model.

[0117] Therefore, by using the precisely matched training search terms and training search contents as data pairs to further fine-tune the parameters of the obtained target search recall model, the recall accuracy of the target search recall model can be better improved.

[0118] In one example, for a certain sample search term, the multiple sample search results associated with it can be prioritized, and one or more sample search results with higher priority can be used to construct training data pairs with the sample search term to obtain the corresponding one or more training data pairs to train the target search recall model and adjust the parameters, so that the target search recall model can prioritize recalling specific sample search results.

[0119] Figure 8 A partial flow chart of another method for training a search recall model according to an embodiment of the present disclosure is shown.

[0120] According to some embodiments, Figure 8 As shown, in addition to steps 210 to 230, method 200 further includes:

[0121] Step 810: Obtain a test search term;

[0122] Step 820: Use the target search recall model to process the test search term to obtain the test search result;

[0123] Step 830: Process the test search term and the test search result using the confidence evaluation model to determine the degree of association between the test search term and the test search result; and

[0124] Step 840: Adjust the parameters of the target search recall model based on the degree of association.

[0125] The trained target search recall model can be tested, and the model parameters can be adjusted based on the test results to further improve the recall accuracy of the target search recall model.

[0126] In step 830, illustratively, the confidence evaluation model may be combined with the target search recall model, and the combined model may be trained for recall discrimination to improve efficiency.

[0127] At step 830 , illustratively, a relevance score may be performed on the test search term and the test search result to determine the degree of association between the two.

[0128] Fig. 9 A flow chart of a search and recall method according to an embodiment of the present disclosure is shown.

[0129] like Fig. 9 As shown, the search and recall method 900 includes:

[0130] Step 910: Obtain the target search term input by the target user; and

[0131] Step 920: Use the target search recall model to process the target search term to recall the target search results corresponding to the target search term and display them to the target user, wherein the target search recall model is trained according to the search recall model training method described above.

[0132] The training process of the target search recall model used in the above method introduces graph structure data of sample search terms and sample search results, and generates training data for the large language model by performing meta-path sampling on the graph data structure. As a result, the large language model can not only understand the sample search terms themselves and their associated search results from a semantic level during training, but also use the associated search results of the sample search terms as an intermediate medium to further learn the association relationship between the current sample search terms and other sample search terms and their associated search results. This allows the large language model to obtain richer semantic information during the training process, and to understand and learn more comprehensively, thereby effectively improving the recall accuracy of the trained search recall model.

[0133] In particular, for some relatively simple search terms, the content that can be enhanced by them is limited.

[0134] Through the above method, it can be associated with more search terms and search results, thereby enriching and expanding its semantic information, and then effectively improving the recall ability and recall relevance of the trained search recall model for this type of search terms.

[0135] Fig.10 A structural block diagram of a training device for a search-recall model according to an embodiment of the present disclosure is shown.

[0136] According to another aspect of the present disclosure, a training device for a search-recall model is provided. Fig.10 As shown, the training device 1000 for the search recall model includes: a first acquisition module 1010, configured to acquire graph structure data, wherein the graph structure data includes multiple nodes connected based on an association relationship, and the multiple nodes include at least one node indicating a sample search term and at least one node indicating a sample search result; a sampling module 1020, configured to perform at least one text sequence sampling based on the graph structure data to obtain at least one corresponding text sequence, wherein each text sequence in the at least one text sequence is associated with at least one of the sample search terms and the sample search results indicated by one or more nodes in the multiple nodes; and a first training module 1030, configured to train an initial large language model using at least one text sequence to obtain a target search recall model, wherein the target search recall model is used to recall the corresponding target search result according to the target search term input by the user.

[0137] Fig.11 A structural block diagram of a search and recall device according to an embodiment of the present disclosure is shown.

[0138] According to another aspect of the present disclosure, a search and recall device is provided. Fig.11 As shown, the search recall device 1100 includes: a fourth acquisition module 1110, configured to acquire a target search term input by a target user; and a processing module 1120, configured to process the target search term using a target search recall model to recall the target search results corresponding to the target search term and display them to the target user, wherein the target search recall model is trained according to the training method of the search recall model as described above.

[0139] According to another aspect of the present disclosure, an electronic device is also provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the aforementioned method.

[0140] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is further provided, wherein the computer instructions are used to enable the computer to execute the aforementioned method.

[0141] According to another aspect of the present disclosure, a computer program product is also provided, including a computer program, wherein the computer program implements the aforementioned method when executed by a processor.

[0142] like Fig.12 As shown, the electronic device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the electronic device 1200 can also be stored. The computing unit 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0143] Multiple components in the electronic device 1200 are connected to the I / O interface 1205, including: an input unit 1206, an output unit 1207, a storage unit 1208, and a communication unit 1209. The input unit 1206 can be any type of device that can input information to the electronic device 1200. The input unit 1206 can receive input digital or character information and generate key signal input related to user settings and / or function control of the electronic device, and can include but is not limited to a mouse, a keyboard, a touch screen, a trackpad, a trackball, a joystick, a microphone and / or a remote control. The output unit 1207 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator and / or a printer. The storage unit 1208 can include but is not limited to a disk and an optical disk. The communication unit 1209 allows the electronic device 1200 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver and / or a chipset, such as Bluetooth TM devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0144] The computing unit 1201 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1201 performs the various methods and processes described above, such as a matrix calculation method based on a GPU. For example, in some embodiments, a matrix calculation method based on a GPU may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1200 via the ROM 1202 and / or the communication unit 1209. When the computer program is loaded into the RAM 1203 and executed by the computing unit 1201, one or more steps of the matrix calculation method based on the GPU described above may be performed. Alternatively, in other embodiments, the computing unit 1201 may be configured to execute a GPU-based matrix calculation method in any other appropriate manner (eg, by means of firmware).

[0145] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0146] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0147] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0148] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0149] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0150] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0151] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0152] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but only by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by their equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that with the evolution of technology, many elements described herein can be replaced by equivalent elements that appear after the present disclosure.

Claims

1. A training method for a search-recall model, comprising: Acquire graph structure data, wherein the graph structure data includes a plurality of nodes connected based on an association relationship, and the plurality of nodes include at least one node indicating a sample search term and at least one node indicating a sample search result; Performing at least one text sequence sampling based on the graph structure data to obtain at least one corresponding text sequence, wherein each text sequence in the at least one text sequence is associated with at least one of a sample search term and a sample search result indicated by one or more nodes in the plurality of nodes; and The at least one text sequence is used to train an initial large language model to obtain a target search recall model, wherein the target search recall model is used to recall corresponding target search results according to a target search term input by a user.

2. The method according to claim 1, wherein: The performing at least one text sequence sampling based on the graph structure data to obtain at least one corresponding text sequence includes: Taking at least one node indicating the sample search term as a search term node, obtaining at least one corresponding search term node; and For each search word node in the at least one search word node, a sampling path for the search word node is determined, and the text sequence is sampled based on the sampling path to obtain a text sequence corresponding to the search word node, thereby obtaining the at least one text sequence.

3. The method according to claim 2, wherein: The step of determining, for each search word node in the at least one search word node, a sampling path for the search word node, and performing text sequence sampling based on the sampling path to obtain a text sequence corresponding to the search word node includes: For each search word node in the at least one search word node, use the search word node as a current sampling node, and determine whether the current sampling node meets a target sampling condition; In response to the current sampling node not meeting the target sampling condition, performing the following operations: Performing text sampling on the current sampling node, and adding the sampled text content to the text sequence corresponding to the search term node; Obtaining the user's historical click volume corresponding to each connection node of the current sampling node; and Determine the next sampling node from each connection node according to the user's historical click volume corresponding to each connection node to update the current sampling node, and return to the step of determining whether the current sampling node meets the target sampling condition; and In response to the current sampling node satisfying the target sampling condition, outputting a text sequence corresponding to the search term node; Wherein, the target sampling condition includes at least one of the following: The number of connected nodes of the current sampling node is equal to 1, the length of the text sequence corresponding to the search term node reaches a first threshold, and the number of currently sampled nodes reaches a second threshold.

4. The method according to any one of claims 1 to 3, wherein: The step of training an initial large language model using the at least one text sequence comprises: For each text sequence in the at least one text sequence, masking a portion of the text included in the text sequence to update the text sequence; and The initial large language model is trained based on the at least one updated text sequence.

5. The method according to claim 4, wherein: Each text sequence includes a search term text and a search result text, and for each text sequence in the at least one text sequence, masking a portion of the text content included in the text sequence includes at least one of the following: For each text sequence, performing the masking process on all or part of the search term text in the text sequence; as well as For each text sequence, the masking process is performed on all or part of the search result text in the text sequence.

6. The method according to any one of claims 1 to 5, further comprising: Acquire a fine-tuning training data set, wherein the fine-tuning training data set includes at least one training data pair, each training data pair in the at least one training data pair includes a training search term and training search content associated with the training search term; and The target search recall model is trained using the fine-tuning training data set to adjust parameters of the target search recall model.

7. The method according to any one of claims 1 to 6, further comprising: Get the test search term; Processing the test search terms using the target search recall model to obtain test search results; Processing the test search term and the test search result using a confidence evaluation model to determine the degree of association between the test search term and the test search result; as well as Parameters of the target search recall model are adjusted based on the degree of association.

8. The method according to any one of claims 1 to 7, wherein: The sample search results include at least one of a recommended search term, a title keyword of a search result page, and a summary keyword of a search result page.

9. The method according to any one of claims 8, wherein: The sample search results also include at least one of a title keyword of the advertisement page and a summary keyword of the advertisement page.

10. A search and recall method, comprising: Get the target search term entered by the target user; as well as The target search term is processed using a target search recall model to recall the target search results corresponding to the target search term and display them to the target user, wherein the target search recall model is trained according to the training method of the search recall model described in any one of claims 1-9.

11. A training device for a search-recall model, comprising: A first acquisition module is configured to acquire graph structure data, wherein the graph structure data includes a plurality of nodes connected based on an association relationship, and the plurality of nodes include at least one node indicating a sample search term and at least one node indicating a sample search result; a sampling module configured to perform at least one text sequence sampling based on the graph structure data to obtain at least one corresponding text sequence, wherein each text sequence in the at least one text sequence is associated with at least one of a sample search term and a sample search result indicated by one or more nodes in the plurality of nodes; and The first training module is configured to train an initial large language model using the at least one text sequence to obtain a target search recall model, wherein the target search recall model is used to recall corresponding target search results according to a target search term input by a user.

12. The device according to claim 11, wherein The sampling module comprises: A first determination submodule is configured to use at least one node indicating the sample search term as a search term node to obtain at least one corresponding search term node; and The second determination submodule is configured to determine a sampling path for each search term node in the at least one search term node, and perform text sequence sampling based on the sampling path to obtain a text sequence corresponding to the search term node, thereby obtaining the at least one text sequence.

13. The device according to claim 12, wherein: The second determining submodule is further configured as follows: For each search word node in the at least one search word node, use the search word node as a current sampling node, and determine whether the current sampling node meets a target sampling condition; In response to the current sampling node not meeting the target sampling condition, performing the following operations: Performing text sampling on the current sampling node, and adding the sampled text content to the text sequence corresponding to the search term node; Obtain the user's historical click volume corresponding to each connection node of the current sampling node; as well as According to the user historical click volume corresponding to each connection node, determine the next sampling node from each connection node to update the current sampling node, and return to the step of determining whether the current sampling node meets the target sampling condition; as well as In response to the current sampling node satisfying the target sampling condition, outputting a text sequence corresponding to the search term node; Wherein, the target sampling condition includes at least one of the following: The number of connected nodes of the current sampling node is equal to 1, the length of the text sequence corresponding to the search term node reaches a first threshold, and the number of currently sampled nodes reaches a second threshold.

14. The device according to any one of claims 11 to 13, wherein: The first training module includes: a processing submodule configured to, for each text sequence in the at least one text sequence, perform a masking process on a portion of text included in the text sequence to update the text sequence; and The training submodule is configured to train the initial large language model based on the at least one updated text sequence.

15. The device according to claim 14, wherein: Each text sequence includes a search term text and a search result text, and for each text sequence in the at least one text sequence, masking a portion of the text content included in the text sequence includes at least one of the following: For each text sequence, performing the masking process on all or part of the search term text in the text sequence; as well as For each text sequence, the masking process is performed on all or part of the search result text in the text sequence.

16. The apparatus according to any one of claims 11 to 15, further comprising: A second acquisition module is configured to acquire a fine-tuning training data set, wherein the fine-tuning training data set includes at least one training data pair, each training data pair in the at least one training data pair includes a training search term and a training search content associated with the training search term; and The second training module is configured to train the target search recall model using the fine-tuning training data set to adjust the parameters of the target search recall model.

17. The apparatus according to any one of claims 11 to 16, further comprising: A third acquisition module is configured to acquire a test search term; A first testing module is configured to process the test search term using the target search recall model to obtain a test search result; A second testing module is configured to process the test search term and the test search result using a confidence evaluation model to determine a degree of association between the test search term and the test search result; as well as An adjustment module is configured to adjust the parameters of the target search recall model based on the degree of association.

18. The device according to any one of claims 11 to 17, wherein: The sample search results include at least one of a recommended search term, a title keyword of a search result page, and a summary keyword of a search result page.

19. The device according to any one of claims 18, wherein: The sample search results also include at least one of a title keyword of the advertisement page and a summary keyword of the advertisement page.

20. A search and recall device, comprising: A fourth acquisition module is configured to acquire a target search term input by a target user; as well as A processing module is configured to use a target search recall model to process the target search term to recall the target search results corresponding to the target search term and display them to the target user, wherein the target search recall model is trained according to the training method of the search recall model described in any one of claims 1-9.

21. An electronic device, comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

23. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images