Matching degree evaluation model training method and matching degree evaluation method and device

Abstract

The invention provides a training method of a matching degree evaluation model and a matching degree evaluation method and device, and relates to the technical field of computers, in particular to the application fields of machine learning, large language models, generative search, search engines, information retrieval, advertisement putting, advertisement promotion, intelligent e-commerce and the like. The specific implementation scheme comprises the steps of obtaining search information obtained based on a search word; obtaining key information obtained based on the keyword; by utilizing a matching degree evaluation model, based on the search information and the key information, respectively obtaining matching degree reasoning results of the search terms and the keywords under the plurality of evaluation rules; training a matching degree evaluation model based on the plurality of matching degree reasoning results; wherein the plurality of matching degree reasoning results are in one-to-one correspondence with the plurality of evaluation rules.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to application fields such as machine learning, large language models, generative search, search engines, information retrieval, advertising placement, advertising promotion, and smart e-commerce, and specifically to a training method for a matching evaluation model, a matching evaluation method, and a device. Background Art

[0002] At present, search engines are not limited to providing information retrieval services, but have also expanded to the advertising business. Specifically, after the search engine receives the search term entered by the user, it will recall the target keywords (for example, advertising keywords) that match it from multiple candidate keywords. This measure is intended to ensure that the advertising content can accurately reach the target audience. In this process, a crucial link is to evaluate the matching degree between the search term and each candidate keyword among multiple candidate keywords. Summary of the invention

[0003] The present disclosure provides a training method for a matching evaluation model, a matching evaluation method and a device.

[0004] According to a first aspect of the present disclosure, a training method for a matching evaluation model is provided, comprising:

[0005] Obtain search information based on search terms;

[0006] Get key information based on keywords;

[0007] Using the matching evaluation model, based on the search information and key information, the matching reasoning results of the search terms and keywords under multiple evaluation rules are obtained respectively;

[0008] Based on the multiple matching degree inference results, the matching degree evaluation model is trained; wherein the multiple matching degree inference results correspond one-to-one to the multiple evaluation rules.

[0009] According to a second aspect of the present disclosure, a matching evaluation method is provided, comprising:

[0010] Obtaining target search information based on target search terms;

[0011] Obtain target key information based on target keywords;

[0012] Using the matching evaluation model, based on the target search information and target key information, the matching evaluation results of the target search terms and target keywords under multiple evaluation rules are obtained respectively.

[0013] According to a third aspect of the present disclosure, a training device for a matching evaluation model is provided, comprising:

[0014] A search information acquisition unit, used to acquire search information based on the search term;

[0015] A key information acquisition unit, used to acquire key information based on keywords;

[0016] An inference result acquisition unit, used to obtain the inference results of the matching degree of the search words and keywords under multiple evaluation rules respectively based on the search information and the key information by using the matching degree evaluation model;

[0017] The first model training unit is used to train the matching evaluation model based on multiple matching reasoning results; wherein the multiple matching reasoning results correspond one-to-one to the multiple evaluation rules.

[0018] According to a fourth aspect of the present disclosure, a matching evaluation device is provided, comprising:

[0019] A target search information acquisition unit, used to acquire target search information based on a target search term;

[0020] A target key information acquisition unit, used to acquire target key information based on target keywords;

[0021] The evaluation result acquisition unit is used to use the matching evaluation model to obtain the matching evaluation results of the target search term and the target keyword under multiple evaluation rules based on the target search information and the target key information.

[0022] According to a fifth aspect of the present disclosure, there is provided an electronic device, including:

[0023] at least one processor;

[0024] a memory communicatively coupled to the at least one processor;

[0025] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method provided in the first aspect of the present disclosure.

[0026] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method provided according to the first aspect of the present disclosure.

[0027] According to a seventh aspect of the present disclosure, a computer program product is provided, comprising a computer program, which implements the method provided according to the first aspect of the present disclosure when executed by a processor.

[0028] The use of the present disclosure can improve the matching evaluation capability of the matching evaluation model, thereby improving the accuracy of multiple matching evaluation results corresponding to multiple evaluation rules.

[0029] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0031] Figure 1 A flowchart of a training method for a matching evaluation model provided in an embodiment of the present disclosure;

[0032] Figure 2 A schematic diagram of a training data acquisition process provided in an embodiment of the present disclosure;

[0033] Figure 3 A schematic diagram of a process for obtaining search information and key information provided in an embodiment of the present disclosure;

[0034] Figure 4 A schematic diagram of the structure of a matching evaluation model provided in an embodiment of the present disclosure;

[0035] Figure 5 A schematic diagram of a training process of a matching evaluation model provided in an embodiment of the present disclosure;

[0036] Figure 6 A flowchart of a matching evaluation method provided in an embodiment of the present disclosure;

[0037] Figure 7 A schematic diagram of an application scenario of a training method for a matching evaluation model provided in an embodiment of the present disclosure;

[0038] Figure 8 A schematic diagram of an application scenario of a matching evaluation method provided in an embodiment of the present disclosure;

[0039] Fig. 9 A schematic structural block diagram of a training device for a matching evaluation model provided in an embodiment of the present disclosure;

[0040] Fig.10 A schematic structural block diagram of a matching degree evaluation device provided in an embodiment of the present disclosure;

[0041] Fig.11 A schematic structural block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0043] As described in the background technology, at present, search engines are not only limited to providing information retrieval services, but also expanded to the field of advertising business. Specifically, after the search engine receives the search term (Query) input by the user, it will recall the target keyword that matches it from multiple candidate keywords (Bidword). This measure is intended to ensure that the advertising content can accurately reach the target audience. In this process, a crucial link is to evaluate the matching degree between the search term and each candidate keyword in the multiple candidate keywords. At present, in order to meet the different delivery needs of advertisers, it is usually necessary to evaluate the matching degree between the search term and each candidate keyword in the multiple candidate keywords based on three evaluation rules. Among them, the three evaluation rules include:

[0044] (1) High-precision rules, that is, determining the matching degree between the target search term and the target keyword by evaluating whether the semantics of the target search term and the target keyword are exactly the same;

[0045] (2) phrase rule, that is, determining the matching degree between the target search term and the target keyword by evaluating whether the target search term and the target keyword contain the same phrase;

[0046] (3) Relevance rule, that is, determining the matching degree between the target search term and the target keyword by evaluating whether there is semantic relevance between the target search term and the target keyword.

[0047] In the prior art, it is usually necessary to prepare three evaluation models (for example, Bidirectional Encoder Representations from Transformers (BERT)), including a first evaluation model, a second evaluation model and a third evaluation model, and then train the first evaluation model into a special model for evaluating the matching degree of the target search term and the target keyword based on high-precision rules, train the second evaluation model into a special model for evaluating the matching degree of the target search term and the target keyword based on phrase rules, and train the third evaluation model into a special model for evaluating the matching degree of the target search term and the target keyword based on relevance rules. Thereafter, the trained first evaluation model can be used to obtain the matching degree of the target search term and the target keyword under the high-precision rule; the trained second evaluation model can be used to obtain the matching degree of the target search term and the target keyword under the phrase rule; and the trained third evaluation model can be used to obtain the matching degree of the target search term and the target keyword under the relevance rule.

[0048] However, the inventors have discovered that in the prior art, since the first evaluation model, the second evaluation model and the third evaluation model are all trained separately, the correlation between the three evaluation rules is not fully considered during the training process. Therefore, the matching evaluation ability of the first evaluation model, the second evaluation model and the third evaluation model cannot be ensured, and thus the accuracy of the matching between the target search terms and the target keywords under the three evaluation rules cannot be ensured.

[0049] In view of the above problems, the embodiments of the present disclosure provide a training method for a matching evaluation model, which can be applied to electronic devices. The electronic device can be a server or a terminal device. Here, the terminal device can be a workbench, a mainframe computer, a conventional computer (for example, a desktop computer, a laptop computer, a car computer, etc.), a personal digital processing unit or other similar computing devices. Figure 1 The flowchart diagram shown in the figure illustrates a training method of a matching evaluation model provided by an embodiment of the present disclosure. It should be noted that although the logical order is shown in the flowchart diagram, in some cases, the steps shown or described in the flowchart can also be performed in other orders.

[0050] Step S101, obtaining search information based on the search term.

[0051] Among them, the search term can be a single word, a phrase, a question or a series of related words to represent the user's query intent, that is, the problem that the user wants to understand, obtain or solve; the search information can be the search term itself, or it can be a more specific, more accurate or more extended query expression obtained after enhancing the search term.

[0052] Step S102: Acquire key information based on keywords.

[0053] Among them, keywords can be advertising keywords, specifically single words, phrases or a series of related words, used to represent the theme, content, target audience, product or service features, promotional information or brand information of the advertisement; key information can be the keywords themselves, or it can be a more specific, more accurate or more extended key expression obtained after enhancing the keywords.

[0054] Step S103, using the matching evaluation model, based on the search information and key information, respectively obtain matching reasoning results of the search terms and keywords under multiple evaluation rules.

[0055] Among them, the matching evaluation model can be built based on the target large model; the multiple evaluation rules can include at least one of high-precision rules, phrase rules and correlation rules. Here, the high-precision rule can be used to indicate whether the semantics of the search term and the keyword are exactly the same to determine the matching reasoning result of the search term and the keyword; the phrase rule can be used to indicate whether the search term and the keyword contain the same phrase to determine the matching reasoning result of the search term and the keyword; the correlation rule can be used to indicate whether the semantic correlation between the search term and the keyword is determined by evaluating whether the search term and the keyword are semantically related.

[0056] Step S104: training a matching evaluation model based on multiple matching reasoning results.

[0057] Among them, multiple matching degree inference results correspond one-to-one to multiple evaluation rules.

[0058] Exemplarily, the multiple evaluation rules include high-precision rules, phrase rules, and relevance rules, and the multiple matching degree reasoning results include a first matching degree reasoning result corresponding to the high-precision rule, a second matching degree reasoning result corresponding to the phrase rule, and a third matching degree reasoning result corresponding to the relevance rule. Then, when the matching degree evaluation model is trained based on the multiple matching degree reasoning results, the first loss value between the first matching degree reasoning result and the first credible matching degree, the second loss value between the second matching degree reasoning result and the second credible matching degree, and the third loss value between the third matching degree reasoning result and the third credible matching degree can be calculated, and the parameters of the matching degree evaluation model can be adjusted based on the first loss value, the second loss value, and the third loss value to achieve the purpose of training the matching degree evaluation model. Among them, the first credible matching degree can be the real matching degree of the search term and the keyword under the high-precision rule; the second credible matching degree can be the real matching degree of the search term and the keyword under the phrase rule; the third credible matching degree can be the real matching degree of the search term and the keyword under the relevance rule. Here, the first credible matching degree, the second credible matching degree, and the third credible matching degree can be obtained by manual marking, which will not be repeated here.

[0059] The training method of the matching evaluation model provided by the embodiment of the present disclosure can obtain the search information obtained based on the search word and the key information obtained based on the keyword, and then use the matching evaluation model to obtain the matching reasoning results of the search word and the keyword under multiple evaluation rules based on the search information and the key information, and train the matching evaluation model based on multiple matching reasoning results corresponding to multiple evaluation rules. In this process, the matching evaluation model is trained on the basis of fully considering the correlation between multiple evaluation rules, which means that the matching evaluation model can learn the common features between multiple evaluation rules, thereby realizing the fusion and sharing of knowledge to improve the matching evaluation ability of the matching evaluation model. In this way, in the subsequent application process, the trained matching evaluation model can be used to accurately obtain the matching evaluation results of the target search word and the target keyword under multiple evaluation rules, that is, the accuracy of multiple matching evaluation results corresponding to multiple evaluation rules can be improved.

[0060] In addition, it should be noted that in the embodiment of the present disclosure, step S101, step S102, step S103 and step S104 can be executed in a loop until the matching evaluation model meets the convergence condition to obtain the final trained matching evaluation model. Among them, the convergence condition can be that the first loss value is less than or equal to the first loss threshold, the second loss value is less than or equal to the second loss threshold, and the third loss value is less than or equal to the third loss threshold; the convergence condition can also be that the matching loss obtained by fusing the first loss value, the second loss value and the third loss value is less than or equal to the preset loss threshold. Here, the first loss threshold, the second loss threshold, the third loss threshold and the preset loss value can be set according to application requirements, and the embodiment of the present disclosure does not limit this.

[0061] As mentioned above, in the embodiments of the present disclosure, the search information may be a more specific, more accurate or more extended query expression obtained after enhancing the search term. Based on this, in some optional implementations, step S101, i.e., "obtaining search information based on the search term" may include:

[0062] Get multiple search expansion terms related to the search term;

[0063] Search word enhancement information including the search word and a plurality of search expansion words is generated as search information.

[0064] The multiple search extension terms may include at least one of a first industry, a first title, a first description, a first category, and a first attribute. Here, the first industry may be used to characterize the economic field or business scope to which the search term belongs; the first title may be used to characterize the title of digital content (e.g., documents, pictures, videos, etc.) related to the search term; the first description may be descriptive information for the search term; the first category may be used to characterize the product category to which the search term belongs; and the first attribute may be used to characterize specific features or parameters of the search term.

[0065] In addition, in the disclosed embodiment, multiple search expansion terms can be obtained from multiple data sources. For example, the first industry can be obtained from the Star Map Industry Information Library, the first title can be obtained from the Big Search Quality Library, the first description can be obtained from the Word Description Information Library, and the first category and the first attribute can be obtained from a database created based on the Knowledge Graph (KG). Among them, the Star Map Industry Information Library can be a database created by a big data analysis organization; the Big Search Quality Library can be a high-quality resource library of a large search platform; the Word Description Information Library can be a pre-created text information library for describing various entity words; and the KG can be a semantic network that describes various entity words and the relationships between them.

[0066] After obtaining multiple search expansion words related to the search word from multiple data sources, search word enhancement information including the search word and multiple search expansion words can be generated as search information according to a preset information template. The preset information template can be set according to application requirements, and the embodiment of the present disclosure does not limit this.

[0067] In the above manner, in the embodiment of the present disclosure, multiple search expansion words related to the search word can be obtained, and search word enhancement information including the search word and the multiple search expansion words can be generated as search information. That is to say, in the embodiment of the present disclosure, the search word can be enhanced through multiple search expansion words to obtain search information, thereby improving the expression ability of the search information. In this way, in the process of training the matching evaluation model, this enhancement processing helps to improve the learning efficiency of the matching evaluation model, thereby further improving the matching evaluation ability of the matching evaluation model.

[0068] Similarly, as mentioned above, in the embodiments of the present disclosure, the key information may be a more specific, more accurate or more extended key expression obtained after the key words are enhanced. Based on this, in some optional implementations, step S102, i.e., "obtaining key information based on the key words" may include:

[0069] Get multiple key expansion words related to the keyword;

[0070] Keyword enhancement information including the keyword and a plurality of key expansion words is generated as key information.

[0071] The multiple key extension words may include at least one of a second industry, a second title, a second description, a second category, and a second attribute. Here, the second industry may be used to characterize the economic field or business scope to which the keyword belongs; the second title may be used to characterize the title of digital content (e.g., documents, pictures, videos, etc.) related to the keyword; the second description may be descriptive information for the keyword; the second category may be used to characterize the product category to which the keyword belongs; and the second attribute may be used to characterize specific features or parameters of the keyword.

[0072] In addition, in the embodiment of the present disclosure, multiple key extension words can be obtained from multiple data sources. For example, the second industry can be obtained from the Star Chart industry information library, the second title can be obtained from the Big Search high-quality library, the second description can be obtained from the word description information library, and the second category and the second attribute can be obtained from a database created based on KG.

[0073] After acquiring multiple key expansion words related to the keyword from multiple data sources, keyword enhancement information including the keyword and the multiple key expansion words may be generated as key information according to a preset information template.

[0074] In the above manner, in the embodiment of the present disclosure, multiple key expansion words related to the keyword can be obtained, and keyword enhancement information including the keyword and multiple key expansion words can be generated as key information. That is to say, in the embodiment of the present disclosure, the keyword can be enhanced through multiple key expansion words to obtain key information, thereby improving the expression ability of the key information. In this way, in the process of training the matching evaluation model, this enhancement processing helps to improve the learning efficiency of the matching evaluation model, thereby further improving the matching evaluation ability of the matching evaluation model.

[0075] In some optional implementations, the matching evaluation model may include a feature processing network, at least one expert network (ie, Expert Network), and multiple gate networks (ie, Gate Network) corresponding to multiple evaluation rules. The feature processing network may be built based on the target large model. Based on this, step S103, that is, "using the matching evaluation model, based on the search information and key information, respectively obtain the matching reasoning results of the search terms and keywords under multiple evaluation rules" may include:

[0076] Using the feature processing network, the search information and key information are fused to obtain the feature fusion result;

[0077] Using at least one expert network, performing feature extraction on the feature fusion result to obtain at least one feature extraction result corresponding to the at least one expert network;

[0078] Each of the multiple gating networks is used as a target gating network, and the target gating network is used to adjust the weight of at least one feature extraction result to obtain a matching degree inference result of the search term and the keyword under the target rule; wherein the target rule is an evaluation rule corresponding to the target gating network among the multiple evaluation rules.

[0079] As mentioned above, in the disclosed embodiment, the feature processing network can be built based on the target large model, that is, the feature processing network can include the target large model. Here, the target large model can be a trained large language model (LLM), and the LLM can be a pre-trained neural network model that has general language knowledge, world knowledge, professional knowledge in various fields, etc., and is stored in the model in the form of parameters. For example, the LLM can be an autoregressive generative model of the Transformer architecture.

[0080] In one example, the LLM can be trained in the following manner:

[0081] Acquire multiple target expansion words related to the target word; wherein the target word is a search word sample or a keyword sample;

[0082] generating target word enhancement information including a target word and a plurality of target extension words;

[0083] Masking at least part of the target extended words in the target word enhancement information to obtain training data;

[0084] Using LLM, based on the training data, we can obtain the enhanced information reasoning results related to the target word;

[0085] The LLM is trained using the enhanced information inference results and the target word enhanced information.

[0086] Among them, the search word sample can be a single word, a phrase, a question or a series of related words to characterize the user's query intention, that is, the problem that the user wants to understand, obtain or solve; the keyword sample can be an advertising keyword, specifically a single word, a phrase or a series of related words to characterize the theme, content, target audience, product or service features, promotional information or brand information of the advertisement; multiple target extension words can include at least one of the third industry, the third title, the third description, the third category and the third attribute. Here, the third industry can be used to characterize the economic field or business scope to which the target word belongs; the third title can be used to characterize the title of the digital content (for example, documents, pictures, videos, etc.) related to the target word; the third description can be the description information for the target word; the third category can be used to characterize the product category to which the target word belongs; the third attribute can be used to characterize the specific characteristics or parameters of the target word.

[0087] In addition, in the disclosed embodiment, multiple target expansion words can be obtained from multiple data sources. For example, the third industry can be obtained from the Star Chart industry information library, the third title can be obtained from the Big Search high-quality library, the third description can be obtained from the word description information library, and the third category and the third attribute can be obtained from a database created based on KG.

[0088] After obtaining multiple target expansion words related to the target word from multiple data sources, target word enhancement information including the target word and multiple target expansion words can be generated according to a preset information template, and at least part of the target expansion words in the target word enhancement information are masked to obtain training data. In a specific example, each of the multiple target expansion words has a corresponding mask probability, so at least part of the target expansion words in the target word enhancement information can be masked based on multiple mask probabilities corresponding to the multiple target expansion words to obtain training data.

[0089] After obtaining the training data, the LLM can be used to obtain the enhanced information inference result related to the target word based on the training data. Specifically, the LLM is used to infer the target word enhanced information corresponding to the training data to obtain the enhanced information inference result related to the target word. Thereafter, the LLM can be trained using the enhanced information inference result and the target word enhanced information. For example, the enhanced loss value between the enhanced information inference result and the target word enhanced information can be calculated, and the parameters of the LLM can be adjusted based on the enhanced loss value to achieve the purpose of training the LLM.

[0090] In the above example, multiple target expansion words related to the target word can be obtained, and target word enhancement information including the target word and multiple target expansion words can be generated, and then at least part of the target expansion words in the target word enhancement information can be masked to obtain training data, so as to use LLM to obtain the enhancement information inference result related to the target word based on the training data, and use the enhancement information inference result and the target word enhancement information to train the LLM, so that the final trained LLM (that is, the target large model) has the ability to supplement the search information and key information (for example, when the search information does not include a full amount of multiple search expansion words, the search information is supplemented with search expansion words; for another example, when the key information does not include a full amount of multiple key expansion words, the key information is supplemented with key expansion words). In this way, before using the feature processing network built based on the target large model to fuse the search information and key information to obtain the feature fusion result, the search information and key information can be supplemented first, thereby further improving the expression ability of the search information and key information, so as to further improve the learning efficiency of the matching evaluation model, thereby further improving the matching evaluation ability of the matching evaluation model.

[0091] In addition, it should be noted that in the above example, "obtain multiple target extended words related to the target word; generate target word enhancement information including the target word and multiple target extended words; mask at least part of the target extended words in the target word enhancement information to obtain training data; use LLM to obtain the enhanced information inference result related to the target word based on the training data; use the enhanced information inference result and the target word enhancement information to train the LLM" can be executed in a loop until the LLM meets the convergence condition to obtain the final trained LLM, that is, to obtain the target large model. Among them, the convergence condition may be that the enhancement loss value is less than or equal to the enhancement loss threshold. Here, the enhancement loss threshold can be set according to application requirements, and the embodiments of the present disclosure are not limited to this.

[0092] In another example, after obtaining the target large model and building a feature processing network based on the target large model, "using the feature processing network to fuse the search information and the key information to obtain a feature fusion result" may include:

[0093] Using the target large model, the search key point reasoning results corresponding to the search information and the key point reasoning results corresponding to the key information are obtained;

[0094] Based on the search information, the search point reasoning results, the key information and the key point reasoning results, the feature fusion results are obtained.

[0095] Among them, the search point reasoning result may include the first core point of the search term and the first explanation information for the first core point; the key point reasoning result may include the second core point of the keyword and the second explanation information for the second core point.

[0096] After obtaining the search point reasoning result corresponding to the search information and the key point reasoning result corresponding to the key information, the feature fusion result can be obtained based on the search information, the search point reasoning result, the key information and the key point reasoning result. In a specific example, the target macro model can be used to fuse the search information, the search point reasoning result, the key information and the key point reasoning result to obtain a preliminary fusion result (here, the preliminary fusion result can be regarded as a multi-dimensional feature vector), and the preliminary fusion result is subjected to feature averaging to obtain a feature fusion result (here, the feature fusion result can be regarded as a one-dimensional feature vector).

[0097] In the above example, the target large model can be used to obtain the search point reasoning results corresponding to the search information and the key point reasoning results corresponding to the key information, and the feature fusion results can be obtained based on the search information, the search point reasoning results, the key information and the key point reasoning results. In other words, the feature fusion results are not only obtained based on the search information and the key information, but also refer to the search point reasoning results and the key point reasoning results. In this way, the representation ability of the feature fusion results can be improved to further improve the learning efficiency of the matching evaluation model, thereby further improving the matching evaluation ability of the matching evaluation model.

[0098] After obtaining the feature fusion result, at least one expert network can be used to perform feature extraction on the feature fusion result to obtain at least one feature extraction result that corresponds one-to-one to the at least one expert network, and each of the multiple gating networks can be used as a target gating network to use the target gating network to adjust the weight of at least one feature extraction result to obtain a matching degree inference result of the search term and the keyword under the target rule.

[0099] The specific number of expert networks can be set according to application requirements, and the embodiments of the present disclosure do not limit this.

[0100] Through the above method, in the embodiment of the present disclosure, the search information and key information can be fused by using the feature processing network to obtain the feature fusion result, and the feature fusion result can be extracted by using at least one expert network to obtain at least one feature extraction result corresponding to at least one expert network, and then each of the multiple gating networks is used as the target gating network, so as to use the target gating network to adjust the weight of at least one feature extraction result to obtain the matching reasoning result of the search term and the keyword under the target rule. That is to say, in the embodiment of the present disclosure, the accurate fusion, feature extraction and weight adjustment of the search information and key information can be achieved through the combination of the feature processing network, at least one expert network and multiple gating networks, thereby significantly improving the accuracy of the matching reasoning result of the search term and the keyword under the target rule, thereby improving the learning efficiency and learning effect of the matching evaluation model, so as to further improve the matching evaluation ability of the matching evaluation model.

[0101] In some optional implementations, step S104, that is, “training a matching evaluation model based on multiple matching reasoning results” may include:

[0102] Obtain the credible matching degree of search terms and keywords under multiple evaluation rules respectively;

[0103] Based on the multiple matching degree reasoning results and the multiple credible matching degrees, a matching degree loss is obtained; wherein the multiple credible matching degrees correspond to the multiple evaluation rules one by one;

[0104] Based on the matching loss, the matching evaluation model is trained.

[0105] As mentioned above, the multiple evaluation rules may include high-precision rules, phrase rules, and relevance rules. For the convenience of distinction, the trustworthy matching degree of the search term and the keyword under the high-precision rule may be defined as the first trustworthy matching degree, the trustworthy matching degree of the search term and the keyword under the phrase rule may be defined as the second trustworthy matching degree, and the trustworthy matching degree of the search term and the keyword under the relevance rule may be defined as the third trustworthy matching degree. Among them, the first trustworthy matching degree, the second trustworthy matching degree, and the third trustworthy matching degree may be obtained by manual marking, which will not be described in detail here.

[0106] In addition, in the embodiment of the present disclosure, the multiple matching degree reasoning results may include a first matching degree reasoning result corresponding to the high-precision rule, a second matching degree reasoning result corresponding to the phrase rule, and a third matching degree reasoning result corresponding to the relevance rule.

[0107] Then, based on multiple matching degree reasoning results and multiple credible matching degrees (first credible matching degree, second credible matching degree and third credible matching degree), the matching degree loss can be obtained by fusing the first loss value, the second loss value and the third loss value to obtain the matching degree loss. The first loss value can be the loss value between the first matching degree reasoning result and the first credible matching degree; the second loss value can be the loss value between the second matching degree reasoning result and the second credible matching degree; and the third loss value can be the loss value between the third matching degree reasoning result and the third credible matching degree.

[0108] After obtaining the matching loss value, as a first example, the matching evaluation model can be trained based on the matching loss, which can include: directly adjusting the parameters of the matching evaluation model (including adjusting the parameters of the feature processing network, at least one expert network, and multiple gating networks) based on the matching loss value to achieve the purpose of training the matching evaluation model; as a second example, "training the matching evaluation model based on the matching loss" can also include:

[0109] Obtaining credible search points corresponding to the search information and credible key points corresponding to the key information;

[0110] Based on the search point reasoning results and the credible search points, the first point loss is obtained;

[0111] Based on the key point reasoning results and the credible key points, the second key point loss is obtained;

[0112] The first key point loss and the second key point loss are combined to obtain the core key point loss;

[0113] The matching evaluation model is trained based on matching loss and core point loss.

[0114] Among them, the trusted search key points may include the first trusted core key points of the search term and the first trusted explanation information for the first trusted core key points; the trusted key points may include the second trusted core key points of the keyword and the second trusted explanation information for the second trusted core key points. Here, the trusted search key points and the trusted key points can be obtained using the key point extraction model or obtained through manual labeling, which is not limited in the embodiments of the present disclosure. In a specific example, the key point extraction model can be other LLMs with excellent performance.

[0115] After obtaining a trusted search point including a first trusted core point and a first trusted explanation information, the loss value between the first core point included in the search point reasoning result and the first trusted core point can be calculated as the first core point loss, and the loss value between the first explanation information included in the search point reasoning result and the first trusted explanation information can be calculated as the first explanation loss, and then the first core point loss and the first explanation loss are fused to obtain the first point loss. Similarly, after obtaining a trusted key point including a second trusted core point and a second trusted explanation information, the loss value between the second core point included in the key point reasoning result and the second trusted core point can be calculated as the second core point loss, and the loss value between the second explanation information included in the key point reasoning result and the second trusted explanation information can be calculated as the second explanation loss, and then the second core point loss and the second explanation loss are fused to obtain the second point loss.

[0116] After obtaining the first key point loss and the second key point loss, and fusing the first key point loss and the second key point loss to obtain the core key point loss, the matching evaluation model can be trained based on the matching loss and the core key point loss. For example, the parameters of the feature processing network in the matching evaluation model can be adjusted based on the matching loss and the core key point loss, and the parameters of at least one expert network and multiple gating networks in the matching evaluation model can be adjusted based on the matching loss to achieve the purpose of training the matching evaluation model.

[0117] Through the above method, in the embodiment of the present disclosure, the credible matching degrees of the search terms and keywords under multiple evaluation rules can be obtained respectively, and the matching loss can be obtained based on multiple matching reasoning results and multiple credible matching degrees, thereby ensuring the comprehensiveness of the matching loss. In this way, when the matching evaluation model is trained based on the matching loss, the learning effect of the matching evaluation model can be further improved, so as to further improve the matching evaluation ability of the matching evaluation model.

[0118] In addition, through the above method, in the embodiment of the present disclosure, when the matching evaluation model is trained based on the matching loss, the first key point loss can be obtained based on the search point reasoning result and the credible search point, and the second key point loss can be obtained based on the key point reasoning result and the credible key point, and the first key point loss and the second key point loss are fused to obtain the core point loss, and then the matching evaluation model is trained based on the matching loss and the core point loss. That is to say, in the embodiment of the present disclosure, when the matching evaluation model is trained, not only the matching loss but also the core point loss is considered. This multi-dimensional loss calculation method can also further improve the learning effect of the matching evaluation model, so as to further improve the matching evaluation ability of the matching evaluation model.

[0119] The following will be combined Figure 2 , Figure 3 , Figure 4 and Figure 5 , the complete process of a training method for a matching evaluation model provided in an embodiment of the present disclosure is described.

[0120] (1) Train the LLM.

[0121] In one example, the training process for LLM may include:

[0122] Acquire multiple target expansion words related to the target word; wherein the target word is a search word sample or a keyword sample;

[0123] generating target word enhancement information including a target word and a plurality of target extension words;

[0124] Masking at least part of the target extended words in the target word enhancement information to obtain training data;

[0125] Using LLM, based on the training data, we can obtain the enhanced information reasoning results related to the target word;

[0126] The LLM is trained using the enhanced information inference results and the target word enhanced information.

[0127] Among them, the search word sample can be a single word, a phrase, a question or a series of related words to characterize the user's query intention, that is, the problem that the user wants to understand, obtain or solve; the keyword sample can be an advertising keyword, specifically a single word, a phrase or a series of related words to characterize the theme, content, target audience, product or service features, promotional information or brand information of the advertisement; multiple target extension words can include at least one of the third industry, the third title, the third description, the third category and the third attribute. Here, the third industry can be used to characterize the economic field or business scope to which the target word belongs; the third title can be used to characterize the title of the digital content (for example, documents, pictures, videos, etc.) related to the target word; the third description can be the description information for the target word; the third category can be used to characterize the product category to which the target word belongs; the third attribute can be used to characterize the specific characteristics or parameters of the target word.

[0128] In addition, in the disclosed embodiment, multiple target expansion words can be obtained from multiple data sources. For example, the third industry can be obtained from the Star Chart industry information library, the third title can be obtained from the Big Search high-quality library, the third description can be obtained from the word description information library, and the third category and the third attribute can be obtained from a database created based on KG.

[0129] like Figure 2As shown, after obtaining multiple target expansion words related to the target word (XX1) from multiple data sources (here, the multiple target expansion words include a third industry (XX11), a third title (XX12), a third description (XX13), a third category (XX14) and a third attribute (XX15)), target word enhancement information including the target word and the multiple target expansion words can be generated according to a preset information template, and at least part of the target expansion words in the target word enhancement information are masked to obtain training data. In a specific example, each of the multiple target expansion words has a corresponding mask probability. Therefore, based on multiple mask probabilities corresponding to the multiple target expansion words, at least part of the target expansion words in the target word enhancement information can be masked to obtain training data.

[0130] After getting the training data ( Figure 2 In the example, the training data is obtained by masking the third title in the target word enhancement information), then the LLM can be used to obtain the enhancement information reasoning result related to the target word based on the training data. Specifically, the LLM can be used to reason about the target word enhancement information corresponding to the training data (more specifically, the target extension word that is missing or masked in the training data is reasoned), and the enhancement information reasoning result related to the target word is obtained ( Figure 2 In the above, the enhanced information reasoning result includes the target word (XX1), and multiple target extended word reasoning results related to the target word (here, the multiple target extended word reasoning results include the third industry reasoning result (XX11), the third title reasoning result (XX12'), the third description reasoning result (XX13), the third category reasoning result (XX14) and the third attribute reasoning result (XX15)). After that, the enhanced information reasoning result and the target word enhanced information can be used to train the LLM. For example, the enhanced loss value between the enhanced information reasoning result and the target word enhanced information can be calculated, and based on the enhanced loss value, the parameters of the LLM can be adjusted to achieve the purpose of training the LLM.

[0131] It should be noted that in the above example, "obtain multiple target extended words related to the target word; generate target word enhancement information including the target word and multiple target extended words; mask at least part of the target extended words in the target word enhancement information to obtain training data; use LLM to obtain the enhanced information inference result related to the target word based on the training data; use the enhanced information inference result and the target word enhancement information to train the LLM" can be executed in a loop until the LLM meets the convergence condition to obtain the final trained LLM, that is, to obtain the target large model. The convergence condition may be that the enhancement loss value is less than or equal to the enhancement loss threshold. Here, the enhancement loss threshold can be set according to application requirements, and the embodiments of the present disclosure are not limited to this.

[0132] (2) Obtain search information based on search terms, and obtain key information based on keywords.

[0133] In one example, “obtaining search information based on a search term” may include:

[0134] Get multiple search expansion terms related to the search term;

[0135] Search word enhancement information including the search word and a plurality of search expansion words is generated as search information.

[0136] The multiple search extension terms may include at least one of a first industry, a first title, a first description, a first category, and a first attribute. Here, the first industry may be used to characterize the economic field or business scope to which the search term belongs; the first title may be used to characterize the title of digital content (e.g., documents, pictures, videos, etc.) related to the search term; the first description may be descriptive information for the search term; the first category may be used to characterize the product category to which the search term belongs; and the first attribute may be used to characterize specific features or parameters of the search term.

[0137] In addition, in the disclosed embodiment, multiple search expansion terms can be obtained from multiple data sources. For example, the first industry can be obtained from the Star Chart industry information library, the first title can be obtained from the Big Search quality library, the first description can be obtained from the word description information library, and the first category and the first attribute can be obtained from a database created based on KG.

[0138] like Figure 3 As shown, after obtaining multiple search extension words related to the search word (XX2) from multiple data sources (here, the multiple search extension words may include at least one of the first industry (XX21), the first title (XX22), the first description (XX23), the first category (XX24) and the first attribute (XX25)), search word enhancement information including the search word and the multiple search extension words can be generated as search information according to a preset information template.

[0139] In one example, “obtaining key information based on keywords” may include:

[0140] Get multiple key expansion words related to the keyword;

[0141] Keyword enhancement information including the keyword and a plurality of key expansion words is generated as key information.

[0142] The multiple key extension words may include at least one of a second industry, a second title, a second description, a second category, and a second attribute. Here, the second industry may be used to characterize the economic field or business scope to which the keyword belongs; the second title may be used to characterize the title of digital content (e.g., documents, pictures, videos, etc.) related to the keyword; the second description may be descriptive information for the keyword; the second category may be used to characterize the product category to which the keyword belongs; and the second attribute may be used to characterize specific features or parameters of the keyword.

[0143] In addition, in the embodiment of the present disclosure, multiple key extension words can be obtained from multiple data sources. For example, the second industry can be obtained from the Star Chart industry information library, the second title can be obtained from the Big Search high-quality library, the second description can be obtained from the word description information library, and the second category and the second attribute can be obtained from a database created based on KG.

[0144] Likewise, if Figure 3 As shown, after obtaining multiple key expansion words related to the keyword (XX3) from multiple data sources (here, the multiple key expansion words may include at least one of the second industry (XX31), the second title (XX32), the second description (XX33), the second category (XX34) and the second attribute (XX35)), keyword enhancement information including the keyword and the multiple key expansion words can be generated as key information according to a preset information template.

[0145] (3) Using the matching evaluation model, based on the search information and key information, respectively obtain the matching inference results of the search terms and keywords under multiple evaluation rules, and train the matching evaluation model based on the multiple matching inference results.

[0146] like Figure 4 As shown, in the disclosed embodiment, the matching evaluation model may include a feature processing network, at least one expert network, and multiple gating networks corresponding to multiple evaluation rules. The feature processing network may be built based on the target large model. Based on this, in one example, "using the matching evaluation model, based on the search information and key information, respectively obtaining the matching reasoning results of the search terms and keywords under multiple evaluation rules" may include:

[0147] Using the feature processing network, the search information and key information are fused to obtain the feature fusion result;

[0148] Using at least one expert network, performing feature extraction on the feature fusion result to obtain at least one feature extraction result corresponding to the at least one expert network;

[0149] Each of the multiple gating networks is used as a target gating network, and the target gating network is used to adjust the weight of at least one feature extraction result to obtain a matching degree inference result of the search term and the keyword under the target rule; wherein the target rule is an evaluation rule corresponding to the target gating network among the multiple evaluation rules.

[0150] Further, in a specific example, “using a feature processing network to fuse the search information and the key information to obtain a feature fusion result” may include:

[0151] Using the target large model, the search key point reasoning results corresponding to the search information and the key point reasoning results corresponding to the key information are obtained;

[0152] Based on the search information, the search point reasoning results, the key information and the key point reasoning results, the feature fusion results are obtained.

[0153] Among them, the search point reasoning result may include the first core point of the search term and the first explanation information for the first core point; the key point reasoning result may include the second core point of the keyword and the second explanation information for the second core point.

[0154] After obtaining the search point reasoning result corresponding to the search information and the key point reasoning result corresponding to the key information, the feature fusion result can be obtained based on the search information, the search point reasoning result, the key information and the key point reasoning result. In a more specific example, the target macro model can be used to fuse the search information, the search point reasoning result, the key information and the key point reasoning result to obtain a preliminary fusion result (here, the preliminary fusion result can be regarded as a multi-dimensional feature vector), and the preliminary fusion result is subjected to feature averaging to obtain a feature fusion result (here, the feature fusion result can be regarded as a one-dimensional feature vector).

[0155] After obtaining the feature fusion result, at least one expert network can be used to perform feature extraction on the feature fusion result to obtain at least one feature extraction result corresponding to at least one expert network, and each of the multiple gating networks is used as a target gating network to use the target gating network to perform weight adjustment on at least one feature extraction result to obtain the matching degree reasoning result of the search term and the keyword under the target rule. The specific number of expert networks can be set according to application requirements. For example, the specific number of expert networks can be 6, which is not limited in the embodiment of the present disclosure.

[0156] In one example, “training a matching evaluation model based on multiple matching reasoning results” may include:

[0157] Obtain the credible matching degree of search terms and keywords under multiple evaluation rules respectively;

[0158] Based on the multiple matching degree reasoning results and the multiple credible matching degrees, a matching degree loss is obtained; wherein the multiple credible matching degrees correspond to the multiple evaluation rules one by one;

[0159] Based on the matching loss, the matching evaluation model is trained.

[0160] As mentioned above, the multiple evaluation rules may include high-precision rules, phrase rules, and relevance rules. For the convenience of distinction, the trustworthy matching degree of the search term and the keyword under the high-precision rule may be defined as the first trustworthy matching degree, the trustworthy matching degree of the search term and the keyword under the phrase rule may be defined as the second trustworthy matching degree, and the trustworthy matching degree of the search term and the keyword under the relevance rule may be defined as the third trustworthy matching degree. Among them, the first trustworthy matching degree, the second trustworthy matching degree, and the third trustworthy matching degree may be obtained by manual marking, which will not be described in detail here.

[0161] In addition, if Figure 5 As shown, in the embodiment of the present disclosure, the multiple matching degree reasoning results may include a first matching degree reasoning result corresponding to the high-precision rule, a second matching degree reasoning result corresponding to the phrase rule, and a third matching degree reasoning result corresponding to the relevance rule.

[0162] Then, based on multiple matching degree reasoning results and multiple credible matching degrees (first credible matching degree, second credible matching degree and third credible matching degree), the matching degree loss can be obtained by fusing the first loss value, the second loss value and the third loss value to obtain the matching degree loss (denoted as, Nlu Loss). Among them, the first loss value can be the loss value between the first matching degree reasoning result and the first credible matching degree; the second loss value can be the loss value between the second matching degree reasoning result and the second credible matching degree; the third loss value can be the loss value between the third matching degree reasoning result and the third credible matching degree.

[0163] After obtaining the matching loss value, the credible search points corresponding to the search information and the credible key points corresponding to the key information can be obtained, and the first point loss is obtained based on the search point reasoning result and the credible search point, and the second point loss is obtained based on the key point reasoning result and the credible key point, and then the first point loss and the second point loss are fused to obtain the core point loss (denoted as Nlg Loss), and the matching evaluation model is trained based on the matching loss and the core point loss. Among them, the credible search points can include the first credible core point of the search term and the first credible explanation information for the first credible core point; the credible key points can include the second credible core point of the keyword and the second credible explanation information for the second credible core point. Here, the credible search points and credible key points can be obtained by manual marking, which will not be repeated here.

[0164] In a specific example, after obtaining a trusted search point including a first trusted core point and a first trusted explanation information, the loss value between the first core point included in the search point reasoning result and the first trusted core point can be calculated as the first core point loss, and the loss value between the first explanation information included in the search point reasoning result and the first trusted explanation information can be calculated as the first explanation loss, and then the first core point loss and the first explanation loss are fused to obtain the first point loss. Similarly, after obtaining a trusted key point including a second trusted core point and a second trusted explanation information, the loss value between the second core point included in the key point reasoning result and the second trusted core point can be calculated as the second core point loss, and the loss value between the second explanation information included in the key point reasoning result and the second trusted explanation information can be calculated as the second explanation loss, and then the second core point loss and the second explanation loss are fused to obtain the second point loss.

[0165] After obtaining the first key point loss and the second key point loss, and fusing the first key point loss and the second key point loss to obtain the core key point loss, the matching evaluation model can be trained based on the matching loss and the core key point loss. For example, the parameters of the feature processing network in the matching evaluation model can be adjusted based on the matching loss and the core key point loss, and the parameters of at least one expert network and multiple gating networks in the matching evaluation model can be adjusted based on the matching loss to achieve the purpose of training the matching evaluation model.

[0166] The disclosed embodiment provides a matching evaluation method, which can be applied to electronic devices. The electronic device can be a server or a terminal device. Here, the terminal device can be a workbench, a mainframe computer, a conventional computer (e.g., a desktop computer, a laptop computer, a car computer, etc.), a personal digital processing unit or other similar computing devices. Figure 6 The flowchart diagram shown in the figure illustrates a matching degree evaluation method provided by an embodiment of the present disclosure. It should be noted that although the logical order is shown in the flowchart diagram, in some cases, the steps shown or described in the flowchart can also be performed in other orders.

[0167] Step S601, obtaining target search information based on the target search term.

[0168] Among them, the target search term can be a single word, a phrase, a question or a series of related words to characterize the user's query intent, that is, the problem that the user wants to understand, obtain or solve; the target search information can be the target search term itself, or it can be a more specific, more accurate or more extended query expression obtained after enhancing the target search term.

[0169] Step S602, obtaining target key information based on the target keyword.

[0170] Among them, the target keyword can be an advertising keyword, specifically a single word, a phrase or a series of related words, which are used to represent the theme, content, target audience, product or service features, promotional information or brand information of the advertisement; the target key information can be the target keyword itself, or it can be a more specific, more accurate or more extended key expression obtained after enhancing the target keyword.

[0171] Step S603, using the matching evaluation model, based on the target search information and the target key information, respectively obtain matching evaluation results of the target search term and the target key word under multiple evaluation rules.

[0172] Among them, the matching evaluation model can be built based on the target large model; the multiple evaluation rules can include at least one of high-precision rules, phrase rules and correlation rules. Here, the high-precision rule can be used to indicate whether the semantics of the search term and the keyword are exactly the same to determine the matching reasoning result of the search term and the keyword; the phrase rule can be used to indicate whether the search term and the keyword contain the same phrase to determine the matching reasoning result of the search term and the keyword; the correlation rule can be used to indicate whether the semantic correlation between the search term and the keyword is determined by evaluating whether the search term and the keyword are semantically related.

[0173] In addition, it should be noted that in the embodiments of the present disclosure, the matching evaluation model may be trained, and the specific training method may refer to the aforementioned matching evaluation model training method embodiment.

[0174] The training method of the matching evaluation model provided by the embodiment of the present disclosure can obtain the target search information obtained based on the target search word and the target key information obtained based on the target keyword, and then use the matching evaluation model to obtain the matching evaluation results of the target search word and the target key information under multiple evaluation rules based on the target search information and the target key information. Since the matching evaluation model is trained and the specific training method is the training method of the aforementioned matching evaluation model, the matching evaluation model has superior matching evaluation capabilities. In this way, using the matching evaluation model, the matching evaluation results of the target search word and the target key under multiple evaluation rules can be accurately obtained, that is, the accuracy of multiple matching evaluation results corresponding to multiple evaluation rules can be improved.

[0175] As mentioned above, in the embodiments of the present disclosure, the target search information may be a more specific, more accurate or more extended query expression obtained after enhancing the target search term. Based on this, in some optional implementations, "obtaining the target search information based on the target search term" may include:

[0176] Acquire multiple target search expansion terms related to the target search term;

[0177] Target search word enhancement information including a target search word and a plurality of target search extension words is generated as target search information.

[0178] The multiple target search extension words may include at least one of a first target industry, a first target title, a first target description, a first target category, and a first target attribute. Here, the first target industry may be used to characterize the economic field or business scope to which the target search word belongs; the first target title may be used to characterize the title of digital content (e.g., documents, pictures, videos, etc.) related to the target search word; the first target description may be descriptive information for the target search word; the first target category may be used to characterize the product category to which the target search word belongs; and the first target attribute may be used to characterize the specific characteristics or parameters of the target search word.

[0179] In addition, in the disclosed embodiment, multiple target search extension words can be obtained from multiple data sources. For example, the first target industry can be obtained from the Star Chart industry information library, the first target title can be obtained from the Big Search quality library, the first target description can be obtained from the word description information library, and the first target category and the first target attribute can be obtained from a database created based on KG.

[0180] After acquiring multiple target search expansion words related to the target search word from multiple data sources, target search word enhancement information including the target search word and the multiple target search expansion words may be generated as target search information according to a preset information template.

[0181] Through the above method, in the embodiment of the present disclosure, multiple target search expansion words related to the target search word can be obtained, and target search word enhancement information including the target search word and the multiple target search expansion words can be generated as the target search information. That is to say, in the embodiment of the present disclosure, the target search word can be enhanced through multiple target search expansion words to obtain target search information, thereby improving the expression ability of the target search information. In this way, in the process of using the matching evaluation model to obtain the matching evaluation results of the target search word and the target keyword under multiple evaluation rules based on the target search information and the target key information, this enhancement processing helps to further improve the accuracy of multiple matching evaluation results corresponding to multiple evaluation rules.

[0182] As mentioned above, in the embodiments of the present disclosure, the target key information may be a more specific, more accurate or more extended key expression obtained after enhancing the target keyword. Based on this, in some optional implementations, "obtaining the target key information based on the target keyword" may include:

[0183] Obtain multiple target keyword extensions related to the target keyword;

[0184] Target keyword enhancement information including a target keyword and a plurality of target keyword extensions is generated as target keyword information.

[0185] The multiple target keyword extension words may include at least one of a second target industry, a second target title, a second target description, a second target category, and a second target attribute. Here, the second target industry may be used to characterize the economic field or business scope to which the target keyword belongs; the second target title may be used to characterize the title of digital content (e.g., documents, pictures, videos, etc.) related to the target keyword; the second target description may be descriptive information for the target keyword; the second target category may be used to characterize the product category to which the target keyword belongs; and the second target attribute may be used to characterize the specific characteristics or parameters of the target keyword.

[0186] In addition, in the disclosed embodiment, multiple target key expansion words can be obtained from multiple data sources. For example, the second target industry can be obtained from the Star Chart industry information library, the second target title can be obtained from the Big Search high-quality library, the second target description can be obtained from the word description information library, and the second target category and the second target attribute can be obtained from a database created based on the KG.

[0187] After acquiring multiple target keyword expansion words related to the target keyword from multiple data sources, target keyword enhancement information including the target keyword and multiple target keyword expansion words may be generated as target keyword information according to a preset information template.

[0188] Through the above method, in the embodiment of the present disclosure, multiple target keyword expansion words related to the target keyword can be obtained, and target keyword enhancement information including the target keyword and multiple target keyword expansion words can be generated as the target keyword information. That is to say, in the embodiment of the present disclosure, the target keyword can be enhanced through multiple target keyword expansion words to obtain target keyword information, thereby improving the expression ability of the target keyword information. In this way, in the process of using the matching evaluation model to obtain the matching evaluation results of the target search word and the target keyword under multiple evaluation rules based on the target search information and the target keyword information, this enhancement processing helps to further improve the accuracy of multiple matching evaluation results corresponding to multiple evaluation rules.

[0189] In some optional implementations, the matching evaluation model includes a feature processing network, at least one expert network, and multiple gated networks corresponding to multiple evaluation rules. The feature processing network can be built based on the target large model. Based on this, step S603, that is, "using the matching evaluation model, based on the target search information and the target key information, respectively obtain the matching evaluation results of the target search term and the target keyword under multiple evaluation rules" may include:

[0190] Using the feature processing network, the target search information and the target key information are fused to obtain the target feature fusion result;

[0191] Using at least one expert network, extracting features from the target feature fusion result to obtain at least one target feature extraction result corresponding to the at least one expert network;

[0192] Each of the multiple gating networks is used as a target gating network, and the target gating network is used to adjust the weight of at least one target feature extraction result to obtain a matching evaluation result of the search term and the keyword under the target rule; wherein the target rule is an evaluation rule corresponding to the target gating network among the multiple evaluation rules.

[0193] As mentioned above, in the disclosed embodiment, the feature processing network can be built based on the target large model, that is, the feature processing network can include the target large model. Here, the target large model can be a trained large language model (LLM), and the LLM can be a pre-trained neural network model that has general language knowledge, world knowledge, professional knowledge in various fields, etc., and is stored in the model in the form of parameters. For example, the LLM can be an autoregressive generative model of the Transformer architecture.

[0194] In addition, it should be noted that in the embodiments of the present disclosure, the training method for LLM can refer to the relevant steps in the aforementioned training method embodiment of the matching evaluation model, which will not be described in detail here.

[0195] In one example, after obtaining a target large model and building a feature processing network based on the target large model, “using the feature processing network to fuse the target search information and the target key information to obtain a target feature fusion result” may include:

[0196] Using the target large model, obtain the search key point acquisition results corresponding to the target search information, and the key point acquisition results corresponding to the target key information;

[0197] Based on the target search information, the search key point acquisition results, the target key information and the key point acquisition results, the target feature fusion result is obtained.

[0198] Among them, the search point acquisition results may include the first target core point of the target search term and the first target explanation information for the first target core point; the key point acquisition results may include the second target core point of the target keyword and the second target explanation information for the second target core point.

[0199] After obtaining the search key point acquisition result corresponding to the target search information and the key point acquisition result corresponding to the target key information, the target feature fusion result can be obtained based on the target search information, the search key point acquisition result, the target key information and the key point acquisition result. In a specific example, the target search information, the search key point acquisition result, the target key information and the key point acquisition result can be fused using the target macro model to obtain a target preliminary fusion result (here, the target preliminary fusion result can be regarded as a multi-dimensional feature vector), and the target preliminary fusion result is subjected to feature averaging processing to obtain a target feature fusion result (here, the target feature fusion result can be regarded as a one-dimensional feature vector).

[0200] In the above example, the target large model can be used to obtain the search key point acquisition results corresponding to the target search information and the key point acquisition results corresponding to the target key information, and the target feature fusion results can be obtained based on the target search information, the search key point acquisition results, the target key information and the key point acquisition results. In other words, the target feature fusion results are not only obtained based on the target search information and the target key information, but also refer to the search key point acquisition results and the key point acquisition results. In this way, the characterization ability of the target feature fusion results can be improved to further improve the accuracy of multiple matching evaluation results corresponding to multiple evaluation rules.

[0201] After obtaining the target feature fusion result, at least one expert network can be used to perform feature extraction on the target feature fusion result to obtain at least one target feature extraction result corresponding to the at least one expert network, and each of the multiple gating networks can be used as a target gating network to use the target gating network to perform weight adjustment on at least one target feature extraction result to obtain a matching evaluation result of the target search term and the target keyword under the target rule.

[0202] The specific number of expert networks can be set according to application requirements, and the embodiments of the present disclosure do not limit this.

[0203] Through the above method, in the embodiment of the present disclosure, the target search information and the target key information can be fused by using the feature processing network to obtain the target feature fusion result, and the target feature fusion result can be extracted by using at least one expert network to obtain at least one target feature extraction result corresponding to at least one expert network, and then each of the multiple gating networks is used as the target gating network, so as to use the target gating network to adjust the weight of at least one target feature extraction result to obtain the matching evaluation result of the search term and the keyword under the target rule. That is to say, in the embodiment of the present disclosure, the accurate fusion, feature extraction and weight adjustment of the target search information and the target key information can be achieved through the combination of the feature processing network, at least one expert network and multiple gating networks, thereby significantly improving the accuracy of the matching evaluation result of the target search term and the target keyword under the target rule.

[0204] See also Figure 7 , which is a schematic diagram of an application scenario of a training method for a matching evaluation model provided in an embodiment of the present disclosure.

[0205] The training method of the matching evaluation model provided in the embodiment of the present disclosure is applied to an electronic device. The electronic device may be a server or a terminal device. The terminal device may be a workbench, a mainframe computer, a conventional computer (e.g., a desktop computer, a laptop computer, a vehicle-mounted computer, etc.), a personal digital processing or other similar computing device. The electronic device is used to:

[0206] Obtain search information based on search terms;

[0207] Get key information based on keywords;

[0208] Using the matching evaluation model, based on the search information and key information, the matching reasoning results of the search terms and keywords under multiple evaluation rules are obtained respectively;

[0209] Based on the multiple matching degree inference results, the matching degree evaluation model is trained; wherein the multiple matching degree inference results correspond one-to-one to the multiple evaluation rules.

[0210] It should be noted that in the embodiments of the present disclosure, Figure 7 The scene diagram shown is only illustrative and not restrictive. Those skilled in the art can Figure 7 Various obvious changes and / or substitutions are made to the examples, and the obtained technical solutions still fall within the scope of the disclosure of the embodiments of the present disclosure.

[0211] See also Figure 8 , which is a schematic diagram of an application scenario of a matching evaluation method provided in an embodiment of the present disclosure.

[0212] The matching evaluation method provided by the embodiment of the present disclosure is applied to an electronic device. The electronic device may be a server or a terminal device. The terminal device may be a workbench, a mainframe computer, a conventional computer (e.g., a desktop computer, a laptop computer, a vehicle-mounted computer, etc.), a personal digital processing device or other similar computing device. The electronic device is used to:

[0213] Obtaining target search information based on target search terms;

[0214] Obtain target key information based on target keywords;

[0215] Using the matching evaluation model, based on the target search information and target key information, the matching evaluation results of the target search terms and target keywords under multiple evaluation rules are obtained respectively.

[0216] It should be noted that in the embodiments of the present disclosure, Figure 8 The scene diagram shown is only illustrative and not restrictive. Those skilled in the art can Figure 8Various obvious changes and / or substitutions are made to the examples, and the obtained technical solutions still fall within the scope of the disclosure of the embodiments of the present disclosure.

[0217] In order to better implement the training method of the aforementioned matching evaluation model, the embodiment of the present disclosure also provides a training device for the matching evaluation model, which can be integrated into an electronic device. The electronic device can be a server or a terminal device. Here, the terminal device can be a workbench, a mainframe computer, a conventional computer (for example, a desktop computer, a laptop computer, a car computer, etc.), a personal digital processing or other similar computing device. Fig. 9 The schematic structural block diagram shown illustrates a training device 900 for a matching evaluation model provided in the disclosed embodiment.

[0218] The training device 900 of the matching evaluation model includes:

[0219] A search information acquisition unit 901, used to acquire search information based on a search term;

[0220] A key information acquisition unit 902, used to acquire key information based on keywords;

[0221] The inference result acquisition unit 903 is used to obtain the matching degree inference results of the search words and keywords under multiple evaluation rules respectively based on the search information and the key information by using the matching degree evaluation model;

[0222] The first model training unit 904 is used to train the matching evaluation model based on multiple matching reasoning results; wherein the multiple matching reasoning results correspond to the multiple evaluation rules one by one.

[0223] In some optional implementations, the matching evaluation model includes a feature processing network, at least one expert network, and multiple gating networks corresponding to multiple evaluation rules one by one; the inference result acquisition unit 903 is used to:

[0224] Using the feature processing network, the search information and key information are fused to obtain the feature fusion result;

[0225] Using at least one expert network, performing feature extraction on the feature fusion result to obtain at least one feature extraction result corresponding to the at least one expert network;

[0226] Each of the multiple gating networks is used as a target gating network, and the target gating network is used to adjust the weight of at least one feature extraction result to obtain a matching degree inference result of the search term and the keyword under the target rule; wherein the target rule is an evaluation rule corresponding to the target gating network among the multiple evaluation rules.

[0227] In some optional implementations, the feature processing network includes a target large model; the inference result acquisition unit 903 is used to:

[0228] Using the target large model, the search key point reasoning results corresponding to the search information and the key point reasoning results corresponding to the key information are obtained;

[0229] Based on the search information, the search point reasoning results, the key information and the key point reasoning results, the feature fusion results are obtained.

[0230] In some optional embodiments, the target large model is a trained LLM; the apparatus further comprises a second model training unit, configured to:

[0231] Acquire multiple target expansion words related to the target word; wherein the target word is a search word sample or a keyword sample;

[0232] generating target word enhancement information including a target word and a plurality of target extension words;

[0233] Masking at least part of the target extended words in the target word enhancement information to obtain training data;

[0234] Using LLM, based on the training data, we can obtain the enhanced information reasoning results related to the target word;

[0235] The LLM is trained using the enhanced information inference results and the target word enhanced information.

[0236] In some optional implementations, the first model training unit 904 is used to:

[0237] Obtain the credible matching degree of search terms and keywords under multiple evaluation rules respectively;

[0238] Based on the multiple matching degree reasoning results and the multiple credible matching degrees, a matching degree loss is obtained; wherein the multiple credible matching degrees correspond to the multiple evaluation rules one by one;

[0239] Based on the matching loss, the matching evaluation model is trained.

[0240] In some optional implementations, the first model training unit 904 is used to:

[0241] Obtaining credible search points corresponding to the search information and credible key points corresponding to the key information;

[0242] Based on the search point reasoning results and the credible search points, the first point loss is obtained;

[0243] Based on the key point reasoning results and the credible key points, the second key point loss is obtained;

[0244] The first key point loss and the second key point loss are combined to obtain the core key point loss;

[0245] Based on the matching loss core point loss, the matching evaluation model is trained.

[0246] In some optional implementations, the search information acquisition unit 901 is used to:

[0247] Get multiple search expansion terms related to the search term;

[0248] Search word enhancement information including the search word and a plurality of search expansion words is generated as search information.

[0249] In some optional implementations, the key information acquisition unit 902 is used to:

[0250] Get multiple key expansion words related to the keyword;

[0251] Keyword enhancement information including the keyword and a plurality of key expansion words is generated as key information.

[0252] In the embodiment of the present disclosure, the specific functions and examples of each unit in the training device 900 of the matching evaluation model can be found in the relevant descriptions of the corresponding steps in the aforementioned training method embodiment of the matching evaluation model, and will not be repeated here.

[0253] In order to better implement the above-mentioned matching evaluation method, the embodiment of the present disclosure also provides a matching evaluation device, which can be integrated into an electronic device. The electronic device can be a server or a terminal device. Here, the terminal device can be a workbench, a mainframe computer, a conventional computer (for example, a desktop computer, a laptop computer, a car computer, etc.), a personal digital processing or other similar computing device. Fig.10 The schematic structural block diagram shown is used to illustrate a matching evaluation device 1000 provided in the disclosed embodiment.

[0254] The matching evaluation device 1000 includes:

[0255] The target search information acquisition unit 1001 is used to acquire target search information based on the target search word;

[0256] A target key information acquisition unit 1002 is used to acquire target key information based on target keywords;

[0257] The evaluation result acquisition unit 1003 is used to obtain the matching evaluation results of the target search term and the target keyword under multiple evaluation rules based on the target search information and the target key information by using the matching evaluation model.

[0258] In some optional implementations, the matching evaluation model includes a feature processing network, at least one expert network, and multiple gating networks corresponding to multiple evaluation rules one by one; the evaluation result acquisition unit 1003 is used to:

[0259] Using the feature processing network, the target search information and the target key information are fused to obtain the target feature fusion result;

[0260] Using at least one expert network, extracting features from the target feature fusion result to obtain at least one target feature extraction result corresponding to the at least one expert network;

[0261] Each of the multiple gating networks is used as a target gating network, and the target gating network is used to adjust the weight of at least one target feature extraction result to obtain a matching evaluation result of the search term and the keyword under the target rule; wherein the target rule is an evaluation rule corresponding to the target gating network among the multiple evaluation rules.

[0262] In some optional implementations, the feature processing network includes a target large model; the evaluation result acquisition unit 1003 is used to:

[0263] Using the target large model, obtain the search key point acquisition results corresponding to the target search information, and the key point acquisition results corresponding to the target key information;

[0264] Based on the target search information, the search key point acquisition results, the target key information and the key point acquisition results, the target feature fusion result is obtained.

[0265] In some optional implementations, the target search information acquisition unit 1001 is used to:

[0266] Acquire multiple target search expansion terms related to the target search term;

[0267] Target search word enhancement information including a target search word and a plurality of target search extension words is generated as target search information.

[0268] In some optional implementations, the target key information acquisition unit 1002 is used to:

[0269] Obtain multiple target keyword extensions related to the target keyword;

[0270] Target keyword enhancement information including a target keyword and a plurality of target keyword extensions is generated as target keyword information.

[0271] In the embodiment of the present disclosure, the specific functions and examples of each unit in the matching evaluation device 1000 can refer to the relevant descriptions of the corresponding steps in the above matching evaluation method embodiment, which will not be repeated here.

[0272] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0273] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0274] Fig.11 A schematic block diagram of an example electronic device 1100 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as vehicle-mounted computing devices, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0275] like Fig.11 As shown, the electronic device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 to a random access memory (RAM) 1103. In RAM 1103, various programs and data required for the operation of the electronic device 1100 can also be stored. The computing unit 1101, ROM 1102, and RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0276] A number of components in the electronic device 1100 are connected to the I / O interface 1105, including: an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of renderers, speakers, etc.; a storage unit 1108, such as a disk, an optical disk, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the electronic device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0277] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, for example, the training method of the matching evaluation model and / or the matching evaluation method. For example, in some embodiments, the training method of the matching evaluation model and / or the matching evaluation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, for example, a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into RAM 1103 and executed by the computing unit 1101, one or more steps in the training method of the matching evaluation model and / or the matching evaluation method described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured as the training method of the matching evaluation model and / or the matching evaluation method in any other appropriate manner (e.g., by means of firmware).

[0278] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0279] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data optimization device so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed completely on the machine, partially on the machine, partially on the machine as an independent software package and partially on a remote machine or completely on a remote machine or server.

[0280] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM) or a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0281] To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having: a rendering device (e.g., a cathode ray tube (CRT) renderer or a liquid crystal display (LCD)) for rendering information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices are also used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0282] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN), and the Internet.

[0283] The computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0284] The embodiments of the present disclosure also provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute a matching evaluation model training method and / or a matching evaluation method.

[0285] The embodiments of the present disclosure also provide a computer program product, including a computer program, which, when executed by a processor, implements a training method for a matching evaluation model and / or a matching evaluation method.

[0286] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in the present disclosure can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and this document does not limit this. In addition, in the present disclosure, relational terms such as "first", "second", "third", etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. In addition, "multiple" in the present disclosure can be understood as at least two.

[0287] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A training method for a matching evaluation model, comprising: Obtain search information based on search terms; Get key information based on keywords; Using a matching evaluation model, based on the search information and the key information, respectively obtain matching reasoning results of the search term and the keyword under multiple evaluation rules; The matching evaluation model is trained based on a plurality of matching reasoning results, wherein the plurality of matching reasoning results correspond one-to-one to the plurality of evaluation rules.

2. The method according to claim 1, wherein: The matching evaluation model includes a feature processing network, at least one expert network, and multiple gating networks corresponding to the multiple evaluation rules one by one; the matching evaluation model is used to obtain matching reasoning results of the search term and the keyword under multiple evaluation rules based on the search information and the key information, including: Using the feature processing network, the search information and the key information are fused to obtain a feature fusion result; Using the at least one expert network, performing feature extraction on the feature fusion result to obtain at least one feature extraction result corresponding to the at least one expert network; Each of the multiple gating networks is used as a target gating network, and the target gating network is used to adjust the weight of the at least one feature extraction result to obtain a matching degree inference result of the search term and the keyword under a target rule; wherein the target rule is an evaluation rule among the multiple evaluation rules corresponding to the target gating network.

3. The method according to claim 2, wherein: The feature processing network includes a target large model; the search information and the key information are fused by using the feature processing network to obtain a feature fusion result, including: Using the target large model, obtaining a search point reasoning result corresponding to the search information and a key point reasoning result corresponding to the key information; The feature fusion result is obtained based on the search information, the search point reasoning result, the key information and the key point reasoning result.

4. The method according to claim 3, wherein: The target large model is a trained large language model; the method further includes: Acquire multiple target expansion words related to the target word; wherein the target word is a search word sample or a keyword sample; generating target word enhancement information including the target word and the plurality of target extended words; Masking at least part of the target extended words in the target word enhancement information to obtain training data; Using the large language model, based on the training data, obtain an enhanced information inference result related to the target word; The large language model is trained using the enhanced information inference result and the target word enhanced information.

5. The method according to claim 3, wherein: The training of the matching evaluation model based on the multiple matching reasoning results includes: Obtaining the trustworthy matching degree of the search term and the keyword under the multiple evaluation rules respectively; Based on the multiple matching degree reasoning results and the multiple credible matching degrees, a matching degree loss is obtained; wherein the multiple credible matching degrees correspond one to one with the multiple evaluation rules; The matching evaluation model is trained based on the matching loss.

6. The method according to claim 5, wherein: The training of the matching evaluation model based on the matching loss includes: Acquire a credible search key point corresponding to the search information and a credible key key point corresponding to the key information; Based on the search point reasoning result and the credible search point, a first point loss is obtained; Based on the key point reasoning result and the credible key point, a second key point loss is obtained; The first key point loss and the second key point loss are combined to obtain a core key point loss; The matching evaluation model is trained based on the matching loss and the core point loss.

7. The method according to any one of claims 1 to 6, wherein: The obtaining of search information based on the search term includes: Acquire multiple search expansion words related to the search word; Search word enhancement information including the search word and the plurality of search expansion words is generated as the search information.

8. The method according to any one of claims 1 to 6, wherein: The key information obtained based on the keywords includes: Acquire multiple key expansion words related to the keyword; Keyword enhancement information including the keyword and the plurality of key expansion words is generated as the key information.

9. A matching evaluation method, comprising: Obtaining target search information based on target search terms; Obtain target key information based on target keywords; Using a matching evaluation model, based on the target search information and the target key information, matching evaluation results of the target search term and the target key word under a plurality of evaluation rules are obtained respectively.

10. The method according to claim 9, wherein: The matching evaluation model includes a feature processing network, at least one expert network, and multiple gating networks corresponding to the multiple evaluation rules one by one; the matching evaluation model is used to obtain matching evaluation results of the target search term and the target keyword under multiple evaluation rules based on the target search information and the target key information, including: Using the feature processing network, the target search information and the target key information are fused to obtain a target feature fusion result; Using the at least one expert network, performing feature extraction on the target feature fusion result to obtain at least one target feature extraction result corresponding to the at least one expert network; Each of the multiple gating networks is used as a target gating network, and the target gating network is used to adjust the weight of the at least one target feature extraction result to obtain a matching evaluation result of the search term and the keyword under a target rule; wherein the target rule is an evaluation rule among the multiple evaluation rules corresponding to the target gating network.

11. The method according to claim 10, wherein: The feature processing network includes a target large model; the target search information and the target key information are fused using the feature processing network to obtain a target feature fusion result, including: Using the target macro model, obtaining search key point acquisition results corresponding to the target search information, and key point acquisition results corresponding to the target key information; The target feature fusion result is obtained based on the target search information, the search key point acquisition result, the target key information and the key point acquisition result.

12. The method according to any one of claims 9 to 11, wherein: The step of obtaining target search information based on the target search term includes: Acquire multiple target search expansion words related to the target search word; Target search word enhancement information including the target search word and the plurality of target search expansion words is generated as the target search information.

13. The method according to any one of claims 9 to 11, wherein: The step of obtaining target key information based on the target keyword includes: Acquire multiple target keyword expansion words related to the target keyword; Target keyword enhancement information including the target keyword and the plurality of target keyword expansion words is generated as the target keyword information.

14. A training device for a matching evaluation model, comprising: A search information acquisition unit, used to acquire search information based on the search term; A key information acquisition unit, used to acquire key information based on keywords; An inference result acquisition unit, configured to obtain, by using a matching evaluation model, matching inference results of the search term and the keyword under a plurality of evaluation rules based on the search information and the key information; The first model training unit is used to train the matching evaluation model based on multiple matching reasoning results; wherein the multiple matching reasoning results correspond to the multiple evaluation rules one by one.

15. The device according to claim 14, wherein: The matching evaluation model includes a feature processing network, at least one expert network, and a plurality of gating networks corresponding to the plurality of evaluation rules one by one; and the inference result acquisition unit is used for: Using the feature processing network, the search information and the key information are fused to obtain a feature fusion result; Using the at least one expert network, performing feature extraction on the feature fusion result to obtain at least one feature extraction result corresponding to the at least one expert network; Each of the multiple gating networks is used as a target gating network, and the target gating network is used to adjust the weight of the at least one feature extraction result to obtain a matching degree inference result of the search term and the keyword under a target rule; wherein the target rule is an evaluation rule among the multiple evaluation rules corresponding to the target gating network.

16. The device according to claim 15, wherein: The feature processing network includes a target large model; the inference result acquisition unit is used to: Using the target large model, obtaining a search point reasoning result corresponding to the search information and a key point reasoning result corresponding to the key information; The feature fusion result is obtained based on the search information, the search point reasoning result, the key information and the key point reasoning result.

17. The device according to claim 16, wherein: The target large model is a trained large language model; the device also includes a second model training unit, which is used to: Acquire multiple target expansion words related to the target word; wherein the target word is a search word sample or a keyword sample; generating target word enhancement information including the target word and the plurality of target extended words; Masking at least part of the target extended words in the target word enhancement information to obtain training data; Using the large language model, based on the training data, obtain an enhanced information inference result related to the target word; The large language model is trained using the enhanced information inference result and the target word enhanced information.

18. The device according to claim 16, wherein: The first model training unit is used for: Obtaining the trustworthy matching degree of the search term and the keyword under the multiple evaluation rules respectively; Based on the multiple matching degree reasoning results and the multiple credible matching degrees, a matching degree loss is obtained; wherein the multiple credible matching degrees correspond one to one with the multiple evaluation rules; The matching evaluation model is trained based on the matching loss.

19. The device according to claim 18, wherein: The first model training unit is used for: Acquire a credible search key point corresponding to the search information and a credible key key point corresponding to the key information; Based on the search point reasoning result and the credible search point, a first point loss is obtained; Based on the key point reasoning result and the credible key point, a second key point loss is obtained; The first key point loss and the second key point loss are combined to obtain a core key point loss; The matching evaluation model is trained based on the matching loss and the core point loss.

20. The device according to any one of claims 14 to 19, wherein: The search information acquisition unit is used for: Acquire multiple search expansion words related to the search word; Search word enhancement information including the search word and the plurality of search expansion words is generated as the search information.

21. The device according to any one of claims 14 to 19, wherein: The key information acquisition unit is used for: Acquire multiple key expansion words related to the keyword; Keyword enhancement information including the keyword and the plurality of key expansion words is generated as the key information.

22. A matching evaluation device, comprising: A target search information acquisition unit, used to acquire target search information based on a target search term; A target key information acquisition unit, used to acquire target key information based on target keywords; The evaluation result acquisition unit is used to obtain the matching evaluation results of the target search term and the target keyword under multiple evaluation rules based on the target search information and the target key information by using the matching evaluation model.

23. The device according to claim 22, wherein: The matching evaluation model includes a feature processing network, at least one expert network, and a plurality of gating networks corresponding to the plurality of evaluation rules one by one; and the evaluation result acquisition unit is used for: Using the feature processing network, the target search information and the target key information are fused to obtain a target feature fusion result; Using the at least one expert network, performing feature extraction on the target feature fusion result to obtain at least one target feature extraction result corresponding to the at least one expert network; Each of the multiple gating networks is used as a target gating network, and the target gating network is used to adjust the weight of the at least one target feature extraction result to obtain a matching evaluation result of the search term and the keyword under a target rule; wherein the target rule is an evaluation rule among the multiple evaluation rules corresponding to the target gating network.

24. The device according to claim 23, wherein: The feature processing network includes a target large model; the evaluation result acquisition unit is used to: Using the target macro model, obtaining search key point acquisition results corresponding to the target search information, and key point acquisition results corresponding to the target key information; The target feature fusion result is obtained based on the target search information, the search key point acquisition result, the target key information and the key point acquisition result.

25. The device according to any one of claims 22 to 24, wherein: The target search information acquisition unit is used for: Acquire multiple target search expansion words related to the target search word; Target search word enhancement information including the target search word and the plurality of target search expansion words is generated as the target search information.

26. The device according to any one of claims 22 to 24, wherein: The target key information acquisition unit is used for: Acquire multiple target keyword expansion words related to the target keyword; Target keyword enhancement information including the target keyword and the plurality of target keyword expansion words is generated as the target keyword information.

27. An electronic device comprising: at least one processor; a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 1 to 13.

28. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1 to 13.

29. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 13.

Images