Large model intelligent recommendation method, system, medium, program product and electronic device

Abstract

The application provides a large model intelligent recommendation method, system, medium, program product and electronic equipment. The method comprises the following steps: analyzing user demand to obtain a demand semantic vector, combining an explicit constraint condition to generate a user demand feature vector, and performing model screening based on the demand semantic vector and a generated model capability vector to obtain a first candidate set; performing hard filtering on candidate models in the first candidate set by using the explicit constraint condition and supplementing recall to obtain a second candidate set; calculating model capability matching degrees, cost-benefit scores, popularity scores and freshness scores of the candidate models, and performing weighted calculation to obtain comprehensive recommendation scores of the candidate models; and finally, arranging the candidate models in the second candidate set in descending order based on the comprehensive recommendation scores, and selecting top candidate models as a recommendation result output. The application can accurately capture deep user needs, and the recommendation result is more in line with actual needs through multi-dimensional comprehensive evaluation.

Description

Technical Field

[0001] This application belongs to the field of natural language processing technology, and in particular relates to a large-scale intelligent recommendation method, system, medium, program product and electronic device. Background Technology

[0002] Currently, Large Language Model (LLM) technology is in a phase of rapid iteration and industrialization, with a large number of large model products with different capabilities emerging in the market. However, due to significant differences in core capability indicators among large models, how to quickly and accurately select the optimal large model based on specific needs has become a core technical challenge restricting the large-scale application of large models. Existing technical solutions struggle to meet the demands for efficient and accurate recommendations, exhibiting the following prominent shortcomings in practical applications:

[0003] 1. Manual screening is inefficient and highly subjective. Users need to review the technical white papers, parameter specifications, user reviews and other materials of each model one by one, and manually compare various indicators. This not only consumes a lot of manpower and time, but is also easily limited by personal experience.

[0004] 2. The keyword matching mechanism is crude and cannot analyze deep needs. Traditional recommendation solutions use tag-based search engines, which can only achieve literal keyword matching and cannot understand the user's vague and complex deep semantic needs.

[0005] 3. Single-dimensional ranking has limitations and is divorced from real-world application scenarios. Existing large-scale model ranking and recommendation tools usually only rank based on a single performance metric, ignoring core factors that are crucial in real-world applications, such as cost, response speed, context length, and adaptability to specific tasks. This results in recommendation results that are "highly performant but not applicable" and cannot match actual business needs.

[0006] 4. Lack of dynamic adaptive optimization capabilities: Existing recommendation solutions mostly adopt static rule presets, which cannot adaptively adjust recommendation strategies based on user feedback behavior, changes in business scenarios, and updates to large-scale models and markets. This makes it difficult to continuously optimize recommendation performance, and long-term use can easily lead to problems such as "recommendation homogenization" and "demand mismatch". Summary of the Invention

[0007] The purpose of this application is to provide a large-scale intelligent recommendation method, system, medium, program product, and electronic device. By constructing a unified representation space of user needs and model capabilities, and adopting a multi-stage hybrid recall and multi-objective comprehensive ranking mechanism, it can quickly and accurately output the optimal model ranking that meets the specific needs of users from a large number of large models, thereby improving the accuracy, applicability, and efficiency of recommendations.

[0008] Firstly, this application provides a large-scale intelligent recommendation method, the method comprising:

[0009] User needs are parsed to obtain a demand semantic vector. Based on the demand semantic vector and explicit constraints, a user demand feature vector is generated. Based on the demand semantic vector and the generated model capability vector, a model selection is performed to obtain a first candidate set.

[0010] The candidate models in the first candidate set are filtered using the explicit constraints in the user demand feature vector, and after filtering, it is determined whether to perform model supplementation and recall to obtain the second candidate set.

[0011] Calculate the model capability matching degree, cost-effectiveness score, popularity score and novelty score of the candidate models, and calculate the comprehensive recommendation score of each candidate model by weighting them.

[0012] Based on the comprehensive recommendation score, the candidate models in the second candidate set are sorted in descending order, and the top-ranked models are selected. The candidate models are output as recommendation results.

[0013] In some embodiments of the first aspect of this application, generating the model capability vector specifically includes:

[0014] Collect complete metadata for various large models, wherein the metadata includes at least one of the following: reasoning ability, code generation ability, mathematical calculation ability, multi-language support ability, multimodal understanding and generation ability, instruction compliance ability, text generation quality, context length, response speed, language support, price parameter, popularity index and freshness index;

[0015] The collected metadata is cleaned and normalized to form the model capability vector. ,in, The total number of capability dimensions. For the first A capability vector with 3 dimensions, where the score for each dimension ranges from [0, 1], and the strength of the model's capability in the current dimension is directly proportional to the score.

[0016] In some embodiments of the first aspect of this application, parsing user requirements yields a requirement semantic vector, generating a user requirement feature vector based on the requirement semantic vector and explicit constraints, and performing model screening based on the requirement semantic vector and the generated model capability vector to obtain a first candidate set, specifically including:

[0017] A pre-trained language model is used to extract the semantic vector of the user's natural language input, and the semantic vector is mapped to the same dimension as the model's capability vector through a fully connected layer.

[0018] The explicit constraints are quantified to obtain quantified indicators, which are then fused with the demand semantic vector to generate a complete user demand feature vector.

[0019] The cosine similarity is calculated based on the user demand semantic vector and the model capability vector, and the results are sorted in descending order based on the similarity scores.

[0020] Select the top rankings The first candidate set consists of 10 candidate models.

[0021] In this implementation, the user's natural language needs are encoded into semantic vectors by a pre-trained language model, which can accurately capture the user's ambiguous and complex deep needs. This overcomes the limitation of traditional keyword matching, which can only achieve literal matching, and greatly improves the accuracy of needs parsing, making the semantic understanding ability outstanding and the needs parsing more accurate.

[0022] In some embodiments of the first aspect of this application, the candidate models in the first candidate set are filtered using explicit constraints in the user demand feature vector, and after filtering, it is determined whether to perform model supplementation recall to obtain a second candidate set, specifically including:

[0023] Hard filtering is performed based on explicit constraints. Specifically, the first candidate set is hard filtered according to the explicit constraints in the user demand feature vector, eliminating models that do not meet any of the constraints to obtain the second candidate set. The filtering logic is calculated as follows:

[0024]

[0025] ;

[0026] in, As the first candidate set, As the second candidate set, As a candidate model, The price parameters for the candidate models, Quantified values ​​for budget constraints The maximum context length supported by the candidate model. Quantize the value for context length constraints. For the language support range of the candidate models, Quantifying language preferences For the response speed of the candidate model, To constrain the quantization value for response speed, For explicit constraints; and

[0027] The number of models in the second candidate set is extracted for judgment. If the number of models is less than a preset threshold, models are supplemented and recalled from the full set of models in descending order of popularity until the number of models equals the preset threshold.

[0028] In some embodiments of the first aspect of this application, the model capability matching degree, cost-effectiveness score, popularity score, and novelty score of the candidate models are calculated, and a weighted comprehensive recommendation score for each model is obtained, specifically including:

[0029] The model capability matching degree is calculated using the following formula:

[0030] ;

[0031] ;

[0032] ;

[0033] in, For model capability matching degree, and These are the weighting coefficients, and ; For semantic similarity, For demand semantic vectors, This is the model capability vector; The context matching degree has a value range of [0, 1]. For the context length required by the user, The maximum context length supported by the candidate model. This represents the maximum context length of all models in the full model. For language matching degree, where if the candidate model supports all target languages ​​required by the user, then If not supported at all, then If partially supported, then ;

[0034] The cost-benefit score is calculated using the following formula:

[0035] ;

[0036] in, The cost-effectiveness score ranges from [0, 1]. Candidate models Price parameters, The lowest price among all models in the second candidate set. To prevent extremely small positive numbers from being divided by zero, It is a cost-sensitive index, and

[0037] The popularity score is calculated using the following formula:

[0038] ;

[0039] in, To score popularity, To take the average value function, This represents the number of downloads for the candidate models. This represents the maximum number of downloads for the candidate models. API call frequency for candidate models The highest API call frequency among the candidate models. Users rate the candidate models. The highest user rating among the candidate models;

[0040] The freshness score is calculated using the following formula:

[0041] ;

[0042] in, To score freshness, The number of days since the candidate model was released. This represents the maximum number of days since the release of the candidate model.

[0043] The weighted average recommendation score for each model is calculated as follows:

[0044] ;

[0045] in, To determine the overall recommendation score, For model capability matching degree, For cost-effectiveness score, To score popularity, To score freshness, and Here, are the exponential weighting coefficients, where .

[0046] In this implementation, a multi-objective optimization method is set up, which considers multiple core dimensions, including model capability matching degree, cost-effectiveness score, popularity score, and novelty score. This avoids the one-sidedness of a single indicator-oriented approach. The recommendation results not only meet the user's core capability needs, but also take into account practical application factors such as cost and timeliness, and are suitable for various business scenarios.

[0047] In some embodiments of the first aspect of this application, the method further includes obtaining feedback data on the recommendation results and dynamically adjusting parameters using a preset algorithm in conjunction with the feedback data, the parameters including weight coefficients. and and the index weighting coefficient and .

[0048] To achieve the above and other related objectives, a second aspect of this application provides a large-scale intelligent recommendation system, the system comprising:

[0049] The parsing module is used to parse user requirements to obtain a requirement semantic vector, generate a user requirement feature vector based on the requirement semantic vector and explicit constraints, and perform model screening based on the requirement semantic vector and the generated model capability vector to obtain a first candidate set.

[0050] The recall module is used to filter candidate models in the first candidate set using the explicit constraints in the user demand feature vector, and after filtering, to determine whether to perform model supplementation recall to obtain a second candidate set.

[0051] The calculation module is used to calculate the model capability matching degree, cost-effectiveness score, popularity score and novelty score of the candidate models, and to calculate the comprehensive recommendation score of each candidate model by weighting.

[0052] The output module is used to sort the candidate models in the second candidate set in descending order based on the comprehensive recommendation score, and select the top-ranked models. The candidate models are output as recommendation results.

[0053] To achieve the above and other related objectives, a third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the large-model intelligent recommendation method described in any of the preceding claims.

[0054] To achieve the above and other related objectives, a fourth aspect of this application provides a computer program product comprising computer program code, which, when executed on a computer, causes the computer to implement the large-model intelligent recommendation method described in any of the preceding claims.

[0055] To achieve the above and other related objectives, a fifth aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the large-model intelligent recommendation method described in any of the preceding claims.

[0056] As described above, the large-model intelligent recommendation method, system, medium, program product, and electronic device described in this application also include other effects: enhanced interpretability, increasing user trust, wherein when outputting recommendation results, this application simultaneously provides detailed scores for each dimension and natural language recommendation reasons, clearly presenting the basis for recommendation, which can help users quickly understand the model's adaptability advantages, increase user trust and recommendation result adoption rate; and strong adaptive optimization capability, continuously improving recommendation effect, through feedback learning mechanism, dynamically adjusting ranking weight parameters based on user feedback, enabling the recommendation strategy to be continuously optimized with changes in user preferences, adjustments in business scenarios, and updates in the large-model market, possessing good adaptability and scalability, and the recommendation accuracy can be continuously improved after long-term use. Attached Figure Description

[0057] Figure 1 The diagram shows a step-by-step illustration of the large-model intelligent recommendation method of this application in one embodiment;

[0058] Figure 2 The diagram shows a flowchart illustrating the generation of model capability vectors in one embodiment of the large-model intelligent recommendation method of this application.

[0059] Figure 3 The diagram shows a flowchart of the large-model intelligent recommendation method of this application in one embodiment for generating user demand features;

[0060] Figure 4 The diagram shows a flowchart of constructing a second candidate set in one embodiment of the large-model intelligent recommendation method of this application;

[0061] Figure 5 The diagram shows a flowchart illustrating the output recommendation results of the large-model intelligent recommendation method of this application in one embodiment.

[0062] Figure 6 The diagram shown is a structural schematic of a large-model intelligent recommendation system according to an embodiment of this application;

[0063] Figure 7 The diagram shown is a structural schematic of the electronic device of this application in one embodiment.

[0064] Component designation explanation

[0065] S102~S108 step 60 Large-scale intelligent recommendation system 61 Parsing module 62 Recall Module 63 Calculation module 64 Output module 700 Electronic terminal 701 processor 702 memory 7021 operating system 7022 app 703 Network interface 704 bus system 705 User Interface Detailed Implementation

[0066] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0067] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0068] Furthermore, the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. If the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed in this application.

[0069] The technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0070] like Figure 1 As shown, in one embodiment of the invention, the large-model intelligent recommendation method of this application includes the following steps:

[0071] Step S102: Parse user requirements to obtain a requirement semantic vector;

[0072] Step S104: Generate a user demand feature vector based on the demand semantic vector and explicit constraints, and perform model screening based on the demand semantic vector and the generated model capability vector to obtain a first candidate set.

[0073] Step S106: Filter the candidate models in the first candidate set using the explicit constraints in the user demand feature vector, and determine whether to perform model supplementation and recall after filtering to obtain the second candidate set.

[0074] Step S108: Calculate the model capability matching degree, cost-effectiveness score, popularity score and novelty score of the candidate models, and calculate the comprehensive recommendation score of each candidate model by weighting them.

[0075] Step S110: Based on the comprehensive recommendation score, sort the candidate models in the second candidate set in descending order and select the top-ranked models. The candidate models are output as recommendation results.

[0076] It should be noted that, in this embodiment, when applied, a standardized model capability feature library must first be constructed to output different model capability vectors, such as... Figure 2 As shown, generating the model capability vector specifically includes: collecting complete metadata for various large models, wherein the metadata includes, but is not limited to, inference ability, code generation ability, mathematical calculation ability, multi-language support ability, multimodal understanding and generation ability, instruction compliance ability, text generation quality, context length, response speed, language support, price parameter, popularity index, and freshness index; and cleaning and normalizing the collected metadata to eliminate dimensional differences and outliers, thereby constructing a standardized model capability feature library and obtaining the model capability vector. ,in, The total number of capability dimensions. For the first A capability vector with 3 dimensions, each dimension having a score range of [0, 1]. The strength of the model's capability in the current dimension is directly proportional to the score; correspondingly, the higher the score, the stronger the model's capability in that dimension.

[0077] Specifically, in one embodiment of the invention, model metadata from the large model market is collected and updated regularly to construct and maintain a standardized model capability feature library. In this embodiment, a total of 327 mainstream large model products on the market are collected. Each model is quantitatively scored on 8 core capability dimensions (reasoning ability, code generation ability, mathematical calculation ability, multilingual support ability, multimodal ability, instruction compliance ability, text generation quality, and stability) to form an 8-dimensional capability vector. For example, taking text generation quality as an example, the specific descriptions for different models include "style controllability (formal / colloquial / creative)," "factuality (frequency of illusions)," "coherence (paragraph connection, long text structure)," and "degree of repetition and redundancy." Or, taking multilingual support as an example, the specific descriptions for different models include "token efficiency for non-Latin languages," "ability to handle mixed multilingual input," etc. At the same time, metadata such as price parameters, context length, and response speed of each model are collected, normalized, and stored in the model capability feature library. A daily incremental update mechanism is adopted to ensure data timeliness.

[0078] Furthermore, in one embodiment of the invention, parsing user needs yields a demand semantic vector; generating a user demand feature vector based on the demand semantic vector and explicit constraints; and performing model screening based on the demand semantic vector and the generated model capability vector to obtain a first candidate set, specifically including:

[0079] A pre-trained language model is used to extract the semantic vector of the user's natural language input, and the semantic vector is mapped to the same dimension as the model's capability vector through a fully connected layer.

[0080] The explicit constraints are quantified to obtain quantified indicators, which are then fused with the demand semantic vector to generate a complete user demand feature vector.

[0081] The cosine similarity is calculated based on the user demand semantic vector and the model capability vector, and the results are sorted in descending order based on the similarity scores.

[0082] Select the top rankings The first candidate set consists of 10 candidate models.

[0083] It should be noted that, in this embodiment, as Figure 3 As shown, a pre-trained language model is used to extract semantic features of user input natural language requirements, resulting in a semantic vector of the requirements. To achieve the model capability vector The cosine similarity calculation of semantic vectors By mapping to the same dimension through a fully connected layer, explicit constraints are quantified and converted into quantitative indicators corresponding to the model's capability feature library, along with the requirement semantic vector. The features are then fused to generate a complete user demand feature vector. ,in, For user demand feature vectors, For demand semantic vectors, Quantified values ​​for budget constraints Quantize the value for context length constraints. Quantifying language preferences The quantized value is used to constrain the response speed.

[0084] Specifically, in one embodiment of the invention, the user inputs a requirement text and explicit constraints. In this embodiment, an ERNIE 3.0 pre-trained language model is used to encode the user requirement text (such as "Need a model that can handle long Chinese documents (context ≥ 64K), has strong reasoning ability, a cost of less than 0.01 yuan / thousand tokens, and a response latency ≤ 200ms") into a "768"-dimensional semantic vector. To achieve the "8"-dimensional capability vector of the model The cosine similarity calculation requires the "768"-dimensional semantic vector to be transformed. The fully connected layer maps to an 8-dimensional capability evaluation space; simultaneously, it quantifies the explicit constraints set by the user. , , , After fusion, a complete user demand feature vector is generated. .

[0085] Furthermore, in this embodiment, a cosine similarity is calculated based on the user demand semantic vector and the model capability vector, and the results are sorted in descending order based on the similarity scores to select the top-ranked results. The first candidate set consists of several candidate models, wherein... It can be adjusted according to the size of the model library, for example, to "100".

[0086] Specifically, in this embodiment, the calculation is first performed. The cosine similarity between the model's capability vector and the model's capability vector is used to recall the 100 most similar models as the first candidate set. .

[0087] Furthermore, in one embodiment of the invention, the candidate models in the first candidate set are filtered using explicit constraints in the user demand feature vector, and after filtering, it is determined whether to perform model supplementation and recall to obtain a second candidate set, specifically including:

[0088] Hard filtering is performed based on explicit constraints. Specifically, the first candidate set is hard filtered according to the explicit constraints in the user demand feature vector, eliminating models that do not meet any of the constraints to obtain the second candidate set. The filtering logic is calculated as follows:

[0089]

[0090] ;

[0091] in, As the first candidate set, As the second candidate set, As a candidate model, The price parameters for the candidate models, Quantified values ​​for budget constraints The maximum context length supported by the candidate model. Quantize the value for context length constraints. For the language support range of the candidate models, Quantifying language preferences For the response speed of the candidate model, Quantization values ​​constrained by response speed; and

[0092] The number of models in the second candidate set is extracted for judgment. If the number of models is less than a preset threshold, supplementary recall is carried out from the full set of models in descending order of popularity until the number of models is equal to the preset threshold.

[0093] It should be noted that, in this embodiment, as Figure 4 As shown, based on the user demand feature vector The explicit constraints in the first candidate set Hard filtering is performed to eliminate models that do not meet any of the constraints, resulting in a second candidate set. The filtering logic is as follows:

[0094]

[0095] ;

[0096] in, As the first candidate set, As the second candidate set, As a candidate model, The price parameters for the candidate models, Quantified values ​​for budget constraints The maximum context length supported by the candidate model. Quantize the value for context length constraints. For the language support range of the candidate models, Quantifying language preferences For the response speed of the candidate model, The quantized value is used to constrain the response speed.

[0097] Specifically, in one embodiment of the invention, the explicit constraints are applied to... Filtering is performed to remove models with prices higher than "0.01 yuan / thousand" tokens, context length less than "64K", no support for Chinese characters, or response latency exceeding "200ms", resulting in the second candidate set. In this embodiment It includes "38" candidate models.

[0098] Furthermore, in this embodiment, the application also includes supplementary recall, specifically extracting the number of models in the second candidate set for judgment. If the number of models is less than a preset threshold, supplementary recall is performed from all models in descending order of popularity until the number of models equals the preset threshold, so as to ensure that there are enough model samples in the subsequent ranking process, while improving the diversity of recommendation results and avoiding homogenization.

[0099] Specifically, in one embodiment of the invention, due to the above... The size (number of candidate models is 38) is less than the preset threshold. Therefore, 12 models were recalled from the full model library in descending order of popularity to supplement the database. Ultimately, 50 candidate models were obtained.

[0100] Furthermore, in one embodiment of the invention, the model capability matching degree, cost-effectiveness score, popularity score, and novelty score of the candidate models are calculated, and a weighted comprehensive recommendation score for each model is obtained, specifically including:

[0101] The model capability matching degree is calculated using the following formula:

[0102] ;

[0103] ;

[0104] ;

[0105] in, For model capability matching degree, and These are the weighting coefficients, and ; For semantic similarity, For demand semantic vectors, This is the model capability vector; The context matching degree has a value range of [0, 1]. For the context length required by the user, The maximum context length supported by the candidate model. This represents the maximum context length of all models in the full model. For language matching degree, where if the candidate model supports all target languages ​​required by the user, then If not supported at all, then If partially supported, then ;

[0106] The cost-benefit score is calculated using the following formula:

[0107] ;

[0108] in, The cost-effectiveness score ranges from [0, 1]. Candidate models Price parameters, The lowest price among all models in the second candidate set. To prevent extremely small positive numbers from being divided by zero, It is a cost-sensitive index, and ;

[0109] The popularity score is calculated using the following formula:

[0110] ;

[0111] in, To score popularity, To take the average value function, This represents the number of downloads for the candidate models. This represents the maximum number of downloads for the candidate models. API call frequency for candidate models The highest API call frequency among the candidate models. Users rate the candidate models. The highest user rating among the candidate models;

[0112] The freshness score is calculated using the following formula:

[0113] ;

[0114] in, To score freshness, The number of days since the candidate model was released. This represents the maximum number of days since the release of the candidate model.

[0115] The weighted average recommendation score for each model is calculated as follows:

[0116] ;

[0117] in, To determine the overall recommendation score, For model capability matching degree, For cost-effectiveness score, To score popularity, To score freshness, and Here, are the exponential weighting coefficients, where .

[0118] It should be noted that, in this embodiment, as Figure 5 As shown, a multi-objective comprehensive ranking is performed to calculate the recommendation score, which includes calculating the matching degree of the computational model capability, as shown in the following formula:

[0119] ;

[0120] ;

[0121] ;

[0122] in, Model capability matching score measures the degree to which a model's capabilities align with user needs. The calculation utilizes... and These are the weighting coefficients, and Initial values ​​can be preset according to actual application scenarios and then dynamically adjusted through feedback learning. For semantic similarity, For demand semantic vectors, This is the model capability vector; Context matching score measures how well the model's context length matches user needs. Its value ranges from [0, 1]. A higher score indicates a stronger fit to the context length. For the context length required by the user, The maximum context length supported by the candidate model. This represents the maximum context length of all models in the full model. For language matching degree, where if the candidate model supports all target languages ​​required by the user, then If not supported at all, then If partially supported, then .

[0123] Furthermore, in this embodiment, the cost-benefit score is calculated using the following formula:

[0124] ;

[0125] in, The cost-effectiveness score ranges from [0, 1]. A higher score indicates better cost-effectiveness of the model. Candidate models Price parameters, The lowest price among all models in the second candidate set. To prevent extremely small positive numbers from being divided by zero, It is a cost-sensitive index, and , , ;

[0126] Furthermore, in this embodiment, the popularity score is calculated using the following formula:

[0127] ;

[0128] in, To score popularity, To take the average value function, This represents the number of downloads for the candidate models. This represents the maximum number of downloads for the candidate models. API call frequency for candidate models The highest API call frequency among the candidate models. Users rate the candidate models. The highest user rating among the candidate models;

[0129] Furthermore, in this embodiment, the freshness score is calculated using the following formula:

[0130] ;

[0131] in, To score freshness, The number of days since the candidate model was released. This represents the maximum number of days since the release of the candidate model.

[0132] Furthermore, in this embodiment, the comprehensive recommendation score of each model is calculated using a weighted average, as shown in the following formula:

[0133] ;

[0134] in, To determine the overall recommendation score, For model capability matching degree, For cost-effectiveness score, To score popularity, To score freshness, and Here, are the exponential weighting coefficients, where The index weighting coefficients can be adjusted according to the actual scenario and dynamically optimized through feedback learning.

[0135] Furthermore, in this embodiment, based on the comprehensive recommendation score... For the second candidate set The candidate models are sorted in descending order, and the top-ranked models are selected. One model ( The final recommendation result is output as a preset number of recommendations (which can be customized by the user). The recommendation result includes model identifier, comprehensive score, detailed scores for each dimension (ability matching degree, cost-effectiveness score, popularity score, and novelty score), and recommendation reasons in natural language form, which makes it easier for users to understand the basis for the recommendation and improves user trust.

[0136] Specifically, in one embodiment of the invention, for The "50" candidate models are used to calculate a comprehensive recommendation score. In this embodiment, the initial weight parameters are set as follows: ; The comprehensive recommendation score is calculated using a log-linear form. Then, the 50 candidate models are sorted in descending order of their comprehensive recommendation scores, and the top 10 models are selected as the recommendation results to be displayed to the user. The output includes: model name, comprehensive score, detailed scores for each dimension (capability matching degree, cost-effectiveness score, popularity score, and novelty score), core model parameters (context length, response speed, and price), and natural language recommendation reasons. A radar chart of each model's score is also generated for easy comparison by the user.

[0137] Furthermore, in one embodiment of the invention, the method further includes obtaining feedback data on the recommendation results and dynamically adjusting parameters using a preset algorithm in conjunction with the feedback data, the parameters including weight coefficients. and and the index weighting coefficient and .

[0138] It should be noted that, in this embodiment, a user feedback collection mechanism is constructed to collect user feedback behavior on the recommendation results in real time, including but not limited to: clicking to view, accepting and using, abandoning the selection, manually filtering and replacing, rating, etc.; every 24 hours, based on the feedback data, the weight coefficients in the above embodiment are dynamically adjusted using Bayesian optimization or reinforcement learning methods. and and index weight parameters and To achieve adaptive optimization of the recommendation strategy and continuously improve recommendation accuracy and user satisfaction, Bayesian optimization can be used in this embodiment. The Bayesian optimization process includes identifying positive user feedback, negative user feedback, no user feedback, and finally normalization processing for parameter updates. In this embodiment, Bayesian optimization is used only as an application and has not been modified, so the update process will not be described in detail here.

[0139] Specifically, in this embodiment, in practical application, users select "accept" for recommended model A, "not interested" for model B, and manually filter and replace model C. Every 24 hours, based on accumulated feedback data, a Bayesian optimization algorithm is used to update the weight coefficients to maximize the user acceptance rate. After one month of operation, the capability matching weight coefficients are... The cost weighting coefficient was adaptively adjusted from "1.0" to "1.2". The popularity weighting coefficient was adjusted from "0.5" to "0.4". The freshness weighting coefficient has been adjusted from "0.3" to "0.2". Keep "0.2" unchanged (totaling to "2"), and apply the parameter update mechanism in this embodiment to ensure that the recommendation strategy can be continuously optimized as user preferences change.

[0140] Furthermore, to verify the technical effectiveness of this application, a comparative experiment was conducted using the same model library and test set, comparing this application with existing mainstream large-scale model recommendation methods. The control group used general semantic retrieval, tag filtering, and simple weighted ranking; the experimental group used the unified representation space, two-stage recall, four-dimensional log-linear weighting, and feedback adaptive optimization method proposed in this application. The experimental results are as follows: the Top 5 recommendation accuracy of this application reached 91.5%, an improvement of 7.3% compared to existing technologies; the constraint satisfaction rate reached 96.1%, an improvement of 13.5%; the average recommendation time was 185ms, a reduction of 11.3%; and the user acceptance rate was 89.8%, an improvement of 5.5%. The experiments demonstrate that this application represents a significant technical advancement in recommendation accuracy, constraint compliance, computational efficiency, and user acceptance.

[0141] This application also provides a large-model intelligent recommendation system, which can implement the large-model intelligent recommendation method described in this application. However, the implementation device of the large-model intelligent recommendation method described in this application includes, but is not limited to, the structure of the large-model intelligent recommendation system listed in this embodiment. All structural modifications and substitutions of the prior art made based on the principles of this application are included within the protection scope of this application.

[0142] Please see Figure 6 In one embodiment, this embodiment provides a large-scale intelligent recommendation system 60, the system comprising:

[0143] The parsing module 61 is used to parse user requirements to obtain a requirement semantic vector, generate a user requirement feature vector based on the requirement semantic vector and explicit constraints, and perform model screening based on the requirement semantic vector and the generated model capability vector to obtain a first candidate set.

[0144] The recall module 62 is used to filter the candidate models in the first candidate set using the explicit constraints in the user demand feature vector, and after filtering, to determine whether to perform model supplementation recall to obtain the second candidate set.

[0145] The calculation module 63 is used to calculate the model capability matching degree, cost-effectiveness score, popularity score and novelty score of the candidate models, and to calculate the comprehensive recommendation score of each candidate model by weighting.

[0146] Output module 64 is used to sort the candidate models in the second candidate set in descending order based on the comprehensive recommendation score, and select the top-ranked models. The candidate models are output as recommendation results.

[0147] Since the specific implementation of this embodiment corresponds to the aforementioned method embodiment, the same details will not be repeated here, and those skilled in the art should also understand this. Figure 6 The division of the modules in the embodiments is only a logical functional division. In actual implementation, they can be fully or partially integrated into one or more physical entities. These modules can be fully implemented in software through processing element calls, fully implemented in hardware, or some modules can be implemented in software through processing element calls and some modules can be implemented in hardware.

[0148] It should be understood that the disclosed apparatus or method can be implemented in other ways, given the several embodiments provided in this application. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules or units may be electrical, mechanical, or other forms.

[0149] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this application, depending on actual needs. For example, the functional modules / units in the various embodiments of this application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0150] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0151] This application also provides an electronic device, such as... Figure 7As shown, the electronic device includes a processor and a memory.

[0152] like Figure 7 As shown, the electronic device includes: at least one processor 701, a memory 702, at least one network interface 703, and a user interface 705. The various components in the device are coupled together via a bus system 704. It is understood that the bus system 704 is used to implement communication between these components. In addition to a data bus, the bus system 704 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 7 The general will label all buses as bus systems.

[0153] The user interface 705 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0154] It is understood that memory 702 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable categories of memory.

[0155] In this embodiment, the memory 702 is used to store various types of data to support the operation of the electronic terminal 700. Examples of this data include any executable program for operation on the electronic terminal 700, such as the operating system 7021 and application programs 7022. The operating system 7021 contains various system programs, such as the framework layer, core library layer, and driver layer, for implementing various basic services and handling hardware-based tasks. The application program 7022 may contain various applications, such as a media player and a browser, for implementing various application services. The methods provided in this embodiment can be included in the application program 7022.

[0156] The methods disclosed in the embodiments of this application can be applied to processor 701, or implemented by processor 701. Processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 701 or by instructions in the form of software. The processor 701 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 701 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. Processor 701 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0157] In an exemplary embodiment, the electronic terminal 700 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to execute the aforementioned method.

[0158] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to execute the method of any of the embodiments of the above-described large model intelligent recommendation method.

[0159] According to the method provided in the embodiments of this application, this application also provides a computer-readable storage medium storing program code, which, when run on a computer, causes the computer to perform the method of any of the embodiments of the above-described large model intelligent recommendation method.

[0160] The terms “component,” “module,” “system,” etc., used in this specification are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0161] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0162] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0163] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0164] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0165] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0166] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0167] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0168] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0169] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A large-scale intelligent recommendation method, characterized in that, include: Parse user requirements to obtain a requirement semantic vector; Based on the demand semantic vector and explicit constraints, a user demand feature vector is generated, and based on the demand semantic vector and the generated model capability vector, a first candidate set is obtained by model screening. The candidate models in the first candidate set are filtered using the explicit constraints in the user demand feature vector, and after filtering, it is determined whether to perform model supplementation and recall to obtain the second candidate set. Calculate the model capability matching degree, cost-effectiveness score, popularity score, and novelty score of the candidate models in the second candidate set, and calculate the comprehensive recommendation score of each candidate model by weighting them. Based on the comprehensive recommendation score, the candidate models in the second candidate set are sorted in descending order, and the top-ranked models are selected. The candidate models are output as recommendation results.

2. The large-model intelligent recommendation method according to claim 1, characterized in that, The method further includes generating the model capability vector, specifically including: Collect complete metadata for various large models, wherein the metadata includes at least one of the following: reasoning ability, code generation ability, mathematical calculation ability, multi-language support ability, multimodal understanding and generation ability, instruction compliance ability, text generation quality, context length, response speed, language support, price parameter, popularity index and freshness index; The collected metadata is cleaned and normalized to form the model capability vector. ,in, The total number of capability dimensions. For the first A capability vector with 3 dimensions, where the score for each dimension ranges from [0, 1], and the strength of the model's capability in the current dimension is directly proportional to the score.

3. The large-model intelligent recommendation method according to claim 1, characterized in that, The process involves parsing user needs to obtain a semantic vector of those needs, generating a user need feature vector based on the semantic vector and explicit constraints, and then performing model screening based on the semantic vector and the generated model capability vector to obtain a first candidate set. Specifically, this includes: A pre-trained language model is used to extract the semantic vector of the user's natural language input, and the semantic vector is mapped to the same dimension as the model's capability vector through a fully connected layer. The explicit constraints are quantified to obtain quantified indicators, which are then fused with the demand semantic vector to generate a complete user demand feature vector. The cosine similarity is calculated based on the user demand semantic vector and the model capability vector, and the results are sorted in descending order based on the similarity scores. Select the top rankings The first candidate set consists of 10 candidate models.

4. The large-model intelligent recommendation method according to claim 1, characterized in that, The candidate models in the first candidate set are filtered using explicit constraints in the user demand feature vector, and after filtering, it is determined whether to perform model supplementation and recall to obtain a second candidate set. Specifically, this includes: Hard filtering is performed based on explicit constraints. Specifically, the first candidate set is hard filtered according to the explicit constraints in the user demand feature vector, eliminating models that do not meet any of the constraints to obtain the second candidate set. The filtering logic is calculated as follows: ； in, As the first candidate set, As the second candidate set, As a candidate model, The price parameters for the candidate models, Quantified values ​​for budget constraints The maximum context length supported by the candidate model. Quantize the value for context length constraints. For the language support range of the candidate models, Quantifying language preferences For the response speed of the candidate model, To constrain the quantization value for response speed, For explicit constraints; and The number of models in the second candidate set is extracted for judgment. If the number of models is less than a preset threshold, models are supplemented and recalled from the full set of models in descending order of popularity until the number of models equals the preset threshold.

5. The large-model intelligent recommendation method according to claim 1, characterized in that, The model capability matching degree, cost-effectiveness score, popularity score, and novelty score of the candidate models are calculated, and a weighted comprehensive recommendation score for each model is obtained, which includes: The model capability matching degree is calculated using the following formula: ； ； ； in, For model capability matching degree, and These are the weighting coefficients, and ; For semantic similarity, For demand semantic vectors, This is the model capability vector; The context matching degree has a value range of [0, 1]. For the context length required by the user, The maximum context length supported by the candidate model. This represents the maximum context length of all models in the full model. For language matching degree, where if the candidate model supports all target languages ​​required by the user, then If not supported at all, then If partially supported, then ; The cost-benefit score is calculated using the following formula: ； in, The cost-effectiveness score ranges from [0, 1]. Candidate models Price parameters, The lowest price among all models in the second candidate set. To prevent extremely small positive numbers from being divided by zero, It is a cost-sensitive index, and ; The popularity score is calculated using the following formula: ； in, To score popularity, To take the average value function, This represents the number of downloads for the candidate models. This represents the maximum number of downloads for the candidate models. API call frequency for candidate models The highest API call frequency among the candidate models. Users rate the candidate models. The highest user rating among the candidate models; The freshness score is calculated using the following formula: ； in, To score freshness, The number of days since the candidate model was released. The maximum number of days since the release of the candidate model; The weighted average recommendation score for each model is calculated as follows: ； in, To determine the overall recommendation score, For model capability matching degree, For cost-effectiveness score, To score popularity, To score freshness, and Here, are the exponential weighting coefficients, where .

6. The large-model intelligent recommendation method according to claim 1, characterized in that, The method further includes obtaining feedback data on the recommendation results and dynamically adjusting parameters using a preset algorithm in conjunction with the feedback data, the parameters including weight coefficients. and and the index weighting coefficient and .

7. A large-scale intelligent recommendation system, characterized in that, include: The parsing module is used to parse user requirements to obtain a requirement semantic vector, generate a user requirement feature vector based on the requirement semantic vector and explicit constraints, and perform model screening based on the requirement semantic vector and the generated model capability vector to obtain a first candidate set. The recall module is used to filter candidate models in the first candidate set using the explicit constraints in the user demand feature vector, and after filtering, to determine whether to perform model supplementation recall to obtain a second candidate set. The calculation module is used to calculate the model capability matching degree, cost-effectiveness score, popularity score and novelty score of the candidate models, and to calculate the comprehensive recommendation score of each candidate model by weighting. The output module is used to sort the candidate models in the second candidate set in descending order based on the comprehensive recommendation score, and select the top-ranked models. The candidate models are output as recommendation results.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the large-model intelligent recommendation method as described in any one of claims 1 to 6.

9. A computer program product, characterized in that, The computer program product includes computer program code, which, when run on a computer, enables the computer to implement the large model intelligent recommendation method as described in any one of claims 1 to 6.

10. An electronic device, characterized in that, The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to enable the electronic device to perform the large model intelligent recommendation method as described in any one of claims 1 to 6.

Images