Product recommendation model training method, product recommendation method, equipment and readable storage medium

Abstract

The invention discloses a product recommendation model training method, a product recommendation method, equipment and a readable storage medium, and relates to the technical field of artificial intelligence, the product recommendation model training method comprises the following steps: screening out candidate product data for each first user portrait from original product data according to user characteristic information; matching the first user portraits with the candidate product data to obtain a plurality of second user portraits meeting preset conditions, recommended product data and recommendation reasons; on the basis of the second user portraits, the recommended product data and the recommendation reasons, performing supervised fine tuning training on the basic reasoning model to obtain a supervised fine tuning model; and performing reinforcement learning training on the supervision fine tuning model according to the plurality of third user portraits which do not meet the preset condition and the plurality of corresponding candidate product data to obtain a target product recommendation model. The invention aims at obtaining a small and medium model with low illusion and high recommendation accuracy through low-cost training, and improving the product recommendation efficiency and accuracy.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a product recommendation model training method, a product recommendation method, a device, and a computer-readable storage medium. Background Art

[0002] Product recommendation is a crucial component of commercial marketing. With the advancement of big model technology, the product recommendation process is increasingly shifting from manual work to the use of big models. However, traditional big language models suffer from significant AI (artificial intelligence) illusions during product recommendation, resulting in low product recommendation accuracy. This is especially true when the input data is in text form, as big language models cannot clearly identify the key information to focus on when making product recommendations. Furthermore, using big language models for product recommendations requires excessive time and computing resources, resulting in significant financial and time costs.

[0003] The above information disclosed in this Background section is only for understanding the background of the present invention and therefore it may contain information that does not constitute prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a product recommendation model training method, product recommendation method, device and computer-readable storage medium, aiming to obtain a small or medium-sized model with low hallucination and high recommendation accuracy through low-cost training, thereby improving the efficiency and accuracy of product recommendations.

[0005] To achieve the above objectives, the present application provides a product recommendation model training method, which includes: Obtaining original product data and user portrait data, and filtering out a first preset number of candidate product data for each first user portrait from the original product data based on user characteristic information of each first user portrait in the user portrait data; Matching each of the first user portraits with a corresponding first preset number of candidate product data, matching multiple second user portraits that meet preset conditions and a corresponding second preset number of recommended product data, and determining a recommendation reason for each of the recommended products, wherein the second preset number is less than the first preset number; Based on each of the second user profiles and the recommended product data and recommendation reasons corresponding to each of the second user profiles, supervised fine-tuning training is performed on the preset basic inference model to obtain a supervised fine-tuning model; Based on multiple third user portraits that do not meet the preset conditions in each of the first user portraits and multiple candidate product data corresponding to each of the third user portraits, reinforcement learning training is performed on the supervised fine-tuning model to obtain a target product recommendation model.

[0006] In one embodiment, the step of selecting a first preset number of candidate product data corresponding to each first user portrait from the original product data based on the user characteristic information of each first user portrait in the user portrait data includes: Eliminate, based on the user characteristic information of each first user portrait, product data that does not match each first user portrait in the original product data, thereby obtaining a plurality of product data corresponding to each first user portrait after elimination; A first preset number of product data are randomly selected from the multiple product data corresponding to each first user portrait after elimination to obtain candidate product data.

[0007] In one embodiment, the steps of matching each of the first user profiles with the corresponding first preset number of candidate product data, matching multiple second user profiles that meet preset conditions and the corresponding second preset number of recommended product data, and determining the recommendation reason for each of the recommended products include: Inputting each of the first user profiles and the candidate product data corresponding to each of the first user profiles into a preset large language model, and inputting a first prompt word into the large language model, guiding the large language model to output a second preset number of recommended product data from the corresponding candidate product data for each of the first user profiles, and outputting the thinking process and the reason for recommending each of the recommended products; The large language model is used to filter the recommended product data or candidate product data corresponding to the recommendation reasons that do not meet the preset conditions, and obtain multiple second user portraits and corresponding recommended product data and recommendation reasons.

[0008] In one embodiment, the step of filtering the recommended product data or candidate product data corresponding to the recommendation reasons that do not meet the preset conditions using the large language model to obtain multiple second user profiles and corresponding recommended product data and recommendation reasons includes: Inputting the second prompt word into the large language model to guide the large language model to filter candidate product data with unreasonable thinking processes; For the filtered first user portrait, the third prompt word is input into the large language model to guide the large language model to filter the candidate product data with inappropriate recommendation reasons, thereby obtaining multiple second user portraits and corresponding recommended product data and recommendation reasons.

[0009] In one embodiment, the step of performing supervised fine-tuning training on a preset basic inference model based on each second user profile and the recommended product data and recommendation reasons corresponding to each second user profile to obtain a supervised fine-tuning model includes: Each of the second user profiles, the recommended product data and the recommendation reasons corresponding to each of the second user profiles are used as training sample data, and are sequentially input into a preset basic reasoning model, and the basic reasoning model outputs a corresponding prediction result; Calculating the loss function value of the basic reasoning model based on the recommended product data and recommendation reasons corresponding to each second user profile and the corresponding prediction results; Based on the loss function value, the basic inference model is trained until a preset training round is reached to obtain a supervised fine-tuning model.

[0010] In one embodiment, the step of performing reinforcement learning training on the supervised fine-tuning model based on multiple third user profiles that do not meet preset conditions in each of the first user profiles and multiple candidate product data corresponding to each of the third user profiles to obtain a target product recommendation model includes: Randomly selecting a third preset number of third user portraits that do not meet the preset conditions from each of the first user portraits, and obtaining a plurality of candidate product data corresponding to each of the third user portraits; Inputting each of the third user portraits into the supervised fine-tuning model in sequence, and having the supervised fine-tuning model output a corresponding prediction result; Determining a reward function value of the supervised fine-tuning model based on the format of the prediction result, the number of recommended product data, the thought process text, and the candidate product data; According to the reward function value of the supervised fine-tuning model, the model parameters of the supervised fine-tuning model are iteratively updated to obtain a target product recommendation model.

[0011] In one embodiment, the reward function value includes at least a format reward value, a quantity reward value, a reasoning reward value, and an accuracy reward value; The step of determining the reward function value of the supervised fine-tuning model according to the format of the prediction result, the number of recommended product data, the thought process text, and the candidate product data includes: If the format in the prediction result satisfies the preset format, the format reward value is determined to be a first preset value; On the basis that the format reward value is the first preset value and the quantity of the recommended product data is within the preset quantity range, the quantity reward value is determined to be the second preset value; Based on the format reward value being a first preset value, determining a reasoning reward value according to the number of reasoning keywords in the thought process text and the rationality of the thought process text, wherein the number of reasoning keywords is proportional to the reasoning reward value, and the rationality is determined by a preset large language model; On the basis that the quantity reward value is a second preset value, an accuracy reward value is determined according to the recommended product data and the candidate product data.

[0012] In one embodiment, the step of determining an accuracy reward value based on the recommended product data and the candidate product data includes: If the recommended product data contains a recommended product that is not included in the candidate product data, the accuracy reward value is determined to be 0; If all the recommended products in the recommended product data are in the candidate product data, then determining the number of accurately recommended products in the recommended product data by using a preset large language model; An accuracy bonus value is calculated based on the number of accurately recommended products in the recommended product data and the number of recommended product data, wherein the greater the number of accurately recommended products, the higher the upper limit of the accuracy bonus value.

[0013] In one embodiment, before the step of obtaining original product data and user portrait data, the method further includes: Inputting product documents corresponding to a plurality of products into a preset large language model, and extracting product labels and product highlights of each product through the large language model; Convert the necessary tags and corresponding content in the product tags and product highlights of each product into long text descriptions; The long text description of each product is input into the large language model, and the large language model converts the long text description of each product into a corresponding short text description to obtain original product data.

[0014] In one embodiment, the user portrait data includes a plurality of first user portraits; Before the step of obtaining original product data and user portrait data, the method further includes: A plurality of preset user portrait generation prompt words are input into a preset large language model, and the large language model generates a plurality of first user portraits according to a preset ratio, wherein each first user portrait includes at least one type of user characteristic information.

[0015] In addition, the present application also provides a product recommendation method, which includes: The original product data and the user portrait data to be recommended are input into the target product recommendation model obtained by the product recommendation model training method as described above, and the target product recommendation model outputs the recommended product data and corresponding recommendation reasons corresponding to at least one user to be recommended in the user portrait data to be recommended.

[0016] In addition, the present application also provides a product recommendation model training device, which includes: a first screening module, configured to obtain original product data and user portrait data, and screen a first preset number of candidate product data for each first user portrait from the original product data based on user characteristic information of each first user portrait in the user portrait data; a second screening module, configured to match each of the first user profiles with a corresponding first preset number of candidate product data, to obtain a plurality of second user profiles that meet preset conditions and a corresponding second preset number of recommended product data, and to determine a recommendation reason for each of the recommended products, wherein the second preset number is less than the first preset number; A supervised fine-tuning module, configured to perform supervised fine-tuning training on a preset basic inference model based on each of the second user profiles and the recommended product data and recommendation reasons corresponding to each of the second user profiles, to obtain a supervised fine-tuning model; The reinforcement learning module is used to perform reinforcement learning training on the supervised fine-tuning model based on multiple third user portraits in each of the first user portraits that do not meet the preset conditions and multiple candidate product data corresponding to each of the third user portraits to obtain a target product recommendation model.

[0017] In addition, the present application also provides a product recommendation device, which includes at least: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program is configured to implement the steps of the product recommendation model training method applied to the product recommendation device as described above.

[0018] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the product recommendation model training method described above are implemented.

[0019] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, including a computer program, which implements the steps of the product recommendation model training method as described above when executed by a processor.

[0020] The present application provides a product recommendation model training method, which includes: first obtaining original product data and user portrait data, and based on the user characteristic information of each first user portrait in the user portrait data, screening out a first preset number of candidate product data for each first user portrait from the original product data, and then matching each first user portrait and the corresponding first preset number of candidate product data to match multiple second user portraits that meet the preset conditions and the corresponding second preset number of recommended product data, and determining the recommendation reasons for each recommended product data, wherein the second preset number is less than the first preset number; and then, based on each second user portrait and the recommended product data and recommendation reasons corresponding to each second user portrait, performing supervised fine-tuning training on the preset basic inference model to obtain a supervised fine-tuning model, and finally, based on the multiple third user portraits that do not meet the preset conditions in each first user portrait and the multiple candidate product data corresponding to each third user portrait, performing reinforcement learning training on the supervised fine-tuning model to obtain a target product recommendation model. In the technical solution of this application, the original product data and user profile data are first screened to select high-quality training data for supervised fine-tuning of small- and medium-scale basic inference models to obtain a supervised fine-tuning model. The low-quality training data that does not meet the preset conditions is further used to reinforce the supervised fine-tuning model, further improving the model's ability to accurately recommend products based on user profiles, thereby obtaining a target product recommendation model. Compared to traditional large language models, the target product recommendation model obtained through supervised fine-tuning and reinforcement learning has fewer parameters and requires less time and computing resources for product recommendation tasks. In addition, user profile data is input into the model in text form, which can capture the deep-seated needs expressed by the user profile data, so that the target product recommendation model can more efficiently output more accurate recommended product data and recommendation reasons. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 This is a flowchart of the product recommendation model training method in the embodiment of the present application; Figure 2This is a schematic diagram of the entire process from constructing training data to training a product recommendation model in an embodiment of the present application; Figure 3 A schematic diagram of a feasible process for constructing training data in an embodiment of the present application; Figure 4 This is a structural diagram of the product recommendation model training device in an embodiment of the present application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment of the device involved in the product recommendation model training method in the embodiment of the present application.

[0024] The purpose, features and advantages of this application will be further explained with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0025] To make the above-mentioned purposes, features, and advantages of the present application more clearly understood, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making any creative work are within the scope of protection of this application.

[0026] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0027] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0028] With the vigorous development of AI big model technology, the application of AI big models to product recommendations is becoming more and more widespread. At present, traditional recommendation solutions that rely on large models have defects such as heavy reliance on structured user portraits and inability to output recommendation reasons. Through research, the existing large language models represented by DeepSeek-R1 (DeepSeek) will have problems such as serious hallucinations and low accuracy in the product recommendation process. Especially in the case of text input, the large language model cannot grasp the key points of product recommendations. In addition, the reasoning process of using a large language model to perform product recommendation tasks requires a lot of time and computing resources. Therefore, there is an urgent need to train a small or medium-sized model with low hallucinations, high accuracy, and support for long-window product recommendations for product recommendations. Moreover, for different types of products, it is only necessary to train according to the same method as the embodiment of the present application to obtain other types of product recommendations.

[0029] In order to overcome the above technical defects and achieve the above beneficial effects, the present application embodiment provides a product recommendation model training method, referring to Figure 1 , Figure 1 This is a flow chart of an embodiment of the product recommendation model training method of the present application, which includes: Step S10: obtaining original product data and user portrait data, and filtering out a first preset number of candidate product data for each first user portrait from the original product data based on user characteristic information of each first user portrait in the user portrait data; The original product data includes various product data items, presented in text form. User profile data includes user characteristic information (such as gender, age, income, and address) for multiple real or virtual users. Each real or virtual user corresponds to a primary user profile, which is a user profile in the user profile data and represents user characteristic information in text form. User profile characteristics include, but are not limited to, age, gender, occupation, city, income, family situation, health status, and insurance company preferences. In specific applications, custom exclusion rules can also be set, for example, to exclude raw product data based on basic characteristics such as age and gender in the user profile data.

[0030] It should be noted that when screening the first preset number of candidate product data corresponding to each first user profile from the original product data, the primary consideration is the match between each product data and the user, for example, whether the product data is applicable to the user (or whether the user has permission to purchase the product corresponding to the product data). If applicable or authorized, a match is determined. Similarly, candidate product data for a first preset number (e.g., 70-90) of candidate products corresponding to each first user profile is determined, forming a candidate product list corresponding to each first user profile, which includes each candidate product data.

[0031] Furthermore, original product data and user portrait data can be extracted through a large language model, thereby obtaining the corresponding data more efficiently.

[0032] In one embodiment, before the step of obtaining original product data and user portrait data, the method may further include: Step A10: Inputting the product documents corresponding to the multiple products into a preset large language model, and extracting the product labels and product highlights of each product through the large language model; Step A20 , converting the necessary tags and corresponding content in the product tags and product highlights of each product into a long text description; In step A30 , the long text description of each product is input into the large language model, and the large language model converts the long text description of each product into a corresponding short text description to obtain the original product data.

[0033] Taking insurance products as an example, a large language model can first be used to extract the labels and product highlights of each insurance product from the uploaded insurance product documents. The insurance product documents can be in Word (text) format or PDF (Portable Document Format) format. Then, necessary product labels such as age requirements, gender requirements, regional restrictions, coverage premiums, insured amounts, and whether claims are troublesome are retained, and the necessary labels and corresponding content are combined into a long text description. The large language model is then used to transcribe the long text description into a short text product overview. In the prompt, the length of each insurance product overview is required to be controlled to approximately 300 tokens (character length units) without losing the main information of the product document, thereby obtaining the original product data. The large language model can be a QwQ-32B model, or a large language model such as DeepSeek-R1 and GPT-4o.

[0034] In one embodiment, the user portrait data includes a plurality of first user portraits; Before the step of obtaining original product data and user portrait data, the method may further include: Step B10: Input the preset multiple user portrait generation prompt words into the preset large language model, and the large language model generates multiple first user portraits according to a preset ratio, wherein each first user portrait includes at least one type of user characteristic information.

[0035] In addition, when acquiring user profile data, a large language model can be used to input multiple prompts so that the model mixes different user characteristic information in a certain proportion to randomly generate 10,000 diverse first user profiles. The large language model can be a model such as GPT-4o, QwQ-32B, or DeepSeek-R1. The user characteristic information of the first user profile can include, but is not limited to, age, gender, occupation, city, income, family situation, health status, insurance company preferences, and other information. The prompt requirement includes randomly missing one or more user profile information for some user profiles, such as age, gender, and income, to meet the diversity of different user profiles. Since it is impossible to collect a very comprehensive user profile in real scenarios, the simulation level of the user profile data is improved. In this embodiment of the present application, the text data of the first user profile is input into the model without the need for structural processing of the user profile (such as structuring the user profile into label data, such as age / income / occupation). This can capture the deep-seated needs expressed in the user profile data and solve the problem that the large language model cannot clearly identify the key information required for product recommendations when the input data is in text form. It also avoids the problems of traditional recommendation schemes that use simple rules to match labels in user structured data to recommend products, resulting in inaccurate product recommendations and inability to output recommendation reasons.

[0036] Furthermore, in a feasible embodiment, the step of selecting a first preset number of candidate product data corresponding to each first user profile from the original product data based on the user characteristic information of each first user profile in the user profile data may include: Step S11: based on the user characteristic information of each first user portrait, product data that does not match each first user portrait is removed from the original product data to obtain a plurality of product data corresponding to each first user portrait after removal; Step S12: randomly select a first preset number of product data from the multiple product data corresponding to each first user portrait after elimination to obtain candidate product data.

[0037] For example, each first user profile is judged using basic rules such as age and gender, and products that do not meet the requirements are first filtered out. For example, if the entry threshold for an insurance product is 60 years old or above, then this insurance product will not meet the requirements for the first user profile whose age information is under 60 years old.

[0038] After completing the first elimination step, a first preset number (e.g., 70-90) of candidate product data are randomly selected from the remaining products for each first user profile, forming a candidate product list corresponding to each first user profile. It should be noted that each candidate product data includes the product name, product overview, and product highlights, which are used for further recommendation and screening in subsequent steps.

[0039] In the embodiment of the present application, a preliminary screening of products is performed on each first user portrait based on the basic rules of user characteristic information, which can effectively reduce the amount of data for subsequent further screening and improve processing efficiency.

[0040] Step S20: Matching each first user profile and a corresponding first preset number of candidate product data to obtain a plurality of second user profiles that meet the preset conditions and a corresponding second preset number of recommended product data, and determining a recommendation reason for each recommended product data, wherein the second preset number is less than the first preset number; After obtaining multiple first user profiles and candidate product data corresponding to each first user profile, further screening is performed. Preset conditions must be applied during the screening process. These conditions may include the degree of match between the recommended product and the user profile exceeding a certain threshold, the recommended product data being relatively compatible with the user profile, and the recommendation being justified and highly credible. The goal is to obtain a second user profile with a higher degree of match and higher accuracy between the user profile and the product data, as well as a second preset number (e.g., 2-5) of recommended product data. When screening the second user profile from the first user profile, the process of determining whether the preset conditions are met can be performed using a large language model (such as DeepSeek-R1, GPT-4o, QwQ-32B, etc.).

[0041] In a feasible embodiment, the steps of matching each first user profile with a corresponding first preset number of candidate product data, matching multiple second user profiles that meet preset conditions and a corresponding second preset number of recommended product data, and determining a recommendation reason for each recommended product may include: Step S21: Input each first user profile and the candidate product data corresponding to each first user profile into a preset large language model, and input a first prompt word into the large language model to guide the large language model's thinking process and output a second preset number of recommended product data based on the corresponding candidate product data for each first user profile, and output a recommendation reason for each recommended product data; In step S22, the recommended product data or the candidate product data corresponding to the recommendation reasons that do not meet the preset conditions are filtered through the large language model to obtain multiple second user portraits and the corresponding recommended product data and recommendation reasons.

[0042] Specifically, each first user portrait and the candidate product data (short text product overview) in the product recommendation list corresponding to each first user portrait and the user characteristic information in the first user portrait can be input into a large language model (such as DeepSeek-R1, GPT-4o, QwQ-32B), and a first prompt word can be input. The first prompt word includes a matching requirement. The first prompt word is used to guide the large language model to match each first user portrait and the candidate product data (short text product overview) in the product recommendation list corresponding to each first user portrait and the user characteristic information in the first user portrait according to the matching requirements, and output a second preset number (for example, 2 to 5) of recommended product data with the relatively largest number of matching information. It is also necessary to output the thinking process and the reason for recommendation of each recommended product data. Among them, whether the thinking process is reasonable can be used as one of the criteria for subsequent filtering and screening.

[0043] For example, during matching using a large language model, the user's first profile, candidate products, a first prompt word, and matching requirements are input, and the output is the thought process, recommended product data, and corresponding recommendation reasons. The input matching requirements can include recommendation experience, which can be accumulated over time. In addition to the preset recommendation experience, the large model can also summarize recommendation experience during the thinking process. This learned recommendation experience can be used during model operation. After training, the model has rich recommendation experience and can output product recommendation data with a higher degree of matching.

[0044] It should be noted that, for ease of browsing, the first prompt word also includes format requirements for the model output text.

[0045] Taking insurance products as an example, the first prompt word Prompt in this embodiment is as follows: Prompt: """Please recommend 2 to 5 suitable products for this user based on the user portrait and insurance product list below.

[0046] ## Matching requirements (including recommended experience): 1. The recommended insurance product should be well-suited to the user's profile, including age, gender, income, job, and region. 2. Recommended insurance products must meet the user's financial conditions; 3. The reasons for the recommendation should be combined with the product features and user profile. They should be sufficient and detailed to explain why this product is suitable for this user. 4. Try to limit the number of recommended insurance products to 2-5, but don't stick to this limit; 5. When matching demand, recommendations should consider factors such as brand, price (premium), and the ease of claims processing. Given the same conditions, priority should be given to insurance products with established brands, lower prices, and easier claims processing. User preferences should also be considered. 6. The thinking process can first eliminate some unsuitable products, and then comprehensively consider and sort the remaining more suitable products.

[0047] ##Product List: {prod_list} ##User Profile: {port} ##Output format: Please output the recommendation results in the following JSON (a lightweight data exchange format) list format. [ {{"product": "Product 1", "reason": "Recommendation Reason 1"}}, {{"product": "Product 2", "reason": "Recommendation Reason 2"}} ...... ] """ The large language model outputs the recommendation result JSON data consisting of the thinking process, recommended product data, and corresponding recommendation reasons.

[0049] After the large language model preliminarily outputs the second preset number of recommended product data and recommendation reasons corresponding to each first user portrait (for example, 10,000), further screening is required to improve the instructions of the training sample data. In the embodiment of the present application, the large language model is continued to be used to further filter the output results. The large language model is used to analyze whether the recommended product data and recommendation reasons corresponding to the first user portrait meet the preset conditions (for example, whether they are reasonable or appropriate). The data corresponding to each first user portrait that does not meet the preset conditions is filtered out (a data includes the first user portrait, multiple recommended product data and recommendation reasons), and multiple filtered second user portraits and corresponding recommended product data and recommendation reasons are obtained (which can be regarded as high-quality sample data, such as 3,000).

[0050] Furthermore, in a feasible embodiment, the step of filtering the recommended product data or candidate product data corresponding to the recommendation reasons that do not meet the preset conditions using the large language model to obtain multiple second user profiles and the corresponding recommended product data and recommendation reasons may include: Step S221: input the second prompt word into the large language model to guide the large language model to filter candidate product data with unreasonable thinking process; Among them, the preset condition can be an unreasonable thinking process. An unreasonable thinking process means that the thinking process is not logical or does not conform to the common sense or conventions in the field. The large language model determines whether it is reasonable. The second prompt word is used to guide the large language model (such as DeepSeek-R1, GPT-4o, QwQ-32B model) to filter candidate product data with unreasonable thinking processes. For example, "Filter the unreasonable user portraits and corresponding recommended product data and recommendation reasons or thinking process texts in the recommendation reasons or thinking process texts output by the aforementioned large language model, and output the filtered data." In this embodiment, the prompt is only an example and does not impose any restrictions on the content of the second prompt word.

[0051] In step S222, for the filtered first user portrait, the third prompt word is input into the large language model to guide the large language model to filter the candidate product data with inappropriate recommendation reasons, thereby obtaining multiple second user portraits and corresponding recommended product data and recommendation reasons.

[0052] Among them, the preset condition may be that the recommendation reason is inappropriate, and the recommendation reason is inappropriate means that the recommended product data is inappropriate (or does not match) with the first user portrait. If there is an inappropriate recommended product data among the multiple recommended product data corresponding to the first user portrait, then all information of this training data is filtered (this data includes the first user portrait, multiple recommended product data and the corresponding recommendation reason).

[0053] The third prompt word is used to guide the large language model (such as DeepSeek-R1, GPT-4o, QwQ-32B model) to filter candidate product data with inappropriate recommendation reasons. For example, "filter the first user profile and corresponding recommended product data and recommendation reasons or reasoning thinking process text that are inappropriate or do not match the first user profile in the recommended product data output by the aforementioned large language model, and output the filtered data." In this embodiment, the prompt is only an example and does not impose any restrictions on the content of the third prompt word.

[0054] Exemplarily, the format of the aforementioned data is JSON format, and the input data includes: user portrait, multiple insurance product data; the output data includes: thinking process, recommended product data and corresponding recommendation reasons.

[0055] In combination with the contents of the aforementioned application embodiments, the present application embodiment uses the world prior knowledge of the large language model to complete the construction of training data in steps S10 to S20. In a feasible embodiment, the specific implementation process is as follows: Figure 3As shown, the method includes: first using a large language model to extract product labels and product highlights from product documents and converting them into long text descriptions; then using a large language model (such as DeepSeek-R1, GPT-4o, QwQ-32B model) to convert the long text description into a short text description to obtain multiple original product data (such as 1,000 original product data); at the same time, using the large language model to generate multiple first user portraits (such as 10,000 first user portraits) according to a preset ratio; based on the user feature information of the first user portrait, screening a first preset number of candidate product data (such as 70 to 90 candidate products) for each first user; and selecting the candidate product data by the large language model. For each first user portrait, the model outputs a second preset number (e.g., 2 to 5) of recommended product data from the corresponding candidate product data, and outputs the thinking process and recommendation reasons for each recommended product; the recommended product data that does not meet the preset conditions or the candidate product data corresponding to the recommendation reasons are filtered through the large language model to obtain multiple second user portraits that meet the conditions and the corresponding recommended product data, which are used as high-quality SFT training sample data for supervised fine-tuning training. For data that does not meet the conditions, a third preset number of third user portraits that do not meet the preset conditions and the corresponding candidate product data are screened as low-quality training sample data for reinforcement learning training. The large language model used in the process of constructing training data in this embodiment can be a large language model such as DeepSeek-R1, GPT-4o, or QwQ-32B.

[0056] Step S30: Based on each second user profile and the recommended product data and recommendation reasons corresponding to each second user profile, supervised fine-tuning training is performed on the preset basic reasoning model to obtain a supervised fine-tuning model; In an embodiment of the present application, each second user portrait and the recommended product data and recommendation reasons corresponding to each second user portrait can be regarded as high-quality training sample data. After obtaining the high-quality training sample data, supervised fine-tuning (SFT) training can be performed to obtain a supervised fine-tuning model (SFT model). The basic reasoning model used in the supervised fine-tuning training process can be a currently commonly used small and medium-sized reasoning model (such as Qwen2.5-7B-Instruct, a multi-language multi-modal reasoning model). Through the training of this application, a small and medium-sized model with low hallucination, high accuracy, and support for long-window text input can obtain a capability comparable to a large language model, but saves the time and computing resources required for reasoning using a large language model. Specifically, each second user portrait and the recommended product data corresponding to each second user portrait can be used as input data for model training, and the recommendation reasons can be used as label data for model training to measure model loss until the optimization process of model parameters.

[0057] In a feasible embodiment, the step of performing supervised fine-tuning training on a preset basic inference model based on each second user profile and the recommended product data and recommendation reasons corresponding to each second user profile to obtain a supervised fine-tuning model may include: Step S31: Each second user profile, the recommended product data and the recommendation reasons corresponding to each second user profile are used as training sample data, and are sequentially input into a preset basic reasoning model, and the basic reasoning model outputs a corresponding prediction result; Step S32: Calculate the loss function value of the basic reasoning model based on the recommended product data and recommendation reasons corresponding to each second user profile and the corresponding prediction results; Step S33: Based on the loss function value, the basic inference model is trained until a preset training round is reached to obtain a supervised fine-tuning model.

[0058] In the embodiment of the present application, the process of iteratively training the basic inference model is equivalent to using each second user portrait as input data, and the recommended product data and recommendation reasons corresponding to each second user portrait as supervised label data. During the training process, the loss function value can be a cross entropy loss or other forms of loss values.

[0059] The prediction results output by the basic inference model also include the predicted recommended product data and predicted recommendation reasons corresponding to each second user profile. By analyzing the differences between the predicted recommended product data and predicted recommendation reasons corresponding to each second user profile in the prediction results and the recommended product data and recommendation reasons determined for the second user profile in the aforementioned steps, the corresponding loss function value can be determined. The training optimization goal can be set to minimize the loss function value.

[0060] In addition, during the supervised fine-tuning of the basic inference model, some parameters of the basic inference model (for example, Qwen2.5-7B-Instruct) can be frozen in advance, and only the other adjustable parameters can be optimized and updated. This not only preserves the original performance of the basic inference model, but also achieves the effect of targeted optimization of processing capabilities in product recommendation scenarios.

[0061] In supervised fine-tuning (SFT) training, the preset training rounds (epochs) can be set to 3, where one epoch means that the model performs a complete cycle of forward propagation (calculating the predicted value) and backpropagation (updating parameters) for each sample in the training set.

[0062] Step S40 , based on multiple third user portraits in each first user portrait that do not meet the preset conditions and multiple candidate product data corresponding to each third user portrait, the supervised fine-tuning model is subjected to reinforcement learning training to obtain a target product recommendation model.

[0063] To further enhance the model's ability to process and recommend challenging sample data, in this embodiment, third user profiles (equivalent to low-quality training sample data) from among the first user profiles filtered out in step S20 that do not meet the preset criteria are used as sample data for reinforcement learning training, specifically improving the model's performance. The third user profiles are a subset of the first user profiles that do not meet the preset criteria. The selection method can be random, and the number of third user profiles can be pre-set. For example, the number of first user profiles that do not meet the preset criteria can be 7,000, and the number of third user profiles can be 3,000.

[0064] In the process of reinforcement learning of the supervised fine-tuning model, reward functions for multiple performance indicators are set, and the model parameters are iteratively optimized with the goal of increasing the reward function value. Finally, a target product recommendation model is obtained whose performance, stability and generalization ability meet the application requirements.

[0065] For example, the product recommendation model training process of the embodiment of the present application can be as follows: Figure 2 As shown, first, training sample data (for example, 10,000 items) are constructed, and a second preset number (for example, 3,000) of recommended product data in the training sample data are used as high-quality SFT training data (corresponding to the second user portrait and the recommended product data corresponding to the second user portrait respectively), which can be used to perform supervised fine-tuning training on small and medium-sized basic inference models. A third preset number (for example, 3,000) items are randomly selected from each third user portrait and the corresponding candidate product data as low-quality training data for further reinforcement learning training of the SFT model, and finally the target product recommendation model is obtained.

[0066] In a feasible embodiment, the step of performing reinforcement learning training on the supervised fine-tuning model based on multiple third user profiles that do not meet preset conditions in each first user profile and multiple candidate product data corresponding to each third user profile to obtain a target product recommendation model may include: Step S41: randomly selecting a third preset number of third user portraits that do not meet the preset conditions from the first user portraits, and obtaining a plurality of candidate product data corresponding to each third user portrait; The third preset number of third user profiles that do not meet the preset conditions refers to randomly selecting a third preset number (e.g., 3,000) of user profiles and candidate product training data from the low-quality data remaining after filtering in step S20 (e.g., the remaining 7,000 data failed to meet the criteria, reflecting the relatively difficult product recommendation process for these user profiles). Using this data for reinforcement learning allows the reinforcement learning training of the model to target more difficult problems, providing targeted training, which helps improve the generalization ability and recommendation accuracy of the product recommendation model.

[0067] Step S42: input each third user portrait into the supervised fine-tuning model in sequence, and the supervised fine-tuning model outputs a corresponding prediction result; Step S43, determining the reward function value of the supervised fine-tuning model based on the format of the prediction result, the number of recommended product data, the thought process text, and the candidate product data; Step S44: Iteratively update the model parameters of the supervised fine-tuning model according to the reward function value of the supervised fine-tuning model to obtain a target product recommendation model.

[0068] After obtaining the training sample data for reinforcement learning (including the third user profile and multiple candidate product data corresponding to each third user profile), rule reward methods and model recommendation reward methods can be used as the reward function of reinforcement learning.

[0069] Specifically, in reinforcement learning, the supervised fine-tuning model after STT training is used as the base model, and the GRPO (Group Relative Policy Optimization) optimization method is used for reinforcement learning training. After training for one epoch, the final target product recommendation model is obtained.

[0070] It should also be noted that after training the target product recommendation model, the model deployment process can be carried out. For example, when the product is an insurance product, the vLLM framework (a high-performance large language model inference framework) can be used to deploy the model. Enabling YaRN (a model deployment mode) can overcome the input window limitations of the original model while maintaining the model's performance and accuracy when processing long texts as much as possible. The target product recommendation model deployed in the embodiment of the present application can accommodate 128k token inputs, approximately 400 insurance product candidate information, greatly expanding the input window size. Based on a specific user profile, the target product recommendation model recommends several of the most suitable insurance products from thousands of insurance products (product data has been pre-organized) and outputs the corresponding recommendation reasons. In addition, because the training data of the target product recommendation model is constructed using the world prior knowledge of the large language model, the target product recommendation model has learned more rich and accurate recommendation experience, which can effectively improve the accuracy of recommendations.

[0071] In actual tests, the high-quality training data constructed using the large language model in the product recommendation model training method provided in the embodiment of the present application can effectively train a small or medium-sized insurance recommendation model (about 7B in size) and effectively improve the recommendation accuracy (93%), far exceeding the accuracy of the large language model DeepSeek-R1 (78%).

[0072] Furthermore, in a feasible embodiment, the reward function value includes at least a format reward value, a quantity reward value, a reasoning reward value, and an accuracy reward value; The step of determining the reward function value of the supervised fine-tuning model according to the format of the prediction result, the amount of recommended product data, the thought process text, and the candidate product data may include: Step S431, if the format in the prediction result meets the preset format, the format reward value is determined to be a first preset value; Step S432: If the format bonus value is the first preset value and the quantity of the recommended product data is within the preset quantity range, the quantity bonus value is determined to be the second preset value; Step S433: Based on the format reward value being a first preset value, a reasoning reward value is determined according to the number of reasoning keywords in the thought process text and the rationality of the thought process text, wherein the number of reasoning keywords is proportional to the reasoning reward value, and the rationality is determined by a preset large language model; Step S434 , determining an accuracy reward value based on the recommended product data and the candidate product data, on the basis that the quantity reward value is a second preset value.

[0073] In the embodiment of the present application, during the reinforcement learning of the supervised fine-tuning model, multiple factors such as format, number of model recommendations, reasoning thinking process, and recommendation accuracy are taken into consideration when constructing the reward function, and all aspects of the model's capabilities are studied more comprehensively, so that all aspects of the trained product recommendation model can meet the requirements, and the readability, logic, and accuracy of the output text such as product recommendation information and recommendation reasons are guaranteed.

[0074] Specifically, the primary calculation criterion for the reward function value is the format reward value. Only when the format is accurate can the reward values ​​of other dimensions be accurately calculated. Therefore, the format reward value is the basis for the reward values ​​of other dimensions.

[0075] For example, when the text output by the supervised fine-tuning model (including the thought process, product recommendation information, and recommendation reasons) meets the format "^\n.*?\n\n\n\[.*?\]\n$", the reward is 0.0001 points (the first preset value). This means that the format of the recommended thought process is placed inside the label, and the recommendation result is placed after it.

[0076] Furthermore, the process of determining the quantity reward value is based on the determined format reward. If the format reward is 0.0001 points and the answer is in JSON format and the number of recommended products is 2 to 5 (i.e., the preset quantity range), the reward is 0.0001 points (the second preset value).

[0077] Furthermore, the thought process text reflects the reasoning process of the supervised fine-tuning model when making product recommendations. The more reasoning keywords the recommendation thought process text contains, the stronger the logic. Reasoning keywords can include "first|next|then|finally|in addition|mean|maybe|take another look|comprehensively|therefore|therefore|in addition|infer|reason|finally|to sum up|check|eliminate|carefully" and so on.

[0078] For example, if the thinking process text contains one of the above reasoning keywords, it will be awarded 0.125 points, and if it contains more than four, the reasoning reward value will be the full score of 0.5 points.

[0079] In another feasible embodiment, when scoring the thinking process text, the number of summary keywords can also be considered. For example, if summary keywords such as "final recommendation" or "in summary" appear more than twice in the thinking process, it can be considered that the text is not concise and concise enough, and the reasoning reward value is determined to be 0 points.

[0080] Furthermore, when the number of characters in the thinking process text is greater than 1000 and the format reward value is not 0, the QwQ-32B model can be used to judge whether the thinking process is reasonable and evaluate its rationality. If it is reasonable, the reasoning reward value will be increased by 0.5 points.

[0081] When the quantity reward value is 0.0001, the accuracy reward value of the supervised fine-tuning model is further evaluated. Accuracy indicates the degree of accuracy of the output recommended product data. The higher the match between the recommended product data and the corresponding user profile, the higher the accuracy. In the embodiment of the present application, the candidate product data corresponding to each first user profile obtained in advance is used as a benchmark to evaluate the accuracy of each predicted recommended product data output by the supervised fine-tuning model. It can be understood that the higher the degree of overlap between the candidate product data corresponding to each first user profile and the predicted recommended product data output by the supervised fine-tuning model, the higher the accuracy.

[0082] Furthermore, in a feasible embodiment, the step of determining the accuracy reward value based on the recommended product data and the candidate product data may include: Step C10: if the recommended product data contains a recommended product that is not included in the candidate product data, the accuracy reward value is set to 0; Step C20: If all recommended products in the recommended product data are in the candidate product data, then the number of accurately recommended products in the recommended product data is determined by using a preset large language model; Step C30 , calculating an accuracy bonus value based on the number of accurately recommended products in the recommended product data and the number of recommended product data, wherein the greater the number of accurately recommended products, the higher the upper limit of the accuracy bonus value.

[0083] In the embodiment of the present application, the candidate product data corresponding to each first user portrait obtained in advance is used as a benchmark to perform a preliminary evaluation on the accuracy of each predicted recommended product data output by the supervised fine-tuning model. Among them, the candidate product data corresponding to each first user portrait generally includes 70 to 90 products. And each predicted recommended product data output by the supervised fine-tuning model generally includes 2 to 5 products. If the products included in each predicted recommended product data are all in the candidate product data, it means that the product recommendation accuracy of the supervised fine-tuning model is within the normal range. Otherwise, it means that the product recommendation of the supervised fine-tuning model is abnormal, and the accuracy reward value is determined to be 0.

[0084] If all products included in the predicted recommended product data are included in the candidate product data, the system further uses a preset large language model (e.g., the QwQ-32B model) to determine which products are accurately recommended in the recommended product data output by the supervised fine-tuning model and to determine the number of accurately recommended products. For example, the recommended product data output by the supervised fine-tuning model and the corresponding first user profile are input into the preset large language model, along with the corresponding prompt words (e.g., the input text content is analyzed to evaluate which recommended product data accurately matches the corresponding user profile, and to summarize the number of accurately recommended products).

[0085] In the process of determining the accuracy bonus value, the total amount of recommended product data and the number of accurately recommended products are comprehensively considered. It can be understood that when the amount of recommended product data is greater and all recommendations are accurate, the performance of the model is better, the corresponding accuracy bonus value is higher, and the upper limit is also higher.

[0086] For example, when the total number of recommended product data is 2, each accurate product recommended is +0.46 points, and the upper limit of the accuracy reward value is 0.92; when the total number of recommended product data is 3: each accurate product recommended is +0.32 points, and the upper limit of the accuracy reward value is 0.96; when the total number of recommended product data is 4, each accurate product recommended is +0.245 points, and the upper limit of the accuracy reward value is 0.98; when the total number of recommended product data is 5, each accurate product recommended is +0.2 points, and the upper limit of the accuracy reward value is 1.0.

[0087] In addition, an embodiment of the present application further provides a product recommendation method, which includes: Step D10: Input the original product data and the user portrait data to be recommended into the target product recommendation model obtained by the aforementioned product recommendation module training method, and the target product recommendation model outputs the recommended product data and corresponding recommendation reasons corresponding to at least one user to be recommended in the user portrait data to be recommended.

[0088] By using the aforementioned product recommendation model training method to perform low-cost supervised fine-tuning and reinforcement learning training on the basic inference model, a target product recommendation model with low hallucination and high recommendation accuracy is obtained. This small- to medium-sized model can be applied to the product recommendation workflow. Specifically, after the target product recommendation model is deployed, the original product data and the user profile data to be recommended are input into the target product recommendation model. The original product data is a short text product overview converted from the long text description of each product using a large language model. It contains all relevant product data, including tags, highlights, etc. The user profile data to be recommended includes user characteristic information of all available users. The target product recommendation model recommends multiple most matching recommended products for each user (corresponding to the recommended product data, for example, 2 to 5), and outputs the recommendation reasons for each recommended product for reference by staff in the product recommendation work, to evaluate whether the recommendation reasons of the recommended products are sufficient and reasonable, which can improve the accuracy of product recommendations.

[0089] In one embodiment, the target product recommendation model obtains matching requirements and, based on the matching requirements, specifically outputs recommended product data and a corresponding recommendation reason for at least one user to be recommended. Because the target product recommendation model has learned richer and more accurate recommendation experience during the training data construction process, the target product recommendation model has a higher accuracy rate in recommending products.

[0090] The present application also provides a product recommendation model training device, such as Figure 4 As shown, the product recommendation model training device at least includes: A first screening module 10 is configured to obtain original product data and user profile data, and to screen a first preset number of candidate product data for each first user profile from the original product data based on user characteristic information of each first user profile in the user profile data; A second screening module 20 is configured to match each of the first user profiles with a corresponding first preset number of candidate product data, to obtain a plurality of second user profiles that meet preset conditions and a corresponding second preset number of recommended product data, and to determine a recommendation reason for each of the recommended products, wherein the second preset number is less than the first preset number; A supervised fine-tuning module 30 is configured to perform supervised fine-tuning training on a preset basic inference model based on each of the second user profiles and the recommended product data and recommendation reasons corresponding to each of the second user profiles, to obtain a supervised fine-tuning model; The reinforcement learning module 40 is used to perform reinforcement learning training on the supervised fine-tuning model based on multiple third user portraits in each of the first user portraits that do not meet the preset conditions and multiple candidate product data corresponding to each of the third user portraits to obtain a target product recommendation model.

[0091] In one embodiment, the first screening module 10 is further configured to: Eliminate, based on the user characteristic information of each first user portrait, product data that does not match each first user portrait in the original product data, thereby obtaining a plurality of product data corresponding to each first user portrait after elimination; A first preset number of product data are randomly selected from the multiple product data corresponding to each first user portrait after elimination to obtain candidate product data.

[0092] In one embodiment, the second screening module 20 is further configured to: Inputting each of the first user profiles and the candidate product data corresponding to each of the first user profiles into a preset large language model, and inputting a first prompt word into the large language model, guiding the large language model to output a second preset number of recommended product data from the corresponding candidate product data for each of the first user profiles, and outputting the thinking process and the reason for recommending each of the recommended products; The large language model is used to filter the recommended product data or candidate product data corresponding to the recommendation reasons that do not meet the preset conditions, and obtain multiple second user portraits and corresponding recommended product data and recommendation reasons.

[0093] In one embodiment, the second screening module 20 is further configured to: Inputting the second prompt word into the large language model to guide the large language model to filter candidate product data with unreasonable thinking processes; For the filtered first user portrait, the third prompt word is input into the large language model to guide the large language model to filter the candidate product data with inappropriate recommendation reasons, thereby obtaining multiple second user portraits and corresponding recommended product data and recommendation reasons.

[0094] In one embodiment, the supervisory fine-tuning module 30 is further configured to: Each of the second user profiles, the recommended product data and the recommendation reasons corresponding to each of the second user profiles are used as training sample data, and are sequentially input into a preset basic reasoning model, and the basic reasoning model outputs a corresponding prediction result; Calculating the loss function value of the basic reasoning model based on the recommended product data and recommendation reasons corresponding to each second user profile and the corresponding prediction results; Based on the loss function value, the basic inference model is trained until a preset training round is reached to obtain a supervised fine-tuning model.

[0095] In one embodiment, the reinforcement learning module 40 is further configured to: Randomly selecting a third preset number of third user portraits that do not meet the preset conditions from each of the first user portraits, and obtaining a plurality of candidate product data corresponding to each of the third user portraits; Inputting each of the third user portraits into the supervised fine-tuning model in sequence, and having the supervised fine-tuning model output a corresponding prediction result; Determining a reward function value of the supervised fine-tuning model based on the format of the prediction result, the number of recommended product data, the thought process text, and the candidate product data; According to the reward function value of the supervised fine-tuning model, the model parameters of the supervised fine-tuning model are iteratively updated to obtain a target product recommendation model.

[0096] In one embodiment, the reward function value includes at least a format reward value, a quantity reward value, a reasoning reward value, and an accuracy reward value; the reinforcement learning module 40 is further configured to: If the format in the prediction result satisfies the preset format, the format reward value is determined to be a first preset value; On the basis that the format reward value is the first preset value and the quantity of the recommended product data is within the preset quantity range, the quantity reward value is determined to be the second preset value; Based on the format reward value being a first preset value, determining a reasoning reward value according to the number of reasoning keywords in the thought process text and the rationality of the thought process text, wherein the number of reasoning keywords is proportional to the reasoning reward value, and the rationality is determined by a preset large language model; On the basis that the quantity reward value is a second preset value, an accuracy reward value is determined according to the recommended product data and the candidate product data.

[0097] In one embodiment, the reinforcement learning module 40 is further configured to: If the recommended product data contains a recommended product that is not included in the candidate product data, the accuracy reward value is determined to be 0; If all the recommended products in the recommended product data are in the candidate product data, then determining the number of accurately recommended products in the recommended product data by using a preset large language model; An accuracy bonus value is calculated based on the number of accurately recommended products in the recommended product data and the number of the recommended product data, wherein the greater the number of the recommended product data, the higher the upper limit of the accuracy bonus value.

[0098] In one embodiment, the first screening module 10 is further configured to: Inputting product documents corresponding to a plurality of products into a preset large language model, and extracting product labels and product highlights of each product through the large language model; Convert the necessary tags and corresponding content in the product tags and product highlights of each product into long text descriptions; The long text description of each product is input into the large language model, and the large language model converts the long text description of each product into a corresponding short text description to obtain original product data.

[0099] In one embodiment, the user portrait data includes a plurality of first user portraits; Before the step of obtaining original product data and user portrait data, the method further includes: A plurality of preset user portrait generation prompt words are input into a preset large language model, and the large language model generates a plurality of first user portraits according to a preset ratio, wherein each first user portrait includes at least one type of user characteristic information.

[0100] The product recommendation model training device provided in the embodiments of the present application adopts the product recommendation model training method of the above-mentioned embodiments to obtain a small or medium-sized model with low hallucination and high recommendation accuracy through low-cost training, thereby improving the efficiency and accuracy of product recommendations. Compared with the prior art, the beneficial effects of the product recommendation model training device provided in the embodiments of the present application are the same as the beneficial effects of the product recommendation model training method provided in the above-mentioned embodiments, and the other technical features of the product recommendation model training device are the same as those disclosed in the above-mentioned embodiment method, and are not further described here.

[0101] An embodiment of the present application also provides a product recommendation device, which includes at least: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the product recommendation model training method of the above embodiment.

[0102] Reference below Figure 5 , which shows a structural schematic diagram of a product recommendation device suitable for implementing an embodiment of the present application. Figure 5 The product recommendation device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0103] like Figure 5 As shown, the product recommendation device may include a processing device 101 (e.g., a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 102 or programs loaded from a storage device 103 into a random access memory (RAM) 104. RAM 104 also stores various programs and data required for the operation of the product recommendation device. Processing device 101, ROM 102, and RAM 104 are interconnected via a bus 105. An input / output (I / O) interface 106 is also connected to the bus. Typically, the following systems may be connected to I / O interface 106: input devices 107, such as a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 108, such as a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 103, such as a magnetic tape or hard disk; and communication device 109. The communication device 109 can allow the product recommendation device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a product recommendation device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0104] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 103, or installed from a ROM 102. When the computer program is executed by the processing device 101, the above-mentioned functions defined in the method of the embodiment of the present application are performed.

[0105] The product recommendation device provided in the embodiment of the present application adopts the product recommendation model training method of the above embodiment to obtain a small or medium-sized model with low hallucination and high recommendation accuracy through low-cost training, thereby improving the efficiency and accuracy of product recommendations. Compared with the prior art, the beneficial effects of the product recommendation device provided in the embodiment of the present application are the same as the beneficial effects of the product recommendation model training method provided in the above embodiment, and the other technical features of the product recommendation device are the same as the features disclosed in the above embodiment method, and are not further described here.

[0106] It should be understood that the various parts of the embodiments of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any appropriate manner in any one or more embodiments or examples.

[0107] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the above claims.

[0108] An embodiment of the present application also provides a computer-readable storage medium storing a computer program that can be run on a processor, and the computer program is used to execute the product recommendation model training method in the above embodiment.

[0109] The computer-readable storage medium provided in the embodiments of the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0110] The computer-readable storage medium may be included in the product recommendation device, or may exist independently without being incorporated into the product recommendation device.

[0111] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the product recommendation device, the product recommendation device: obtains original product data and user portrait data, and according to the user characteristic information of each first user portrait in the user portrait data, screens out a first preset number of candidate product data for each first user portrait from the original product data; matches each first user portrait and the corresponding first preset number of candidate product data, matches multiple second user portraits that meet the preset conditions and the corresponding second preset number of recommended product data, and determines the recommendation reason for each recommended product, wherein the second preset number is less than the first preset number; based on each second user portrait and the recommended product data and recommendation reasons corresponding to each second user portrait, supervised fine-tuning training is performed on the preset basic inference model to obtain a supervised fine-tuning model; based on the multiple third user portraits that do not meet the preset conditions in each first user portrait and the multiple candidate product data corresponding to each third user portrait, reinforcement learning training is performed on the supervised fine-tuning model to obtain a target product recommendation model.

[0112] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0113] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0114] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0115] The computer-readable storage medium provided in the embodiments of this application stores computer-readable program instructions for executing the aforementioned product recommendation model training method. Through low-cost training, a small or medium-sized model with low hallucinations and high recommendation accuracy is obtained, thereby improving the efficiency and accuracy of product recommendations. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the embodiments of this application are the same as those of the product recommendation model training method provided in the aforementioned embodiments, and are not further elaborated here.

[0116] An embodiment of the present application also provides a computer program product, including a computer program, which implements the steps of the product recommendation model training method as described above when the computer program is executed by a processor.

[0117] The computer program product provided in the embodiments of this application uses low-cost training to produce a small or medium-sized model with low hallucinations and high recommendation accuracy, thereby improving the efficiency and accuracy of product recommendations. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiments of this application are the same as those of the product recommendation model training method provided in the above embodiments, and will not be elaborated here.

[0118] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A product recommendation model training method, characterized in that: The product recommendation model training method includes: Obtaining original product data and user portrait data, and filtering out a first preset number of candidate product data for each first user portrait from the original product data based on user characteristic information of each first user portrait in the user portrait data; Matching each of the first user portraits with a corresponding first preset number of candidate product data, matching multiple second user portraits that meet preset conditions and a corresponding second preset number of recommended product data, and determining a recommendation reason for each of the recommended products, wherein the second preset number is less than the first preset number; Based on each of the second user profiles and the recommended product data and recommendation reasons corresponding to each of the second user profiles, supervised fine-tuning training is performed on the preset basic inference model to obtain a supervised fine-tuning model; Based on multiple third user portraits that do not meet the preset conditions in each of the first user portraits and multiple candidate product data corresponding to each of the third user portraits, reinforcement learning training is performed on the supervised fine-tuning model to obtain a target product recommendation model.

2. The product recommendation model training method according to claim 1, characterized in that: The step of selecting a first preset number of candidate product data corresponding to each first user portrait from the original product data based on the user characteristic information of each first user portrait in the user portrait data includes: Eliminate, based on the user characteristic information of each first user portrait, product data that does not match each first user portrait in the original product data, thereby obtaining a plurality of product data corresponding to each first user portrait after elimination; A first preset number of product data are randomly selected from the multiple product data corresponding to each first user portrait after elimination to obtain candidate product data.

3. The product recommendation model training method according to claim 1, characterized in that: The steps of matching each of the first user portraits with the corresponding first preset number of candidate product data, matching multiple second user portraits that meet preset conditions and the corresponding second preset number of recommended product data, and determining the recommendation reason for each of the recommended products include: Inputting each of the first user profiles and the candidate product data corresponding to each of the first user profiles into a preset large language model, and inputting a first prompt word into the large language model, guiding the large language model to output, for each first user profile, a thought process based on the corresponding candidate product data, a second preset number of recommended product data, and a recommendation reason for each of the recommended products; The large language model is used to filter the recommended product data or candidate product data corresponding to the recommendation reasons that do not meet the preset conditions, and obtain multiple second user portraits and corresponding recommended product data and recommendation reasons.

4. The product recommendation model training method according to claim 3, wherein: The step of filtering the recommended product data or candidate product data corresponding to the recommendation reasons that do not meet the preset conditions by using the large language model to obtain multiple second user portraits and corresponding recommended product data and corresponding recommendation reasons includes: Inputting the second prompt word into the large language model to guide the large language model to filter candidate product data with unreasonable thinking processes; For the filtered first user portrait, the third prompt word is input into the large language model to guide the large language model to filter the candidate product data with inappropriate recommendation reasons, thereby obtaining multiple second user portraits and corresponding recommended product data and recommendation reasons.

5. The product recommendation model training method according to claim 1, wherein: The step of performing supervised fine-tuning training on a preset basic inference model based on each of the second user profiles and the recommended product data and recommendation reasons corresponding to each of the second user profiles to obtain a supervised fine-tuning model includes: Each of the second user profiles, the recommended product data and the recommendation reasons corresponding to each of the second user profiles are used as training sample data, and are sequentially input into a preset basic reasoning model, and the basic reasoning model outputs a corresponding prediction result; Calculating the loss function value of the basic reasoning model based on the recommended product data and recommendation reasons corresponding to each second user profile and the corresponding prediction results; Based on the loss function value, the basic inference model is trained until a preset training round is reached to obtain a supervised fine-tuning model.

6. The product recommendation model training method according to claim 1, wherein: The step of performing reinforcement learning training on the supervised fine-tuning model based on multiple third user profiles that do not meet the preset conditions in each of the first user profiles and multiple candidate product data corresponding to each of the third user profiles to obtain a target product recommendation model includes: Randomly selecting a third preset number of third user portraits that do not meet the preset conditions from each of the first user portraits, and obtaining a plurality of candidate product data corresponding to each of the third user portraits; Inputting each of the third user portraits into the supervised fine-tuning model in sequence, and having the supervised fine-tuning model output a corresponding prediction result; Determining a reward function value of the supervised fine-tuning model based on the format of the prediction result, the number of recommended product data, the thought process text, and the candidate product data; According to the reward function value of the supervised fine-tuning model, the model parameters of the supervised fine-tuning model are iteratively updated to obtain a target product recommendation model.

7. The product recommendation model training method according to claim 6, characterized in that: The reward function value includes at least a format reward value, a quantity reward value, a reasoning reward value, and an accuracy reward value; The step of determining the reward function value of the supervised fine-tuning model according to the format of the prediction result, the number of recommended product data, the thought process text, and the candidate product data includes: If the format in the prediction result satisfies the preset format, the format reward value is determined to be a first preset value; On the basis that the format reward value is the first preset value and the quantity of the recommended product data is within the preset quantity range, the quantity reward value is determined to be the second preset value; Based on the format reward value being a first preset value, determining a reasoning reward value according to the number of reasoning keywords in the thought process text and the rationality of the thought process text, wherein the number of reasoning keywords is proportional to the reasoning reward value, and the rationality is determined by a preset large language model; On the basis that the quantity reward value is a second preset value, an accuracy reward value is determined according to the recommended product data and the candidate product data.

8. The product recommendation model training method according to claim 7, characterized in that: The step of determining the accuracy reward value based on the recommended product data and the candidate product data includes: If the recommended product data contains a recommended product that is not included in the candidate product data, the accuracy reward value is determined to be 0; If all the recommended products in the recommended product data are in the candidate product data, then determining the number of accurately recommended products in the recommended product data by using a preset large language model; An accuracy bonus value is calculated based on the number of accurately recommended products in the recommended product data and the number of recommended product data, wherein the greater the number of accurately recommended products, the higher the upper limit of the accuracy bonus value.

9. The product recommendation model training method according to claim 1, wherein: Before the step of obtaining original product data and user portrait data, the method further includes: Inputting product documents corresponding to a plurality of products into a preset large language model, and extracting product labels and product highlights of each product through the large language model; Convert the necessary tags and corresponding content in the product tags and product highlights of each product into long text descriptions; The long text description of each product is input into the large language model, and the large language model converts the long text description of each product into a corresponding short text description to obtain original product data.

10. The product recommendation model training method according to claim 1, wherein: The user portrait data includes multiple first user portraits; Before the step of obtaining original product data and user portrait data, the method further includes: A plurality of preset user portrait generation prompt words are input into a preset large language model, and the large language model generates a plurality of first user portraits according to a preset ratio, wherein each first user portrait includes at least one type of user characteristic information.

11. A product recommendation method, characterized in that: The product recommendation method comprises: The original product data and the user portrait data to be recommended are input into the target product recommendation model obtained by the product recommendation model training method according to any one of claims 1 to 10, and the target product recommendation model outputs the recommended product data and the corresponding recommendation reason corresponding to at least one user to be recommended in the user portrait data to be recommended.

12. A product recommendation device, characterized in that: The product recommendation device includes at least: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the product recommendation model training method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program for implementing the product recommendation model training method, and the program for implementing the product recommendation model training method is executed by a processor to implement the steps of the product recommendation model training method as described in any one of claims 1 to 10.

Images