Package recommendation method and system based on multi-modal large model

Abstract

The invention relates to a package recommendation method and system based on a multi-modal large model, and the method comprises the steps: extracting the multi-modal features of a user from operator data, and carrying out the fusion of the multi-modal features of the user, so as to obtain a standardized fusion feature. And based on the similarity among different packages in the historical package data of the user, recalling a plurality of historical packages of which the similarity exceeds a first threshold value, so as to determine candidate package characteristics in combination with a preset service scene filtering rule. And calling an LLaMA3.1-8B large model to perform reasoning sorting based on the fusion features and the candidate package features so as to output a plurality of recommended packages of which the ranking is not lower than a second threshold value. And checking whether the plurality of recommended packages are located in a preset package list, and performing fine adjustment on the service scene filtering rule so as to determine the priority of each recommended package according to the fine-adjusted service scene filtering rule. According to the method, the recommended package is dynamically generated by fusing the multi-dimensional features, and accurate matching of package recommendation and user requirements is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, and particularly relates to a package recommendation method and system based on a multi-modal large model. BACKGROUND

[0002] With the continuous development of the communication industry, the types of packages launched by operator companies are increasingly rich, covering multiple forms such as traffic packages, voice packages, and fusion packages. Users face the problem of "selection overload" when choosing. At the same time, user demand presents diversified and personalized characteristics. Some users focus on traffic quotas, some users pay attention to voice duration, and some users have high demand for value-added services (such as video membership and cloud storage), which puts higher requirements on the accuracy of package recommendation.

[0003] Traditional package recommendation methods mostly rely on manual or simple rule matching, such as recommending similar packages based on user historical package types, recommending packages in corresponding price intervals according to user consumption amounts, etc. This method is difficult to capture dynamic changes in user demand, such as high-speed traffic demand generated by users changing 5G terminals or family fusion package demand generated by an increase in family members. The recommendation effect is limited and is prone to cause the problem of "homogeneous recommendation".

[0004] In recent years, recommendation algorithm technology has been widely applied in e-commerce, content platforms, and other fields. Multi-modal large models have significant advantages in accurate recommendation due to their ability to integrate and understand multiple types of data such as text, images, and behaviors. Applying multi-modal large models to operator package recommendation can integrate user multi-dimensional information and provide technical support for personalized recommendation. However, existing operator package recommendation systems, even if they introduce recommendation algorithms, are mostly limited to single-modal data and lack deep integration capabilities for multi-modal data. They also have weak adaptability to complex scenarios such as new packages, new users, and sudden changes in user demand, making it difficult to achieve truly accurate recommendation.

[0005] However, existing operator package recommendation systems rely on a single type of data (such as only based on user consumption amount or historical package handling records), and cannot integrate multi-dimensional information such as real-time user behavior data (such as night traffic usage peak) and value-added service preferences (such as video membership receipt records). For example, it is difficult to recommend a fusion package containing 5G high-speed traffic and video membership based on the comprehensive characteristics of users "frequently using 5G network and receiving video membership", resulting in low matching degree between recommendation and actual demand. In addition, in the prior art, the package recommendation system simply splices text and user behavior characteristics, without deeply mining the internal correlation between different modal data (such as the matching of user traffic usage peak period and the targeted traffic preferential period included in the package), and the feature fusion lacks semantic level collaboration, so that the value of multi-modal data cannot be fully utilized, and the recommendation accuracy is limited. For example, it is not possible to conduct deep correlation analysis between the behavior characteristic of users using video applications frequently at night to generate a large amount of traffic and the package text description containing night traffic preferential period. Finally, traditional recommendation models (such as shallow neural networks) have poor semantic understanding ability for multi-modal data, and it is difficult to quickly capture effective information in the face of newly launched packages (cold start scene), changes in user demand due to replacement of terminal equipment, and other complex situations, resulting in poor timeliness and adaptability of the final recommendation result. For example, it is difficult to quickly recommend a newly launched 5G fusion package to a user who has just replaced a 5G mobile phone.

[0006] In summary, due to the limitations of data dimension and model function, the existing package recommendation method is difficult to realize comprehensive analysis and recommendation of user required packages, resulting in poor matching degree between package recommendation result and user demand, and it is difficult to capture effective information in complex situations and difficult to realize the timeliness of matching package recommendation. SUMMARY

[0007] The present application aims to provide a package recommendation method and system based on a multi-modal large model to solve the problems existing in the prior art. The technical problems to be solved by the present application are solved by the following technical solutions.

[0008] The present application provides a package recommendation method based on a multi-modal large model, the method comprising: extracting user multi-modal features from operator data, and fusing the user multi-modal features to obtain standardized fused features; based on the similarity between different packages in the user's historical package data, recalling a plurality of historical packages with similarity exceeding a first threshold value to determine candidate package features in combination with a preset business scene filtering rule; calling a LLaMA3.1-8B large model to perform inference sorting based on the fused features and candidate package features to output a plurality of recommended packages with a ranking not lower than a second threshold value; checking whether the plurality of recommended packages are located in a preset package list, and fine-tuning the service scenario filtering rule to determine a priority of each recommended package according to the fine-tuned service scenario filtering rule; The user multi-modal feature is divided into a static feature, a behavior feature, and a semantic feature. The static feature is user information, number information, and package information. The behavior feature is data change in a package. The semantic feature is a summary text of a user handling a package.

[0009] In one of the embodiments, the user multi-modal feature is extracted from the operator data, and the user multi-modal feature is fused to obtain a standardized fusion feature, including: The static feature, the behavior feature, and the semantic feature are collected through a user data platform, and the static feature is encoded into a discrete vector through Embedding splicing; The SASRec recommendation model is called to generate a behavior embedding vector based on the behavior feature, and a large model is called to generate a text embedding vector corresponding to the semantic feature, so as to fuse the discrete vector, the behavior embedding vector, and the text embedding vector into the fusion feature.

[0010] In one of the embodiments, the similarity between different packages in the historical package data of the user is used to recall a plurality of historical packages with a similarity exceeding a first threshold, to determine candidate package features in combination with a preset service scenario filtering rule, including: The historical package data of the current user is obtained, and the similarity between different packages in the historical package data is calculated, to recall a plurality of historical packages with a similarity exceeding a first threshold in combination with package handling records in the historical package data; Based on the plurality of historical packages with the similarity exceeding the first threshold, an initial package candidate set is constructed, and candidate packages in the initial package candidate set are screened according to the preset service scenario filtering rule to obtain a secondary package candidate set.

[0011] In one of the embodiments, the similarity between different packages in the historical package data of the user is used to recall a plurality of historical packages with a similarity exceeding a first threshold, to determine candidate package features in combination with a preset service scenario filtering rule, further including: The candidate package features are extracted from the candidate packages in the secondary package candidate set. The candidate package features include package basic features, package semantic features, and package popularity features. The package basis feature is package information of the candidate package, including a package ID, a package price, a traffic quota, a call duration, and a package contract period; the package semantic feature is a text embedding vector of a package name and package content of the candidate package; and the package heat feature is a handling rate and a favorable rate of the candidate package.

[0012] In one of the embodiments, the LLaMA3.1-8B large model is called to perform inference ranking based on the fusion feature and the candidate package feature to output a plurality of recommended packages ranked no lower than a second threshold, including: The behavior embedding vector is mapped to a text embedding vector space through a Projector, and a hierarchical prompt dialogue box is constructed, which is divided into a first layer prompt dialogue box, a second layer prompt dialogue box, and a third layer prompt dialogue box; In the first layer prompt dialogue box, text data of package content is input, while the behavior feature is superimposed in the second layer prompt dialogue box, and the semantic feature is written in the third layer prompt dialogue box.

[0013] In one of the embodiments, the LLaMA3.1-8B large model is called to perform inference ranking based on the fusion feature and the candidate package feature to output a plurality of recommended packages ranked no lower than a second threshold, including: Based on the writing of the semantic feature in the third layer prompt dialogue box, the LLaMA3.1-8B large model is called to perform inference ranking on the candidate packages in the secondary package candidate set based on the third layer prompt dialogue box and the candidate package feature to obtain a ranking result; A plurality of candidate packages ranked no lower than a second threshold are selected from the ranking result to determine and output the plurality of recommended packages, and a recommended reason corresponding to each recommended package is output; The recommended reason is generated by the LLaMA3.1-8B large model based on the candidate package feature and a summary text of user package handling.

[0014] In one of the embodiments, the plurality of recommended packages are checked whether they are located in a preset package list, and the business scene filtering rule is fine-tuned to determine a priority of each recommended package according to the fine-tuned business scene filtering rule, including: The BERT model is called to calculate a cosine similarity between the recommended reason and the corresponding candidate package content to determine whether the plurality of recommended packages are located in the secondary package candidate set and the preset package list at the same time; Based on the priority of each recommended package, the recommended package and the recommended reason corresponding to the recommended package are output according to the priority size.

[0015] The application further provides a package recommendation system based on a multi-modal large model, which is used to implement the package recommendation method based on the multi-modal large model. The feature extraction and fusion module is used to extract user multi-modal features from the operator data and fuse the user multi-modal features to obtain standardized fused features. The package recall module is used to recall a plurality of historical packages with a similarity exceeding a first threshold based on the similarity between different packages in the historical package data of the user, and determine candidate package features in combination with a preset business scenario filtering rule. The package recommendation module is used to call the LLaMA3.1-8B large model to perform inference and sorting based on the fused features and the candidate package features, so as to output a plurality of recommended packages with a ranking not lower than a second threshold. The package verification and priority output module is used to verify whether the plurality of recommended packages are located in a preset package list, and fine-tune the business scenario filtering rule, so as to determine the priority of each recommended package according to the fine-tuned business scenario filtering rule. The user multi-modal features are divided into static features, behavior features and semantic features, the static features are user information, number information and package information, the behavior features are data changes in the package, and the semantic features are summary texts of the user handling the package.

[0016] The application further provides an electronic device including a memory and a processor, the memory stores a computer program, and the processor implements the package recommendation method based on the multi-modal large model when executing the computer program.

[0017] The application further provides a computer storage medium storing a computer program, and the computer program is executed by a processor to implement the package recommendation method based on the multi-modal large model.

[0018] The package recommendation method and system based on the multi-modal large model, by extracting user multi-modal features from operator data, fusing the user multi-modal features to obtain standardized fused features, recalling a plurality of historical package data with high similarity based on the similarity between different packages in the user's historical package data, determining candidate package features in combination with a preset business scenario filtering rule. Then, calling the LLaMA3.1-8B large model to infer and sort based on the extracted fused features and candidate package features to output a plurality of recommended packages with high ranking. Finally, verifying whether the plurality of recommended packages output are located in the preset package list, and fine-tuning the business scenario filtering rule to further screen the recommended packages according to the fine-tuned business scenario filtering rule, and then determining the priority of each recommended package, and recommending the appropriate recommended package to the user according to the priority order. The present application dynamically generates recommended packages by fusing multi-dimensional features, realizes accurate matching of package recommendation and user demand, and has high flexibility and real-time performance of recommended packages, improves the efficiency and satisfaction of users selecting recommended packages, and also reduces the operation and promotion cost. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0020] Figure 1 One of the flowcharts of the package recommendation method based on the multi-modal large model provided by the present application; Figure 2 The package recommendation flowchart of the package recommendation method based on the multi-modal large model in the specific embodiment provided by the present application; Figure 3 The package recall and embedded vector generation flowchart of the package recommendation method based on the multi-modal large model in the specific embodiment provided by the present application; Figure 4 The second flowchart of the package recommendation method based on the multi-modal large model provided by the present application; Figure 5 The third flowchart of the package recommendation method based on the multi-modal large model provided by the present application; Figure 6 The fourth flowchart of the package recommendation method based on the multi-modal large model provided by the present application; Figure 7 The fifth flowchart of the package recommendation method based on the multi-modal large model provided by the present application; Figure 8The sixth flowchart illustrates the multimodal large model-based package recommendation method provided by this invention. Figure 9 This is a schematic diagram of the package recommendation system based on a multimodal large model provided by the present invention; Figure 10 This is a diagram of the internal structure of the electronic device provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] The following is combined with Figures 1 to 10 This invention describes a method and system for recommending meal packages based on a multimodal large model.

[0023] like Figure 1 As shown, in one embodiment, a package recommendation method based on a multimodal large model includes the following steps: Step S110: Extract user multimodal features from operator data and fuse user multimodal features to obtain standardized fused features.

[0024] Among them, user multimodal features are divided into static features, behavioral features and semantic features. Static features include user information, number information and package information. Behavioral features include data changes within the package. Semantic features include summary text of the user's package application.

[0025] Specifically, static features, behavioral features, and semantic features of users regarding subscription packages are extracted from operator data, i.e., user multimodal features. The extracted user multimodal features are then fused in a unified format to obtain standardized fused features.

[0026] Combination Figure 2 and Figure 3As shown in the specific embodiment, the multimodal large model-based package recommendation method provided by the present invention is based on the core of integrating multimodal information such as user behavior and package semantics, and combining the collaborative filtering capability of traditional recommendation models with the semantic reasoning capability of large models to provide operators with more accurate package recommendations. It includes four core steps: user feature extraction, candidate package recall, multimodal recommendation decision, and package recommendation result optimization. Each step works in concert, that is, the user feature extraction step generates multi-dimensional user features, the candidate package recall step initially screens candidate packages, the multimodal recommendation decision step generates recommendation results based on multimodal fusion reasoning, and the package recommendation result optimization step calibrates the output, ultimately forming a complete recommendation chain of "feature-recall-decision-optimization".

[0027] Step 1, User Feature Extraction: This step is responsible for extracting multimodal features of users from operator data and generating a unified representation to provide a basis for recommending service plans.

[0028] Includes the following sub-steps: Sub-step 1.1: Multi-dimensional feature acquisition.

[0029] Specifically, the user data platform integrates three core types of data, including: (1) Static characteristics: user age, gender, location (e.g., “xx city xx district”), user level (A card / B card), package contract status (e.g., “online for xx years, currently on xx GB data package”).

[0030] (2) Behavioral characteristics: average data usage over the past 3 months (e.g., “18GB / month”), frequency of package changes (e.g., “no changes in the past six months”), records of value-added services (e.g., “once a video membership per month”), and peak data usage periods (e.g., “40% of data usage between 20:00 and 22:00”).

[0031] (3) Semantic features: User interest text refined through a general large model (such as: LLaMA3.1-8B large model). For example, “18GB / month data traffic, 2 years on the network, and have applied for a campus package” can be summarized as “high data traffic demand, preference for stable and cost-effective packages, and sensitivity to campus / youth exclusive rights and interests”.

[0032] Sub-step 1.2, Feature fusion and standardization.

[0033] Specifically, the embedding concatenation method is adopted to encode static features into discrete vectors, generate behavioral embeddings (embedded vectors) from behavioral features through traditional recommendation models (such as SASRec), generate text embeddings from semantic features through a large model, and finally integrate the three into a unified user multimodal embedding.

[0034] Sub-step 1.3, feature dynamic update.

[0035] Specifically, the user behavior data of the real-time synchronization data platform (such as "real-time monitoring of user daily traffic reaching 15GB") is updated once every 24 hours, and the semantic features are purified once every 7 days based on the latest behavior to ensure that the recommendation adapts to the change of user interest (such as "traffic usage increases during summer vacation, and the interest is updated to 'large traffic package demand'").

[0036] Step S120, based on the similarity between different packages in the user's historical package data, recalling a plurality of historical packages with similarity exceeding a first threshold to determine the candidate package features combined with the preset business scene filtering rules.

[0037] Specifically, based on the historical package data of the current user, the similarity between different packages handled by the user in the historical package data is calculated, and then a plurality of historical packages handled with similarity exceeding a set threshold (i.e. the first threshold) are recalled to determine the user's package preference, and the candidate package features are determined combined with the preset business scene filtering rules.

[0038] In combination Figure 2 and Figure 3 As shown in the specific embodiment, the package recommendation method based on the multi-modal large model provided by the application comprises the following steps:

[0039] Sub-step 2.1, fast recall based on ItemCF (item-based collaborative filtering).

[0040] Specifically, based on the historical package handling data, the similarity between packages is calculated (such as "among users who handle '10GB campus package', 60% of them subsequently upgrade to '20GB youth package', and the similarity between the two is set to 0.6"); combined with the user's historical handling record, "the top 20 most similar packages" are recalled as the initial candidate set (such as the user has handled "10GB package", and "20GB package" and "15GB + voice package" are recalled).

[0041] Sub-step 2.2, scene rule filtering.

[0042] Specifically, combined with the business scene, the filtering rules are added to further narrow down the candidate range, including: (1) Home adaptation: such as A city users are not recommended for local packages only for B city.

[0043] (2) User level adaptation: such as A card users prefer to keep high-end packages (such as "50GB + international roaming").

[0044] (3) Timeliness filtering: exclude offline or limited-time and limited-quantity packages (such as excluding "Spring Festival short-term traffic package").

[0045] Sub-step 2.3, candidate package feature preprocessing.

[0046] Specifically, a multi-modal representation is generated for the recalled candidate package, including: (1) Basic features: package ID, price, traffic quota, voice duration, and contract period.

[0047] (2) Semantic features: text Embedding of package name and description (such as "Campus 5G package: 30 yuan per month, including 20GB traffic + campus network access").

[0048] (3) Hotness features: 30-day handling rate and user praise rate (such as "handling rate 85%, praise rate 4.8 / 5").

[0049] Step S130, calling LLaMA3.1-8B large model to perform inference and ranking based on the fusion features and candidate package features to output multiple recommended packages with a ranking not lower than a second threshold.

[0050] It should be noted that the first threshold and the second threshold are both set thresholds, and "first threshold" and "second threshold" are used to distinguish that they are different set thresholds to meet different screening needs.

[0051] Specifically, calling LLaMA3.1-8B large model to perform inference and ranking based on the fusion features and candidate package features obtained in the preceding steps to output multiple candidate packages with a higher ranking (i.e., a ranking not lower than a second threshold) as recommended packages.

[0052] In combination with Figure 2 and Figure 3 shown, in specific embodiments, the package recommendation method based on a multi-modal large model provided by the present application includes the following sub-steps in step 3, multi-modal recommendation decision-making, which is the core of the system and is responsible for generating personalized recommendation results based on user multi-modal features and candidate package features through multi-modal large model inference. Includes the following sub-steps:

[0053] Sub-step 3.1, multi-modal fusion inference.

[0054] Specifically, the collaborative filtering signal of the traditional recommendation model is fused with the semantic inference ability of the large model, including: (1) Behavior modality: generate user behavior Embedding through SASRec model.

[0055] (2) Text modality: generate user interest text and package semantic Embedding through LLM.

[0056] (3) Modality Alignment: Project the behavior Embedding to the text Embedding space through the Projector, so that the large model can understand both "user behavior patterns" and "package semantics".

[0057] Sub-step 3.2, hierarchical Prompt engineering.

[0058] Specifically, a three-level Prompt is designed to gradually enhance the recommendation accuracy, including: (1) Text-Only Prompt (Text-only level Prompt): only input the package text description (such as "user 20 years old, candidate package: campus 5G package (20GB, 30 yuan); youth traffic package (15GB, 25 yuan)").

[0059] (2) Hybrid Prompt1 (Hybrid recommendation mechanism Prompt1): superimpose user behavior characteristics (such as "user's traffic usage in the past 3 months is 18GB, and the history handles 10GB package, candidate package: …").

[0060] (3) Hybrid Prompt2 (Hybrid recommendation mechanism Prompt2): add purified user interest (such as "user prefers'sufficient traffic and price ≤50 yuan', candidate package: …").

[0061] Sub-step 3.3, large model inference and sorting.

[0062] Specifically, LLaMA3.1-8B large model is used to infer and sort the candidate packages based on Hybrid Prompt2 (Hybrid recommendation mechanism Prompt2), output "the top 5 packages that the user is most likely to handle" and the corresponding recommendation reasons (such as "recommend campus 5G package: traffic quota matches user's 18GB usage demand, price meets young user's budget").

[0063] Step S140, verify whether the multiple recommended packages are in the preset package list, and fine-tune the business scene filtering rules to determine the priority of each recommended package according to the fine-tuned business scene filtering rules.

[0064] Specifically, after obtaining the recommended packages, it is also necessary to verify whether the output multiple recommended packages are in the preset package list (such as: on-sale package list) to prevent output of invalid packages, and according to the demand, fine-tune the business scene filtering rules, and then determine the priority of each recommended package according to the fine-tuned business scene filtering rules, so as to recommend the package to the user according to the priority.

[0065] In combination with Figure 2 and Figure 3As shown, in specific embodiments, the multi-modal large model-based package recommendation method provided by the present application includes the following steps: 1, data preprocessing; 2, candidate set construction; 3, candidate set filtering; 4, package recommendation result optimization; 5, package recommendation result packaging.

[0066] Substep 4.1, validity test.

[0067] Specifically, to solve the "invalid output" problem of large model generative recommendation, consistency check is performed, i.e., to confirm that the recommended package is in the candidate set and the list of packages on sale; similarity matching is also performed, i.e., to use BERT to encode the recommended reasons output by the large model and the candidate package text to calculate the cosine similarity.

[0068] Substep 4.2, secondary adjustment of business rules.

[0069] Specifically, the recommendation order is optimized in combination with the operation strategy, if there is a new package in the candidate set that has been online for less than one month, and the recommendation score meets the standard, the ranking is improved by 1-2 positions; if the package is associated with the operator's own interests (such as "contains xxx video membership"), and the user has used the interest in the past, the package is recommended preferentially.

[0070] Substep 4.3, recommended result packaging.

[0071] Specifically, user-understandable recommendation copy is generated, and multi-modal information is combined to enhance appeal, for example, "20GB package" is converted into "20GB traffic per month, enough for daily traffic consumption, no need to worry about traffic overage". At the same time, scene labels (such as "commuting traffic package" and "weekend xx exclusive") are added in combination with user behavior.

[0072] In actual application, the multi-modal large model-based package recommendation method provided by the present application first extracts historical data (user package handling records and traffic usage data in the past two years) and package basic data (full-amount package attributes and interest information) from the data center, and constructs a training set after cleaning. Secondly, the initial recall rules (such as package similarity calculation threshold 0.5), business filtering rules (such as home location adaptation), and optimized sorting adjustment strategies (such as "new package sorting weighting 0.1") are input by the operation team. In the model training process, the recall device ItemCF is first trained to narrow down the candidate package range, and then the LLM is trained through text prompt to ensure that the LLM is familiar with the basic idea of the recommendation mechanism.

[0073] In the multi-modal feature fusion process, the user behavior Embedding is obtained by training the SasRec model, the projector Projector is trained to align it with the package text Embedding, and the collaborative filtering signal is transmitted to the large model, so that the large model lacking collaborative filtering enhances the reasoning ability. At the same time, the LLM is fine-tuned with labeled data, and the LoRA light-weight training is used to reduce the cost.

[0074] In this embodiment, when the user initiates a recommendation request (such as logging in to the APP to view the package), the user's current features (such as "current traffic used 15GB (remaining 5 days this month)") are extracted in real time, and multi-modal Embedding is generated by combining historical features. Based on the "10GB package" that the user has handled in the past, similar packages (such as "20GB package" and "15GB + voice package") are recalled through ItemCF, and after filtering by scene rules, 10 candidate packages are retained. Subsequently, Hybrid Prompt2 is generated (such as "user 25 years old, xx city, near 3 months of traffic usage 18GB, prefers cost-effective packages. Candidate packages: campus 5G package (20GB, 30 yuan); youth traffic package (15GB, 25 yuan) …”), the large model is used for reasoning and ranking, and the top 5 recommended results are output. Check the effectiveness of the results, adjust the ranking (such as promoting the newly launched "20GB + member package" by 1 position), generate a semantic recommendation script, and finally display it on the user's APP home page.

[0075] In this embodiment, daily statistics of recommended key indicators (click-through rate, handling rate) are performed, and the operation team samples checks the recommended results (every 100), and labels "misrecommendation" (such as recommending a large traffic package to an old person) or "missed recommendation" (such as not recommending a suitable new package). The model is fine-tuned every week (such as updating the projector parameters and the LLM LoRA fine-tuning), and the recall rules are updated every month (such as adjusting the package similarity threshold), and the strategy is optimized (such as adding "holiday exclusive package" ranking weighting). The LLM summarizes discrete behaviors into interest text - "the user is a college student, with high traffic demand (nearly 18GB used in the past 3 months), and prefers cost-effective campus-related packages."

[0076] The package recommendation method based on the multi-modal large model extracts user multi-modal features from the operator data, fuses the user multi-modal features to obtain standardized fusion features, recalls a plurality of historical package data with high similarity based on the similarity between different packages in the historical package data of the user, determines candidate package features in combination with a preset business scenario filtering rule. Then, the LLaMA3.1-8B large model is called to perform inference and sorting based on the extracted fusion features and candidate package features to output a plurality of recommended packages with high ranking. Finally, it is verified whether the plurality of recommended packages output are located in the preset package list, and the business scenario filtering rule is fine-tuned to further screen the recommended packages according to the fine-tuned business scenario filtering rule, and then the priority of each recommended package is determined, and the appropriate recommended package is recommended to the user in order of priority. The method dynamically generates recommended packages by fusing multi-dimensional features, realizes accurate matching of package recommendation and user demand, has high flexibility and real-time performance of recommended packages, improves the efficiency and satisfaction of users selecting recommended packages, and also reduces the operation and promotion cost.

[0077] As shown in Figure 4 In one embodiment, the package recommendation method based on the multi-modal large model provided by the application specifically includes the following steps in step S110: In step S111, static features, behavior features and semantic features are collected through a user data platform, and the static features are encoded into discrete vectors through Embedding splicing.

[0078] In step S112, a SASRec recommendation model is called to generate a behavior embedding vector based on the behavior features, and a large model is called to generate a corresponding text embedding vector based on the semantic features, so as to fuse the discrete vector, the behavior embedding vector and the text embedding vector into a fusion feature.

[0079] As shown in Figure 5 In one embodiment, the package recommendation method based on the multi-modal large model provided by the application specifically includes the following steps in step S120: In step S121, the historical package data of the current user is obtained, and the similarity between different packages in the historical package data is calculated to recall a plurality of historical packages with similarity exceeding a first threshold in combination with the package handling records in the historical package data.

[0080] In step S122, based on the plurality of historical packages with similarity exceeding the first threshold, an initial package candidate set is constructed, and candidate packages in the initial package candidate set are screened according to a preset business scenario filtering rule to obtain a secondary package candidate set.

[0081] Step S123, extracting candidate package features from the candidate packages in the secondary package candidate set, the candidate package features including package basic features, package semantic features, and package popularity features.

[0082] The package basic features are package information of the candidate package, including a package ID, a package price, a traffic quota, a call duration, and a package contract period; the package semantic features are a text embedding vector of a package name and package content of the candidate package; and the package popularity features are a handling rate and a favorable rate of the candidate package.

[0083] As shown in the figure, Figure 6 In one embodiment, the package recommendation method based on the multi-modal large model provided by the application specifically includes the following steps of step S130: Step S131, mapping the behavior embedding vector to the text embedding vector space through the Projector projector, and constructing a hierarchical prompt dialogue box, the hierarchical prompt dialogue box being divided into a first layer prompt dialogue box, a second layer prompt dialogue box, and a third layer prompt dialogue box.

[0084] Step S132, inputting the text data of the package content in the first layer prompt dialogue box, superimposing the behavior features in the second layer prompt dialogue box, and writing the semantic features in the third layer prompt dialogue box.

[0085] As shown in the figure, Figure 7 In one embodiment, the package recommendation method based on the multi-modal large model provided by the application specifically includes the following steps of step S130: Step S133, based on writing the semantic features in the third layer prompt dialogue box, calling the LLaMA3.1-8B large model to combine the third layer prompt dialogue box to reason and sort the candidate packages in the secondary package candidate set according to the candidate package features, and obtaining a sorting result.

[0086] Step S134, selecting multiple candidate packages with a ranking not lower than a second threshold value from the sorting result to determine and output multiple recommended packages, and outputting a recommended reason corresponding to each recommended package.

[0087] The recommended reason is generated by the LLaMA3.1-8B large model based on the candidate package features and a summary text of the user handling the package.

[0088] As shown in the figure, Figure 8 In one embodiment, the package recommendation method based on the multi-modal large model provided by the application specifically includes the following steps of step S140: Step S141, calling the BERT model to calculate the cosine similarity between the recommended reason and the corresponding candidate package content to determine whether the multiple recommended packages are located in the secondary package candidate set and the preset package list at the same time.

[0089] Step S142: Based on the priority of each recommended package, output the recommended package and the corresponding recommendation reason according to the priority.

[0090] The following describes the multimodal large model-based package recommendation system provided by the present invention. The multimodal large model-based package recommendation system described below can be referred to in correspondence with the multimodal large model-based package recommendation method described above.

[0091] like Figure 9 As shown, in one embodiment, a package recommendation system based on a multimodal large model includes a feature extraction and fusion module 910, a package recall module 920, a package recommendation module 930, and a package verification and priority output module 940.

[0092] The feature extraction and fusion module 910 is used to extract user multimodal features from operator data and fuse user multimodal features to obtain standardized fused features.

[0093] The package recall module 920 is used to recall multiple historical packages whose similarity exceeds a first threshold based on the similarity between different packages in the user's historical package data, in order to determine the characteristics of candidate packages by combining preset business scenario filtering rules.

[0094] The package recommendation module 930 is used to call the LLaMA3.1-8B large model to perform inference and ranking based on fused features and candidate package features, so as to output multiple recommended packages with a ranking not lower than the second threshold.

[0095] The package verification and priority output module 940 is used to verify whether multiple recommended packages are in the preset package list and to fine-tune the business scenario filtering rules in order to determine the priority of each recommended package based on the fine-tuned business scenario filtering rules.

[0096] Among them, user multimodal features are divided into static features, behavioral features and semantic features. Static features include user information, number information and package information. Behavioral features include data changes within the package. Semantic features include summary text of the user's package application.

[0097] In this embodiment, the feature extraction and fusion module 910 of the multimodal large model-based package recommendation system provided by the present invention is specifically used for: Static features, behavioral features, and semantic features are collected through a user data platform, and the static features are encoded into discrete vectors by embedding.

[0098] The SASRec recommendation model is called to generate a behavior embedding vector based on the behavior features, and a text embedding vector is generated based on semantic features by a large model, so as to fuse the discrete vector, the behavior embedding vector and the text embedding vector into fusion features.

[0099] In the embodiment, the package recommendation system based on the multi-modal large model provided by the application is provided, and the package recall module 920 is specifically used for: The historical package data of the current user is obtained, and the similarity between different packages in the historical package data is calculated, so as to combine the package handling records in the historical package data, and recall a plurality of historical packages with a similarity exceeding a first threshold.

[0100] Based on the plurality of historical packages with the similarity exceeding the first threshold, an initial package candidate set is constructed, and candidate packages in the initial package candidate set are screened according to a preset business scenario filtering rule to obtain a secondary package candidate set.

[0101] In the embodiment, the package recommendation system based on the multi-modal large model provided by the application is provided, and the package recall module 920 is specifically used for: The candidate package features are extracted from the candidate packages in the secondary package candidate set, and the candidate package features include package basic features, package semantic features and package popularity features.

[0102] The package basic features are package information of the candidate package, including a package ID, a package price, a traffic quota, a call duration and a package contract period; the package semantic features are a text embedding vector of a package name and package content of the candidate package; and the package popularity features are a handling rate and a praise rate of the candidate package.

[0103] In the embodiment, the package recommendation system based on the multi-modal large model provided by the application is provided, and the package recommendation module 930 is specifically used for: The behavior embedding vector is mapped to a text embedding vector space by a Projector, and a hierarchical prompt dialog box is constructed, which is divided into a first layer prompt dialog box, a second layer prompt dialog box and a third layer prompt dialog box.

[0104] The text data of the package content is input in the first layer prompt dialog box, the behavior features are superimposed in the second layer prompt dialog box, and the semantic features are written in the third layer prompt dialog box.

[0105] In the embodiment, the package recommendation system based on the multi-modal large model provided by the application is provided, and the package recommendation module 930 is specifically used for: Based on writing semantic features in the third layer prompt dialog box, the LLaMA3.1-8B large model is called to combine the third layer prompt dialog box to reason and sort the candidate packages in the secondary package candidate set according to the candidate package features, and the sorting result is obtained.

[0106] From the sorting result, multiple candidate packages with a ranking not lower than the second threshold value are selected to determine and output multiple recommended packages, and the recommended reasons corresponding to each recommended package are output.

[0107] The recommended reasons are generated by the LLaMA3.1-8B large model based on the candidate package features and the summary text of the user handling the package.

[0108] In this embodiment, the package recommendation system based on the multi-modal large model provided by the application is provided, and the package verification and priority output module 940 is specifically used for: The cosine similarity between the recommended reasons and the corresponding candidate package content is calculated by calling the BERT model to determine whether the multiple recommended packages are located in the secondary package candidate set and the preset package list at the same time.

[0109] Based on the priority of each recommended package, the recommended packages and the recommended reasons corresponding to the recommended packages are output according to the priority size.

[0110] Figure 10 An example of an entity structure diagram of an electronic device is shown, which can be a smart terminal, and its internal structure diagram can be as shown in Figure 10 The electronic device includes a processor, a memory, and a network interface connected by a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the electronic device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement a package recommendation method based on a multi-modal large model, and the method includes:

[0111] Extracting user multi-modal features from operator data and fusing the user multi-modal features to obtain standardized fused features; Based on the similarity between different packages in the user's historical package data, multiple historical packages with a similarity exceeding a first threshold value are recalled to determine candidate package features in combination with a preset business scenario filtering rule; The LLaMA3.1-8B large model is called to reason and sort based on the fused features and the candidate package features to output multiple recommended packages with a ranking not lower than a second threshold value; checking whether the plurality of recommended packages are located in the preset package list, and fine-tuning the business scenario filtering rule to determine the priority of each recommended package according to the fine-tuned business scenario filtering rule; The user multi-modal features include static features, behavior features, and semantic features, the static features are user information, number information, and package information, the behavior features are data changes in the package, and the semantic features are summary texts of the user handling the package.

[0112] Those skilled in the art can understand that, Figure 10 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0113] In another aspect, the present application also provides a computer storage medium storing a computer program, the computer program being executed by a processor to implement a package recommendation method based on a multi-modal large model, the method comprising: extracting user multi-modal features from operator data, and fusing the user multi-modal features to obtain standardized fused features; based on the similarity between different packages in the historical package data of the user, recalling a plurality of historical packages with a similarity exceeding a first threshold value, to determine candidate package features in combination with a preset business scenario filtering rule; calling a LLaMA3.1-8B large model to perform inference ranking based on the fused features and the candidate package features, to output a plurality of recommended packages with a ranking not lower than a second threshold value; checking whether the plurality of recommended packages are located in the preset package list, and fine-tuning the business scenario filtering rule to determine the priority of each recommended package according to the fine-tuned business scenario filtering rule; The user multi-modal features include static features, behavior features, and semantic features, the static features are user information, number information, and package information, the behavior features are data changes in the package, and the semantic features are summary texts of the user handling the package.

[0114] In yet another aspect, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer readable storage medium. A processor of an electronic device reads the computer instructions from the computer readable storage medium, and the processor implements a package recommendation method based on a multi-modal large model when executing the computer instructions, the method comprising:

[0115] extracting user multi-modal features from operator data, and fusing the user multi-modal features to obtain standardized fused features; based on the similarity between different packages in the user's historical package data, recalling a plurality of historical packages with similarity exceeding a first threshold value to determine candidate package features in combination with preset business scenario filtering rules; calling an LLaMA3.1-8B large model to perform inference ranking based on the fusion features and the candidate package features to output a plurality of recommended packages with a ranking not lower than a second threshold value; checking whether the plurality of recommended packages are located in a preset package list, and fine-tuning the business scenario filtering rules to determine the priority of each recommended package according to the fine-tuned business scenario filtering rules; Among them, the user multi-modal feature is divided into static feature, behavior feature and semantic feature, the static feature is user information, number information and package information, the behavior feature is the change of data in the package, and the semantic feature is the summary text of the user handling the package.

[0116] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory.

[0117] As an illustration but not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM) and the like.

[0118] Each technical feature of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of technical features in the above embodiments are described. However, as long as the combination of technical features does not exist contradictory, it should be considered as the scope of the present disclosure.

[0119] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for package recommendation based on a multi-modal large model, characterized in that, The method comprises: extracting user multi-modal features from operator data and fusing the user multi-modal features to obtain standardized fused features; based on the similarity between different packages in the user's historical package data, recalling a plurality of historical packages with a similarity exceeding a first threshold, and determining candidate package features in combination with a preset business scenario filtering rule; calling an LLaMA3.1-8B large model to perform inference ranking based on the fused features and the candidate package features to output a plurality of recommended packages with a ranking not lower than a second threshold; verifying whether the plurality of recommended packages are located in a preset package list, and fine-tuning the business scenario filtering rule to determine the priority of each recommended package according to the fine-tuned business scenario filtering rule; wherein the user multi-modal features are divided into static features, behavior features and semantic features, the static features are user information, number information and package information, the behavior features are data changes within the package, and the semantic features are summary texts of the user's package. 2.The multi-modal large model-based package recommendation method of claim 1, wherein, The method comprises: collecting the static features, behavior features and semantic features through a user data platform, and encoding the static features into discrete vectors through Embedding splicing; calling an SASRec recommendation model to generate behavior embedding vectors based on the behavior features, and generating corresponding text embedding vectors based on the semantic features through a large model, so as to fuse the discrete vectors, behavior embedding vectors and text embedding vectors into the fused features. 3.The multi-modal large model-based package recommendation method of claim 2, wherein, The method comprises: obtaining historical package data of the current user, calculating the similarity between different packages in the historical package data, and recalling a plurality of historical packages with a similarity exceeding a first threshold in combination with package handling records in the historical package data; based on the plurality of historical packages with a similarity exceeding the first threshold, constructing an initial package candidate set, and screening candidate packages in the initial package candidate set according to the preset business scenario filtering rule to obtain a secondary package candidate set. 4.The multi-modal large model-based package recommendation method of claim 3, wherein, The method further comprises: extracting the candidate package features from the candidate packages in the secondary package candidate set, the candidate package features including package basic features, package semantic features and package popularity features; wherein the package basic features are package information of the candidate package, including package ID, package price, traffic quota, call duration and package contract period; the package semantic features are text embedding vectors of the package name and package content of the candidate package; and the package popularity features are handling rate and praise rate of the candidate package. 5.The multi-modal large model-based package recommendation method of claim 3, wherein, The calling LLaMA3.1-8B large model performs inference ranking based on the fused features and candidate package features to output a plurality of recommended packages ranked no lower than a second threshold, including: The behavior embedding vector is mapped to a text embedding vector space through a Projector projector, and a hierarchical prompt dialog box is constructed, which is divided into a first layer prompt dialog box, a second layer prompt dialog box, and a third layer prompt dialog box; Text data of package content is input in the first layer prompt dialog box, while the behavior features are superimposed in the second layer prompt dialog box, and the semantic features are written in the third layer prompt dialog box. 6.The multi-modal large model-based package recommendation method of claim 5, wherein, The calling LLaMA3.1-8B large model performs inference ranking based on the fused features and candidate package features to output a plurality of recommended packages ranked no lower than a second threshold, including: Based on the writing of the semantic features in the third layer prompt dialog box, the LLaMA3.1-8B large model is called to combine the third layer prompt dialog box to perform inference ranking on the candidate packages in the secondary package candidate set based on the candidate package features, to obtain a ranking result; From the ranking result, a plurality of candidate packages ranked no lower than a second threshold are selected to determine and output the plurality of recommended packages, and the recommended reasons corresponding to each recommended package are output; The recommended reasons are generated by the LLaMA3.1-8B large model based on the candidate package features and the summary text of the user's package. 7.The multi-modal large model-based package recommendation method according to claim 6, characterized in that, The verification of whether the plurality of recommended packages are located in the preset package list, and the fine-tuning of the business scenario filtering rule, to determine the priority of each recommended package according to the fine-tuned business scenario filtering rule, including: The BERT model is called to calculate the cosine similarity between the recommended reasons and the corresponding candidate package content to determine whether the plurality of recommended packages are located in the secondary package candidate set and the preset package list at the same time; Based on the priority of each recommended package, the recommended packages and the recommended reasons corresponding to the recommended packages are output in order of priority. 8.A package recommendation system based on a multi-modal large model, characterized in that, The system for implementing the package recommendation method based on a multi-modal large model according to any one of claims 1 to 7, comprising: A feature extraction and fusion module for extracting user multi-modal features from operator data and fusing the user multi-modal features to obtain standardized fused features; A package recall module for recalling a plurality of historical packages with a similarity exceeding a first threshold based on the similarity between different packages in the user's historical package data, and determining candidate package features in combination with a preset business scenario filtering rule; A package recommendation module for calling an LLaMA3.1-8B large model to perform inference ranking based on the fused features and candidate package features to output a plurality of recommended packages ranked no lower than a second threshold; A package verification and priority output module for verifying whether the plurality of recommended packages are located in the preset package list, and fine-tuning the business scenario filtering rule to determine the priority of each recommended package according to the fine-tuned business scenario filtering rule. The user multi-modal feature is divided into static features, behavior features and semantic features, the static features are user information, number information and package information, the behavior features are data changes in the package, and the semantic features are summary texts of the user handling the package. 9.An electronic device comprising a memory and a processor, the memory storing a computer program, wherein, The processor executes the computer program to implement the steps of the multi-modal large model-based package recommendation method in any one of claims 1 to 7.

10. A computer storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the multi-modal large model-based package recommendation method in any one of claims 1 to 7.

Images