A Package Recommendation Method and System Based on Multimodal Large Model

Abstract

This invention relates to a method and system for recommending service packages based on a multimodal large model. The method includes: extracting user multimodal features from operator data and fusing these features to obtain standardized fused features; recalling multiple historical packages with similarity exceeding a first threshold based on the similarity between different packages in the user's historical package data, and combining this with preset business scenario filtering rules to determine candidate package features; calling an LLaMA 3.1-8B large model to perform inference and ranking based on the fused features and candidate package features, and outputting multiple recommended packages with a ranking not lower than a second threshold; verifying whether the multiple recommended packages are in a preset package list, and fine-tuning the business scenario filtering rules to determine the priority of each recommended package based on the fine-tuned business scenario filtering rules. This invention achieves accurate matching between package recommendations and user needs by fusing multi-dimensional features and dynamically generating recommended packages.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a method and system for recommending service packages based on a multimodal large model. Background Technology

[0002] With the continuous development of the telecommunications industry, operators are offering an increasingly diverse range of service packages, including data packages, voice packages, and bundled packages, leading to "choice overload" for users. Meanwhile, user needs are becoming more diversified and personalized; some users prioritize data allowances, others focus on voice minutes, and still others have high demands for value-added services (such as video streaming and cloud storage). This places higher demands on the accuracy of package recommendations.

[0003] Traditional package recommendation methods mostly rely on manual intervention or simple rule matching, such as recommending similar packages based on the user's past subscriptions or recommending packages within a corresponding price range based on the user's spending amount. This approach struggles to capture dynamic changes in user needs, such as the high-speed data demands arising from switching to 5G devices or the need for family bundled packages due to an increase in family members. The recommendation effectiveness is limited, and it easily leads to the problem of "recommendation homogenization."

[0004] In recent years, recommendation algorithms have been widely used in e-commerce, content platforms, and other fields. Multimodal large models, with their ability to integrate and understand multiple types of data such as text, images, and behavior, have shown significant advantages in accurate recommendations. Applying multimodal large models to operator package recommendations can integrate multi-dimensional user information, providing technical support for personalized recommendations. However, even when recommendation algorithms are introduced, existing operator package recommendation systems are mostly limited to single-modal data, lacking the ability to deeply integrate multimodal data, and have weak adaptability to complex scenarios such as new packages, new users, and sudden changes in user needs, making it difficult to achieve truly accurate recommendations.

[0005] However, existing operator package recommendation systems largely rely on single-type data (such as user spending amounts or historical package purchase records), failing to integrate multi-dimensional information such as real-time user behavior data (e.g., peak nighttime data usage) and value-added service preferences (e.g., video membership purchase records). For example, it's difficult to recommend a bundled package including high-speed 5G data and video memberships based on the combined characteristics of a user who "frequently uses 5G networks and purchases video memberships," resulting in low matching between recommendations and actual needs. Furthermore, existing package recommendation systems that simply concatenate text with user behavior features do not delve into the inherent relationships between different modalities of data (e.g., the matching between peak user data usage times and the targeted data discount periods included in the package). Feature fusion lacks semantic synergy, preventing the full realization of the value of multimodal data and limiting improvements in recommendation accuracy. For instance, it cannot perform deep correlation analysis between the user's high-frequency use of video applications at night, generating significant data usage, and the textual description of packages including nighttime data discounts. Finally, traditional recommendation models (such as shallow neural networks) have poor semantic understanding of multimodal data. When faced with complex situations such as newly launched packages (cold start scenarios) and changes in user needs due to switching terminal devices, they struggle to quickly capture effective information, resulting in poor timeliness and adaptability of the final recommendation results. For example, newly launched 5G converged packages are difficult to recommend quickly to users who have just switched to 5G phones.

[0006] In summary, existing package recommendation methods, due to limitations in data dimensions and model functionality, struggle to comprehensively analyze and recommend packages to users, resulting in poor matching between recommended packages and user needs. Furthermore, they are less able to capture effective information in complex situations, making it difficult to achieve timely matching of package recommendations. Summary of the Invention

[0007] The present invention aims to provide a method and system for recommending packages based on a multimodal large model, so as to solve the shortcomings of the existing technology. The technical problem to be solved by the present invention is achieved through the following technical solution.

[0008] This invention provides a method for recommending meal packages based on a multimodal large model, the method comprising: Extract user multimodal features from operator data and fuse the user multimodal features to obtain standardized fused features; Based on the similarity between different packages in the user's historical package data, multiple historical packages with similarity exceeding the first threshold are recalled, and candidate package features are determined by combining preset business scenario filtering rules. The LLaMA3.1-8B large model is invoked to perform inference and ranking based on the fused features and candidate package features, so as to output multiple recommended packages with a ranking not lower than the second threshold. Verify whether the multiple recommended packages are in the preset package list, and fine-tune the business scenario filtering rules to determine the priority of each recommended package based on the fine-tuned business scenario filtering rules; The user multimodal features are divided into static features, behavioral features, and semantic features. The static features are user information, number information, and package information. The behavioral features are data changes within the package. The semantic features are summary text of the user's package application.

[0009] In one embodiment, the step of extracting user multimodal features from operator data and fusing the user multimodal features to obtain standardized fused features includes: The static features, behavioral features, and semantic features are collected through a user data platform, and the static features are encoded into discrete vectors by embedding. The SASRec recommendation model is invoked to generate behavioral embedding vectors based on the behavioral features, and a large model is used to generate corresponding text embedding vectors based on the semantic features, so as to fuse the discrete vectors, behavioral embedding vectors and text embedding vectors into the fused features.

[0010] In one embodiment, the step of recalling multiple historical packages with similarity exceeding a first threshold based on the similarity between different packages in the user's historical package data, and determining candidate package characteristics by combining preset business scenario filtering rules, includes: Obtain the current user's historical package data and calculate the similarity between different packages in the historical package data, so as to recall multiple historical packages with similarity exceeding a first threshold by combining the package application records in the historical package data. Based on multiple historical packages with similarity exceeding a first threshold, an initial package candidate set is constructed, and the candidate packages in the initial package candidate set are filtered according to the preset business scenario filtering rules to obtain a secondary package candidate set.

[0011] In one embodiment, the step of recalling multiple historical packages with similarity exceeding a first threshold based on the similarity between different packages in the user's historical package data, and determining candidate package features in conjunction with preset business scenario filtering rules, further includes: Extract the candidate package features from the candidate packages in the secondary package candidate set. The candidate package features include basic package features, semantic features of the package, and popularity features of the package. The basic features of the package are the package information of the candidate package, including package ID, package price, data allowance, call duration and package contract period; the semantic features of the package are the text embedding vectors of the package name and package content of the candidate package; and the popularity features of the package are the application rate and positive review rate of the candidate package.

[0012] In one embodiment, the invocation of the LLaMA3.1-8B large model performs inference and ranking based on the fused features and candidate package features to output multiple recommended packages with a ranking not lower than a second threshold, including: The behavior embedding vector is mapped to the text embedding vector space using the Projector, and a hierarchical prompt dialog box is constructed, which is divided into a first-level prompt dialog box, a second-level prompt dialog box, and a third-level prompt dialog box. Enter the text data of the package details in the first-level prompt dialog box, overlay the behavioral features in the second-level prompt dialog box, and write the semantic features in the third-level prompt dialog box.

[0013] In one embodiment, the step of calling the LLaMA3.1-8B large model to perform inference and ranking based on the fused features and candidate package features, so as to output multiple recommended packages with a ranking not lower than a second threshold, further includes: Based on the semantic features written in the third-level prompt dialog box, the LLaMA3.1-8B large model is called in conjunction with the third-level prompt dialog box to perform reasoning and sorting on the candidate packages in the secondary package candidate set according to the candidate package features, and the sorting result is obtained. From the ranking results, select multiple candidate packages with a ranking not lower than the second threshold, determine and output the multiple recommended packages, and output the recommendation reason for each recommended package; The recommendation reason is generated by the LLaMA3.1-8B big model based on the candidate package features and the summary text of the user's package application.

[0014] In one embodiment, the step of verifying whether the multiple recommended packages are in a preset package list and fine-tuning the business scenario filtering rules to determine the priority of each recommended package according to the fine-tuned business scenario filtering rules includes: The BERT model is called to calculate the cosine similarity between the recommendation reason and the corresponding candidate package content, so as to determine whether the multiple recommended packages are simultaneously located in the secondary package candidate set and the preset package list; Based on the priority of each recommended package, the recommended packages and the corresponding reasons for recommending them are output according to their priority.

[0015] This invention also provides a multimodal large model-based package recommendation system for implementing the multimodal large model-based package recommendation method described in any of the above claims, the system comprising: The feature extraction and fusion module is used to extract user multimodal features from operator data and fuse the user multimodal features to obtain standardized fused features. The package recall module is used to recall multiple historical packages whose similarity exceeds a first threshold based on the similarity between different packages in the user's historical package data, in order to determine the characteristics of candidate packages by combining preset business scenario filtering rules. The package recommendation module is used to call the LLaMA3.1-8B large model to perform inference and ranking based on the fused features and candidate package features, so as to output multiple recommended packages with a ranking not lower than the second threshold. The package verification and priority output module is used to verify whether the multiple recommended packages are in the preset package list and to fine-tune the business scenario filtering rules so as to determine the priority of each recommended package according to the fine-tuned business scenario filtering rules. The user multimodal features are divided into static features, behavioral features, and semantic features. The static features are user information, number information, and package information. The behavioral features are data changes within the package. The semantic features are summary text of the user's package application.

[0016] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the package recommendation method based on a multimodal large model as described above.

[0017] The present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the package recommendation method based on a multimodal large model as described above.

[0018] The aforementioned multimodal large-scale model-based package recommendation method and system extracts user multimodal features from operator data and fuses these features to obtain standardized fused features. Then, based on the similarity between different packages in the user's historical package data, it recalls multiple historical package data with high similarity, and combines this with preset business scenario filtering rules to determine candidate package features. Next, it calls the LLaMA 3.1-8B large-scale model to perform inference and ranking based on the extracted fused features and candidate package features, outputting multiple recommended packages with higher rankings. Finally, it verifies whether the output multiple recommended packages are in the preset package list and fine-tunes the business scenario filtering rules to further filter the recommended packages according to the fine-tuned rules, thereby determining the priority of each recommended package and recommending suitable packages to users in priority order. This invention achieves accurate matching between package recommendations and user needs by fusing multi-dimensional features and dynamically generating recommended packages. It also offers high flexibility and real-time performance in recommending packages, improving user efficiency and satisfaction in selecting recommended packages, and reducing operational and promotional costs. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is one of the flowcharts illustrating the multimodal large model-based package recommendation method provided by the present invention; Figure 2 This is a schematic diagram of the package recommendation process for a multimodal large model-based package recommendation method provided in a specific embodiment of the present invention; Figure 3 A schematic diagram illustrating the package recall and embedding vector generation process of the package recommendation method based on a multimodal large model in a specific embodiment of the present invention; Figure 4 The second flowchart illustrates the multimodal large model-based package recommendation method provided by this invention. Figure 5 The third flowchart illustrates the multimodal large model-based package recommendation method provided by this invention. Figure 6 The fourth flowchart illustrates the multimodal large model-based package recommendation method provided by this invention. Figure 7 The fifth flowchart illustrates the multimodal large model-based package recommendation method provided by this invention. Figure 8The sixth flowchart illustrates the multimodal large model-based package recommendation method provided by this invention. Figure 9 This is a schematic diagram of the package recommendation system based on a multimodal large model provided by the present invention; Figure 10 This is a diagram of the internal structure of the electronic device provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] The following is combined with Figures 1 to 10 This invention describes a method and system for recommending meal packages based on a multimodal large model.

[0023] like Figure 1 As shown, in one embodiment, a package recommendation method based on a multimodal large model includes the following steps: Step S110: Extract user multimodal features from operator data and fuse user multimodal features to obtain standardized fused features.

[0024] Among them, user multimodal features are divided into static features, behavioral features and semantic features. Static features include user information, number information and package information. Behavioral features include data changes within the package. Semantic features include summary text of the user's package application.

[0025] Specifically, static features, behavioral features, and semantic features of users regarding subscription packages are extracted from operator data, i.e., user multimodal features. The extracted user multimodal features are then fused in a unified format to obtain standardized fused features.

[0026] Combination Figure 2 and Figure 3As shown in the specific embodiment, the multimodal large model-based package recommendation method provided by the present invention is based on the core of integrating multimodal information such as user behavior and package semantics, and combining the collaborative filtering capability of traditional recommendation models with the semantic reasoning capability of large models to provide operators with more accurate package recommendations. It includes four core steps: user feature extraction, candidate package recall, multimodal recommendation decision, and package recommendation result optimization. Each step works in concert, that is, the user feature extraction step generates multi-dimensional user features, the candidate package recall step initially screens candidate packages, the multimodal recommendation decision step generates recommendation results based on multimodal fusion reasoning, and the package recommendation result optimization step calibrates the output, ultimately forming a complete recommendation chain of "feature-recall-decision-optimization".

[0027] Step 1, User Feature Extraction: This step is responsible for extracting multimodal features of users from operator data and generating a unified representation to provide a basis for recommending service plans.

[0028] Includes the following sub-steps: Sub-step 1.1: Multi-dimensional feature acquisition.

[0029] Specifically, the user data platform integrates three core types of data, including: (1) Static characteristics: user age, gender, location (e.g., “xx city xx district”), user level (A card / B card), package contract status (e.g., “online for xx years, currently on xx GB data package”).

[0030] (2) Behavioral characteristics: average data usage over the past 3 months (e.g., “18GB / month”), frequency of package changes (e.g., “no changes in the past six months”), records of value-added services (e.g., “once a video membership per month”), and peak data usage periods (e.g., “40% of data usage between 20:00 and 22:00”).

[0031] (3) Semantic features: User interest text refined through a general large model (such as: LLaMA3.1-8B large model). For example, “18GB / month data traffic, 2 years on the network, and have applied for a campus package” can be summarized as “high data traffic demand, preference for stable and cost-effective packages, and sensitivity to campus / youth exclusive rights and interests”.

[0032] Sub-step 1.2, Feature fusion and standardization.

[0033] Specifically, the embedding concatenation method is adopted to encode static features into discrete vectors, generate behavioral embeddings (embedded vectors) from behavioral features through traditional recommendation models (such as SASRec), generate text embeddings from semantic features through a large model, and finally integrate the three into a unified user multimodal embedding.

[0034] Sub-step 1.3: Dynamic feature update.

[0035] Specifically, user behavior data from the real-time synchronization data platform (e.g., "the user's daily data usage has reached 15GB") is updated every 24 hours; semantic features are refined every 7 days based on the latest behavior to ensure that recommendations adapt to changes in user interests (e.g., "data usage increased during the summer vacation, so the interest is updated to 'high data plan demand'").

[0036] Step S120: Based on the similarity between different packages in the user's historical package data, recall multiple historical packages with similarity exceeding a first threshold, and combine them with preset business scenario filtering rules to determine the characteristics of candidate packages.

[0037] Specifically, based on the current user's historical package data, the similarity between different packages that the user has subscribed to in the historical package data is calculated, and then multiple historical packages that have been subscribed to with a similarity exceeding a set threshold (i.e., the first threshold) are recalled to determine the user's package preferences. Then, the characteristics of candidate packages are determined by combining the preset business scenario filtering rules.

[0038] Combination Figure 2 and Figure 3 As shown in the specific embodiment, the multimodal large model-based package recommendation method provided by the present invention includes the following sub-steps in step 2, candidate package recall:

[0039] Sub-step 2.1, fast recall based on ItemCF (item-based collaborative filtering).

[0040] Specifically, based on historical subscription data, the similarity between subscription packages is calculated (e.g., "among users who subscribe to the '10GB Campus Package,' 60% later upgrade to the '20GB Youth Package,' so the similarity between the two is set to 0.6"); combined with users' historical subscription records, the "top 20 most similar packages" are recalled as the initial candidate set (e.g., if a user has subscribed to the "10GB Package," the "20GB Package" and "15GB + Voice Package" are recalled).

[0041] Sub-step 2.2, scenario-based rule filtering.

[0042] Specifically, filter rules can be added based on business scenarios to further narrow down the candidate pool, including: (1) Location adaptation: For example, local packages that are only available to users in City B are not recommended for users in City A.

[0043] (2) User level adaptation: For example, A card users are given priority to retain high-end packages (such as "50GB + international roaming").

[0044] (3) Time-sensitive filtering: Remove offline packages or time-limited packages (such as excluding "Spring Festival short-term data packages").

[0045] Sub-step 2.3: Preprocessing of candidate package features.

[0046] Specifically, multimodal representations are generated for the candidate packages to be recalled, including: (1) Basic features: Package ID, price, data allowance, voice call duration and contract period.

[0047] (2) Semantic features: Text embedding of package name and description (e.g., “Campus 5G Package: 30 yuan per month, including 20GB of traffic + campus network access”).

[0048] (3) Popularity characteristics: the processing rate and user satisfaction rate in the past 30 days (e.g., "processing rate 85%, satisfaction rate 4.8 / 5").

[0049] Step S130: Call the LLaMA3.1-8B large model to perform inference and ranking based on fusion features and candidate package features, so as to output multiple recommended packages with a ranking not lower than the second threshold.

[0050] It should be noted that both the first threshold and the second threshold are set thresholds. The terms "first threshold" and "second threshold" are used to distinguish them as different set thresholds to meet different filtering needs.

[0051] Specifically, the LLaMA3.1-8B large model is invoked to perform inference and ranking based on the fusion features and candidate package features obtained in the aforementioned steps, and to output multiple candidate packages with higher rankings (i.e., rankings not lower than the second threshold) as recommended packages.

[0052] Combination Figure 2 and Figure 3 As shown in the specific embodiment, the multimodal large model-based package recommendation method provided by the present invention includes step 3, multimodal recommendation decision. As the core of the system, it is responsible for generating personalized recommendation results based on user multimodal features and candidate package features through multimodal large model inference. This includes the following sub-steps:

[0053] Sub-step 3.1, multimodal fusion reasoning.

[0054] Specifically, it integrates the collaborative filtering signals of traditional recommendation models with the semantic reasoning capabilities of large models, including: (1) Behavioral modality: User behavior embedding is generated through the SASRec model.

[0055] (2) Text modality: User interest text and package semantic embedding are generated through LLM.

[0056] (3) Modal alignment: The behavior embedding is mapped to the text embedding space through the projector, so that the large model can understand both "user behavior pattern" and "package semantics" at the same time.

[0057] Sub-step 3.2, Layered Prompt Project.

[0058] Specifically, a three-level Prompt design is implemented to progressively enhance recommendation accuracy, including: (1) Text-Only Prompt: Only enter the text description of the package (e.g., "User is 20 years old, candidate packages: Campus 5G Package (20GB, 30 yuan); Youth Data Package (15GB, 25 yuan)").

[0059] (2) Hybrid Prompt1: Overlaying user behavior features (such as "User has used 18GB of data in the past 3 months, has previously subscribed to a 10GB data plan, candidate plans: ...").

[0060] (3) Hybrid Prompt2: Add refined user interests (e.g., “Users prefer ‘sufficient data and price ≤ 50 yuan’, candidate packages: …”).

[0061] Sub-step 3.3, Large Model Reasoning and Sorting.

[0062] Specifically, the LLaMA3.1-8B large model is used, and the candidate packages are inferred and ranked based on Hybrid Prompt2 (a hybrid recommendation mechanism), outputting the "top 5 packages that users are most likely to subscribe to" and the corresponding reasons for recommendation (such as "Recommended campus 5G package: the data allowance matches the user's 18GB usage needs, and the price is in line with the budget of young users").

[0063] Step S140: Verify whether multiple recommended packages are in the preset package list, and fine-tune the business scenario filtering rules to determine the priority of each recommended package based on the fine-tuned business scenario filtering rules.

[0064] Specifically, after obtaining the recommended packages, it is necessary to verify whether the multiple recommended packages are in the preset package list (e.g., the list of packages for sale) to prevent the output of invalid packages. At the same time, the business scenario filtering rules are fine-tuned according to the requirements, and the priority of each recommended package is determined according to the fine-tuned business scenario filtering rules, so as to recommend packages to users according to the priority.

[0065] Combination Figure 2 and Figure 3As shown in the specific embodiment, the multimodal large model-based package recommendation method provided by the present invention includes step 4, package recommendation result optimization. This step is responsible for calibrating the recommended package decision output to ensure that the package recommendation results comply with the operator's business rules, thereby improving user experience. It includes the following sub-steps:

[0066] Sub-step 4.1, validity check.

[0067] Specifically, the solution to the "invalid output" problem in large-scale generative recommendations includes consistency checks, which confirm that all recommended packages are in the candidate set and the list of packages on sale; it also includes similarity matching, which uses BERT to encode the recommendation reasons output by the large model and the candidate package text, and calculates the cosine similarity.

[0068] Sub-step 4.2: Second adjustment of business rules.

[0069] Specifically, the recommendation order will be optimized in conjunction with the operational strategy. If there are new packages launched less than one month ago in the candidate pool and the recommendation score meets the standard, the package will be promoted by 1-2 places. If the package is associated with operator-owned benefits (such as "includes xxx video membership") and the user has used the benefits in the past, it will be recommended first.

[0070] Sub-step 4.3 recommends packaging the results.

[0071] Specifically, generate user-understandable recommendation copy and enhance appeal by incorporating multimodal information. For example, transform "20GB package" into "20GB of data per month, enough for daily data consumption, no need to worry about exceeding the data limit." At the same time, add scenario tags based on user behavior (such as "commuter data package" or "weekend xx exclusive").

[0072] In practical applications, the multimodal large model-based package recommendation method provided by this invention first extracts historical data (user package application records and data usage data from the past two years) and basic package data (all package attributes and benefits information) from a data platform, and then cleans and constructs a training set. Secondly, the operations team inputs initial recall rules (such as a package similarity calculation threshold of 0.5), business filtering rules (such as location matching), and optimized ranking adjustment strategies (such as "new packages are ranked with a weighting of 0.1"). During model training, the recall engine ItemCF is trained first to narrow down the candidate package range. Then, the LLM is trained using text prompts to ensure that the LLM is familiar with the basic ideas of the recommendation mechanism.

[0073] In the multimodal feature fusion process, user behavior embeddings are obtained by training the SasRec model, and the projector is trained to align these embeddings with the package text embeddings. The collaborative filtering signal is then passed to the large model, enhancing its inference capabilities, which previously lacked collaborative filtering. Simultaneously, the LLM is fine-tuned using labeled data, and LoRA lightweight training is employed to reduce costs.

[0074] In this embodiment, when a user initiates a recommendation request (such as logging into the app to view data plans), the user's current features (such as "15GB of data has been used (5 days remaining in the month)") are extracted in real time and combined with historical features to generate a multimodal embedding. Based on the user's historical "10GB data plan", similar data plans (such as "20GB data plan" and "15GB + voice data plan") are recalled through ItemCF, and 10 candidate data plans are retained after filtering by scenario rules. Subsequently, a Hybrid Prompt2 is generated (such as "User is 25 years old, from xx city, has used 18GB of data in the past 3 months, and prefers cost-effective data plans. Candidate data plans: Campus 5G data plan (20GB, 30 yuan); Youth data package (15GB, 25 yuan)..."), and the top 5 recommendation results are output through large model inference and sorting. The validity of the results is verified, the sorting is adjusted (such as promoting the newly launched "20GB + membership data plan" by 1 position), semantic recommendation copy is generated, and finally displayed on the user's app homepage.

[0075] In this embodiment, key recommendation metrics (click-through rate, processing rate) are statistically analyzed daily. The operations team samples and checks the recommendation results (every 100 items), marking cases of "misrecommendation" (e.g., recommending high-data plans to the elderly) or "missed recommendation" (e.g., not recommending suitable new plans). The model is fine-tuned weekly using quality inspection data (e.g., updating projector parameters, fine-tuning the LoRA of the large model), and recall rules (e.g., adjusting the plan similarity threshold) and optimization strategies (e.g., adding a weighted ranking for "holiday-exclusive plans") are updated monthly. LLM summarizes discrete behaviors into interest text—"The user is a college student with high data usage (nearly 18GB used in the past 3 months) and prefers high-value campus-related plans."

[0076] The aforementioned multimodal large-scale model-based package recommendation method extracts user multimodal features from operator data and fuses these features to obtain standardized fused features. Then, based on the similarity between different packages in the user's historical package data, it recalls multiple historical package data with high similarity, and combines this with preset business scenario filtering rules to determine candidate package features. Next, it calls the LLaMA 3.1-8B large-scale model to perform inference and ranking based on the extracted fused features and candidate package features, outputting multiple recommended packages with higher rankings. Finally, it verifies whether the output multiple recommended packages are in the preset package list and fine-tunes the business scenario filtering rules to further filter the recommended packages according to the fine-tuned rules, thereby determining the priority of each recommended package and recommending suitable packages to users in priority order. This method, by fusing multi-dimensional features and dynamically generating recommended packages, achieves accurate matching between package recommendations and user needs, and offers high flexibility and real-time performance in package recommendations, improving user efficiency and satisfaction in selecting recommended packages, while also reducing operational and promotional costs.

[0077] like Figure 4 As shown, in one embodiment, the package recommendation method based on a multimodal large model provided by the present invention includes the following steps in step S110: Step S111: Collect static features, behavioral features, and semantic features through the user data platform, and encode the static features into discrete vectors by embedding.

[0078] Step S112: Call the SASRec recommendation model to generate behavioral embedding vectors based on behavioral features, and use the large model to generate corresponding text embedding vectors based on semantic features, so as to fuse discrete vectors, behavioral embedding vectors and text embedding vectors into fused features.

[0079] like Figure 5 As shown, in one embodiment, the package recommendation method based on a multimodal large model provided by the present invention includes the following steps in step S120: Step S121: Obtain the current user's historical package data and calculate the similarity between different packages in the historical package data, so as to recall multiple historical packages with similarity exceeding the first threshold by combining the package application records in the historical package data.

[0080] Step S122: Based on multiple historical packages with similarity exceeding the first threshold, construct an initial package candidate set, and filter the candidate packages in the initial package candidate set according to the preset business scenario filtering rules to obtain a secondary package candidate set.

[0081] Step S123: Extract candidate package features from the candidate packages in the secondary package candidate set. The candidate package features include basic package features, semantic features of the package, and popularity features of the package.

[0082] Among them, the basic features of the package are the package information of the candidate package, including package ID, package price, data allowance, call duration and package contract period; the semantic features of the package are the text embedding vectors of the package name and package content of the candidate package; the popularity features of the package are the application rate and positive review rate of the candidate package.

[0083] like Figure 6 As shown, in one embodiment, the package recommendation method based on a multimodal large model provided by the present invention includes the following steps in step S130: Step S131: The behavior embedding vector is mapped to the text embedding vector space through the Projector, and a hierarchical prompt dialog box is constructed. The hierarchical prompt dialog box is divided into a first-level prompt dialog box, a second-level prompt dialog box, and a third-level prompt dialog box.

[0084] Step S132: Enter the text data of the package content in the first-level prompt dialog box, overlay behavioral features in the second-level prompt dialog box, and write semantic features in the third-level prompt dialog box.

[0085] like Figure 7 As shown, in one embodiment, the package recommendation method based on a multimodal large model provided by the present invention further includes the following steps in step S130: Step S133: Based on the semantic features written in the third-level prompt dialog box, call the LLaMA3.1-8B large model and combine it with the third-level prompt dialog box to perform reasoning and sorting on the candidate packages in the secondary package candidate set according to the candidate package features, and obtain the sorting result.

[0086] Step S134: Select multiple candidate packages with a ranking not lower than the second threshold from the sorting results to determine and output multiple recommended packages, and output the recommendation reason for each recommended package.

[0087] The recommendation reason is that the LLaMA3.1-8B large model is generated based on the characteristics of candidate packages and the summary text of user package transactions.

[0088] like Figure 8 As shown, in one embodiment, the package recommendation method based on a multimodal large model provided by the present invention includes the following steps in step S140: Step S141: Call the BERT model to calculate the cosine similarity between the recommendation reason and the corresponding candidate package content, in order to determine whether multiple recommended packages are simultaneously located in the secondary package candidate set and the preset package list.

[0089] Step S142: Based on the priority of each recommended package, output the recommended package and the corresponding recommendation reason according to the priority.

[0090] The following describes the multimodal large model-based package recommendation system provided by the present invention. The multimodal large model-based package recommendation system described below can be referred to in correspondence with the multimodal large model-based package recommendation method described above.

[0091] like Figure 9 As shown, in one embodiment, a package recommendation system based on a multimodal large model includes a feature extraction and fusion module 910, a package recall module 920, a package recommendation module 930, and a package verification and priority output module 940.

[0092] The feature extraction and fusion module 910 is used to extract user multimodal features from operator data and fuse user multimodal features to obtain standardized fused features.

[0093] The package recall module 920 is used to recall multiple historical packages whose similarity exceeds a first threshold based on the similarity between different packages in the user's historical package data, in order to determine the characteristics of candidate packages by combining preset business scenario filtering rules.

[0094] The package recommendation module 930 is used to call the LLaMA3.1-8B large model to perform inference and ranking based on fused features and candidate package features, so as to output multiple recommended packages with a ranking not lower than the second threshold.

[0095] The package verification and priority output module 940 is used to verify whether multiple recommended packages are in the preset package list and to fine-tune the business scenario filtering rules to determine the priority of each recommended package based on the fine-tuned business scenario filtering rules.

[0096] Among them, user multimodal features are divided into static features, behavioral features and semantic features. Static features include user information, number information and package information. Behavioral features include data changes within the package. Semantic features include summary text of the user's package application.

[0097] In this embodiment, the feature extraction and fusion module 910 of the multimodal large model-based package recommendation system provided by the present invention is specifically used for: Static features, behavioral features, and semantic features are collected through a user data platform, and the static features are encoded into discrete vectors by embedding.

[0098] The SASRec recommendation model is invoked to generate behavioral embedding vectors based on behavioral features, and a large model is used to generate corresponding text embedding vectors based on semantic features, so as to fuse discrete vectors, behavioral embedding vectors and text embedding vectors into fused features.

[0099] In this embodiment, the package recommendation system based on a multimodal large model provided by the present invention, the package recall module 920 is specifically used for: Obtain the current user's historical package data and calculate the similarity between different packages in the historical package data. Combine this with the package subscription records in the historical package data to recall multiple historical packages with a similarity exceeding a first threshold.

[0100] Based on multiple historical packages with similarity exceeding a first threshold, an initial package candidate set is constructed. Then, the candidate packages in the initial package candidate set are filtered according to preset business scenario filtering rules to obtain a secondary package candidate set.

[0101] In this embodiment, the package recommendation system based on a multimodal large model provided by the present invention, the package recall module 920 is further used for: Extract candidate package features from the candidate packages in the secondary package candidate set. The candidate package features include basic package features, semantic features, and popularity features.

[0102] Among them, the basic features of the package are the package information of the candidate package, including package ID, package price, data allowance, call duration and package contract period; the semantic features of the package are the text embedding vectors of the package name and package content of the candidate package; the popularity features of the package are the application rate and positive review rate of the candidate package.

[0103] In this embodiment, the package recommendation system based on a multimodal large model provided by the present invention, the package recommendation module 930 is specifically used for: The Projector maps behavior embedding vectors to text embedding vector space and constructs hierarchical tooltip dialogs, which are divided into a first-level tooltip dialog, a second-level tooltip dialog, and a third-level tooltip dialog.

[0104] Enter the text data of the package details in the first-level prompt dialog box, overlay behavioral features in the second-level prompt dialog box, and write semantic features in the third-level prompt dialog box.

[0105] In this embodiment, the package recommendation system based on a multimodal large model provided by the present invention, the package recommendation module 930 is further used for: Based on the semantic features written in the third-level prompt dialog box, the LLaMA3.1-8B large model is called to combine the third-level prompt dialog box with the candidate package features to infer and rank the candidate packages in the secondary package candidate set, and obtain the ranking result.

[0106] Select multiple candidate packages from the ranking results that rank no lower than the second threshold, determine and output multiple recommended packages, and output the recommendation reason for each recommended package.

[0107] The recommendation reason is that the LLaMA3.1-8B large model is generated based on the characteristics of candidate packages and the summary text of user package transactions.

[0108] In this embodiment, the package recommendation system based on a multimodal large model provided by the present invention, specifically uses the package verification and priority output module 940 for: The BERT model is called to calculate the cosine similarity between the recommendation reason and the corresponding candidate package content, in order to determine whether multiple recommended packages are simultaneously located in the secondary package candidate set and the preset package list.

[0109] Based on the priority of each recommended package, output the recommended packages and the corresponding reasons for recommending them according to their priority.

[0110] Figure 10 This example illustrates a schematic diagram of the physical structure of an electronic device, which can be a smart terminal. Its internal structure diagram can be as follows: Figure 10 As shown. The electronic device includes a processor, internal memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a package recommendation method based on a multimodal large model, which includes:

[0111] Extract user multimodal features from operator data and fuse these features to obtain standardized fused features; Based on the similarity between different packages in the user's historical package data, multiple historical packages with similarity exceeding the first threshold are recalled, and candidate package features are determined by combining preset business scenario filtering rules. The LLaMA 3.1-8B large model is invoked to perform inference and ranking based on fused features and candidate package features, so as to output multiple recommended packages with a ranking not lower than the second threshold; Verify whether multiple recommended packages are in the preset package list, and fine-tune the business scenario filtering rules to determine the priority of each recommended package based on the fine-tuned business scenario filtering rules; Among them, user multimodal features are divided into static features, behavioral features and semantic features. Static features include user information, number information and package information. Behavioral features include data changes within the package. Semantic features include summary text of the user's package application.

[0112] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the electronic device to which the present invention is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0113] On the other hand, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements a package recommendation method based on a multimodal large model, the method comprising: Extract user multimodal features from operator data and fuse these features to obtain standardized fused features; Based on the similarity between different packages in the user's historical package data, multiple historical packages with similarity exceeding the first threshold are recalled, and candidate package features are determined by combining preset business scenario filtering rules. The LLaMA 3.1-8B large model is invoked to perform inference and ranking based on fused features and candidate package features, so as to output multiple recommended packages with a ranking not lower than the second threshold; Verify whether multiple recommended packages are in the preset package list, and fine-tune the business scenario filtering rules to determine the priority of each recommended package based on the fine-tuned business scenario filtering rules; Among them, user multimodal features are divided into static features, behavioral features and semantic features. Static features include user information, number information and package information. Behavioral features include data changes within the package. Semantic features include summary text of the user's package application.

[0114] In another aspect, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, it implements a package recommendation method based on a multimodal large model, the method comprising:

[0115] Extract user multimodal features from operator data and fuse these features to obtain standardized fused features; Based on the similarity between different packages in the user's historical package data, multiple historical packages with similarity exceeding the first threshold are recalled, and candidate package features are determined by combining preset business scenario filtering rules. The LLaMA 3.1-8B large model is invoked to perform inference and ranking based on fused features and candidate package features, so as to output multiple recommended packages with a ranking not lower than the second threshold; Verify whether multiple recommended packages are in the preset package list, and fine-tune the business scenario filtering rules to determine the priority of each recommended package based on the fine-tuned business scenario filtering rules; Among them, user multimodal features are divided into static features, behavioral features and semantic features. Static features include user information, number information and package information. Behavioral features include data changes within the package. Semantic features include summary text of the user's package application.

[0116] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0117] By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0118] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0119] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for recommending meal packages based on a multimodal large model, characterized in that, The method includes: Extract user multimodal features from operator data and fuse the user multimodal features to obtain standardized fused features; Based on the similarity between different packages in the user's historical package data, multiple historical packages with similarity exceeding the first threshold are recalled, and candidate package features are determined by combining preset business scenario filtering rules. The LLaMA3.1-8B large model is invoked to perform inference and ranking based on the fused features and candidate package features, so as to output multiple recommended packages with a ranking not lower than the second threshold; Verify whether the multiple recommended packages are in the preset package list, and fine-tune the business scenario filtering rules to determine the priority of each recommended package based on the fine-tuned business scenario filtering rules; The user multimodal features are divided into static features, behavioral features, and semantic features. The static features are user information, number information, and package information. The behavioral features are data changes within the package. The semantic features are summary text of the user's package application. The method involves recalling multiple historical packages with similarity exceeding a first threshold based on the similarity between different packages in the user's historical package data, and then determining candidate package characteristics by combining them with preset business scenario filtering rules, including: Obtain the current user's historical package data and calculate the similarity between different packages in the historical package data, so as to recall multiple historical packages with similarity exceeding a first threshold by combining the package application records in the historical package data. Based on multiple historical packages with similarity exceeding the first threshold, an initial package candidate set is constructed, and the candidate packages in the initial package candidate set are filtered according to the preset business scenario filtering rules to obtain a secondary package candidate set. Extract the candidate package features from the candidate packages in the secondary package candidate set. The candidate package features include basic package features, semantic features of the package, and popularity features of the package. The basic features of the package are the package information of the candidate package, including package ID, package price, data allowance, call duration, and package contract period; the semantic features of the package are the text embedding vectors of the package name and package content of the candidate package; and the popularity features of the package are the subscription rate and positive review rate of the candidate package. The process of verifying whether the multiple recommended packages are in the preset package list and fine-tuning the business scenario filtering rules to determine the priority of each recommended package based on the fine-tuned business scenario filtering rules includes: The BERT model is called to calculate the cosine similarity between the recommendation reason and the corresponding candidate package content, in order to determine whether the multiple recommended packages are simultaneously located in the secondary package candidate set and the preset package list; Based on the priority of each recommended package, the recommended packages and the corresponding reasons for recommending them are output according to their priority.

2. The package recommendation method based on a multimodal large model according to claim 1, characterized in that, The process of extracting user multimodal features from operator data and fusing these features to obtain standardized fused features includes: The static features, behavioral features, and semantic features are collected through a user data platform, and the static features are encoded into discrete vectors by embedding. The SASRec recommendation model is invoked to generate behavioral embedding vectors based on the behavioral features, and a large model is used to generate corresponding text embedding vectors based on the semantic features, so as to fuse the discrete vectors, behavioral embedding vectors and text embedding vectors into the fused features.

3. The package recommendation method based on a multimodal large model according to claim 2, characterized in that, The LLaMA 3.1-8B large model is invoked to perform inference and ranking based on the fused features and candidate package features, so as to output multiple recommended packages with a ranking not lower than the second threshold, including: The behavior embedding vector is mapped to the text embedding vector space using the Projector, and a hierarchical prompt dialog box is constructed, which is divided into a first-level prompt dialog box, a second-level prompt dialog box, and a third-level prompt dialog box. Enter the text data of the package details in the first-level prompt dialog box, overlay the behavioral features in the second-level prompt dialog box, and write the semantic features in the third-level prompt dialog box.

4. The package recommendation method based on a multimodal large model according to claim 3, characterized in that, The step of calling the LLaMA3.1-8B large model to perform inference and ranking based on the fused features and candidate package features, and outputting multiple recommended packages with a ranking not lower than the second threshold, also includes: Based on the semantic features written in the third-level prompt dialog box, the LLaMA3.1-8B large model is called in conjunction with the third-level prompt dialog box to perform reasoning and sorting on the candidate packages in the secondary package candidate set according to the candidate package features, and the sorting result is obtained. From the ranking results, select multiple candidate packages with a ranking not lower than the second threshold, determine and output the multiple recommended packages, and output the recommendation reason for each recommended package; The recommendation reason is generated by the LLaMA3.1-8B big model based on the candidate package features and the summary text of the user's package application.

5. A package recommendation system based on a multimodal large model, characterized in that, The system is used to implement the multimodal large model-based package recommendation method according to any one of claims 1 to 4, the system comprising: The feature extraction and fusion module is used to extract user multimodal features from operator data and fuse the user multimodal features to obtain standardized fused features. The package recall module is used to recall multiple historical packages whose similarity exceeds a first threshold based on the similarity between different packages in the user's historical package data, in order to determine the characteristics of candidate packages by combining preset business scenario filtering rules. The package recommendation module is used to call the LLaMA3.1-8B large model to perform inference and ranking based on the fused features and candidate package features, so as to output multiple recommended packages with a ranking not lower than the second threshold. The package verification and priority output module is used to verify whether the multiple recommended packages are in the preset package list and to fine-tune the business scenario filtering rules so as to determine the priority of each recommended package according to the fine-tuned business scenario filtering rules. The user multimodal features are divided into static features, behavioral features, and semantic features. The static features are user information, number information, and package information. The behavioral features are data changes within the package. The semantic features are summary text of the user's package application.

6. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the package recommendation method based on a multimodal large model as described in any one of claims 1 to 4.

7. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the package recommendation method based on a multimodal large model as described in any one of claims 1 to 4.

Images