A method and system for generating a visit display guide suggestion based on AIGC technology

Abstract

The application provides a visit display guide suggestion generation method and system based on AIGC technology, which comprises the following steps: obtaining image data of product display of a store uploaded by a mobile terminal, and using a preset product display recognition model to recognize the image data to obtain a summary text of the product display of the store. Then, the summary text is combined with a display agreement of the store and a historical product sales report of the store into a prompt word in a preset prompt word format, and the prompt word is input into a preset LLM model to enable the LLM model to generate a corresponding product display suggestion. Finally, the product display suggestion is transmitted back to the mobile terminal for display. Compared with the prior art, the application simplifies a series of processes from display visit verification to giving display guide suggestion in the fast consumption field, so that a business staff member can obtain a corresponding product display suggestion only by shooting and uploading product display images, thereby improving the business visit efficiency of the business staff member.

Description

Technical Field

[0001] This invention relates to the field of multimodal understanding and generation technology, and in particular to a method and system for generating visitor display guidance suggestions based on AIGC technology. Background Technology

[0002] The application of artificial intelligence in the fast-moving consumer goods (FMCG) industry has always been primarily driven by market demand. Meanwhile, business models, operating philosophies, marketing strategies, and technologies based on digital infrastructure are catalyzing the iteration and development of various industries. New technologies such as Metaverse, GPT (Generative Pre-trained Transformer), and AIGC (Artificial intelligence generates content) are providing the best opportunity for enterprises to undergo digital transformation.

[0003] In the fast-moving consumer goods (FMCG) retail sector, to better formulate marketing strategies, retailers need to employ a team of sales representatives to regularly visit and verify the merchandise displays in offline stores. This allows them to understand the product placement and sales performance at each terminal store and to make timely adjustments to the merchandise displays to respond to changes in market demand. However, managing these sales representatives is challenging for retailers, and the upward and downward communication of orders in the management process is often time-consuming. From the time sales representatives collect store display results to uploading them to the management platform, managers analyze the results to obtain optimization suggestions, which are then passed down to the sales representatives for implementation. While many mature AI capabilities have been developed to assist in the collection of store display results, the optimization suggestions require professional management personnel for analysis and formulation. Currently, there is no ideal AI-enabled solution because the optimization suggestions are highly specialized and require personnel with professional analytical skills and years of management experience. However, the time and resource costs of information transmission during the entire visit process are high. Furthermore, because information transmission involves numerous process nodes, it is prone to loss, distortion, or even corruption during dissemination, leading to even greater losses. How to reduce information loss or information asymmetry during business visits is a critical pain point that major FMCG retail brands urgently need to address. The traditional approach is to reduce intermediate information transmission processes; that is, after salespeople report store display results, they can obtain display optimization suggestions from the stores without going through multiple layers of information transmission, and then implement optimization measures. However, due to the nature of salespeople's work and their own qualities and inertia, having frontline salespeople formulate display optimization suggestions is unrealistic and impractical. First, frontline salespeople have limited capabilities and lack the ability to formulate decision-making suggestions; second, having the same person make and implement the decisions can easily breed corruption and cheating, and contradicts management principles. Similarly, having managers conduct visits is not only costly but also a waste of resources and a waste of talent. Therefore, in response to the above problems, it is of great significance to provide a method that has the ability to analyze display results and generate display guidance suggestions for salespersons when they visit and collect display results. Summary of the Invention

[0004] This invention provides a method and system for generating visit display guidance suggestions based on AIGC technology, which improves the efficiency of business visits in the fast-moving consumer goods (FMCG) industry.

[0005] To address the aforementioned technical problems, in a first aspect, embodiments of the present invention provide a method for generating visitor display guidance suggestions based on AIGC technology, comprising:

[0006] Acquire image data of store merchandise displays uploaded by mobile devices;

[0007] Based on a preset merchandise display recognition model, the image data is recognized to obtain a summary text of the store's merchandise display;

[0008] The summary text, the store's display agreement, and the store's historical product sales report are combined into a prompt word using a preset prompt word format.

[0009] The prompt words are input into a preset LLM model to obtain product display suggestions corresponding to the store's product display, and the product display suggestions are transmitted back to the mobile terminal.

[0010] The LLM model was obtained by training the initial LLM model based on corpus data related to product display.

[0011] This invention provides a method for generating product display guidance suggestions based on AIGC technology. First, a product display recognition model is used to convert product display images taken by salespersons into text data. Then, the text data is input into an LLM model in a preset prompt word format to generate corresponding product display suggestions. This simplifies the entire process from display visit verification to providing display guidance suggestions in the FMCG industry, allowing salespersons to obtain corresponding product display suggestions simply by taking and uploading product display images, thus improving the efficiency of sales visits.

[0012] Furthermore, the product display recognition model is obtained by modeling the products in the store based on a target detection algorithm.

[0013] This invention provides a modeling method for a product display recognition model. By using a target detection algorithm to model the products in the store, the product display recognition model can accurately identify various products in the image and convert the image data uploaded by the salesperson into text, which facilitates the subsequent summarization of the product recognition results.

[0014] In one possible implementation, the LLM model is obtained by training an initial LLM model based on corpus data related to product display, specifically:

[0015] Collect corpus data related to product display and construct a corpus database for training. The corpus data includes: historical product information, historical store information, historical display report summaries, historical display agreements, historical store sales data, and historical store product display suggestions.

[0016] The corpus data in the corpus database is converted into prompt word data according to the prompt word format;

[0017] The prompt word data is segmented and mapped using a word segmenter, and the prompt word data is converted into corresponding number sequences. The set of each number sequence is used as a fine-tuning instruction dataset, and the answer is manually labeled for each data in the fine-tuning instruction dataset.

[0018] The initial LLM model is trained based on the fine-tuning instruction dataset to obtain the LLM model.

[0019] This invention provides a method for training an initial LLM model based on corpus data related to product display. The method involves collecting corpus data to construct a corpus database, and then converting the corpus database into a fine-tuning instruction dataset through a series of processing operations. The initial LLM model is then trained using this fine-tuning instruction dataset, enabling the trained LLM model to understand and generate professional knowledge in the FMCG field. It can generate corresponding answers based on professional prompts in the FMCG field, providing display suggestions to sales personnel and improving their business visit efficiency.

[0020] Furthermore, the initial LLM model is trained based on the fine-tuning instruction dataset to obtain the LLM model, specifically as follows:

[0021] Based on the fine-tuning instruction dataset, the initial LLM model is trained using supervised fine-tuning techniques to reduce the trainable parameters of the downstream tasks of the initial LLM model, thereby obtaining the first LLM model.

[0022] The fine-tuning instruction dataset is input into the first LLM model, and the initial machine answer corresponding to each data point in the fine-tuning instruction dataset is output. After the initial machine answer is manually scored, the initial machine answer is input into the initial reward model to obtain the initial machine score corresponding to each initial machine answer. The scoring result of the initial reward model is adjusted by adjusting the parameters of the initial reward model so that the scoring result of the initial reward model is close to the result of the manual scoring, thus obtaining the first reward model.

[0023] A reinforcement learning model is constructed by combining the first LLM model and the first reward model. The first LLM model is subjected to reinforcement learning through multiple iterations. In each iteration, the fine-tuning instruction dataset is input into the reinforcement learning model, and the first LLM model outputs the reinforcement learning answer corresponding to each data point in the fine-tuning instruction dataset. The first reward model scores each reinforcement learning answer. The reinforcement learning model updates the trainable parameters of the first LLM model based on the scoring results of the first reward model until the rate of change of the trainable parameters is lower than a preset threshold, at which point the iteration stops, and the LLM model is obtained.

[0024] This invention provides a method for training an initial LLM model based on a fine-tuning instruction dataset. First, supervised fine-tuning techniques are used to initially train the initial LLM model, reducing the trainable parameters for downstream tasks, simplifying training complexity, and shortening the time required to adjust subsequent trainable parameters, thus improving the overall efficiency of fine-tuning the initial LLM model. Second, after generating answers using the first LLM model, the answers are scored by both an initial reward model and a human. The machine scores are compared with the human scores, and by adjusting the parameters of the initial reward model, the machine scores are made closer to the human scores, enabling the initial reward model to correctly evaluate the answers generated by the first LLM model, thus obtaining the first reward model. In subsequent training, the first reward model can replace human scoring, significantly reducing the workload of manual scoring and improving the efficiency of training the initial LLM model. Finally, the first LLM model was combined with the first reward model to construct a reinforcement learning model. The first LLM model was then reinforced and trained, which further improved the first LLM model's ability to understand and generate professional knowledge in the FMCG field, resulting in the final LLM model that can be used to generate display suggestions in the FMCG field.

[0025] In one possible implementation, the prompt words are input into a preset LLM model to obtain product display suggestions corresponding to the store's product display, and the product display suggestions are then transmitted back to the mobile terminal, specifically as follows:

[0026] After obtaining the prompt word, the LLM model uses a word segmenter to segment and map the prompt word, converting the prompt word into a corresponding number sequence.

[0027] Calculate the corresponding sequence of numerical answers based on the given numerical sequence;

[0028] The numerical answer sequence is converted into product display suggestions corresponding to the store's product display using a preset answer format and then output.

[0029] The product display suggestions are transmitted back to the mobile terminal.

[0030] This invention provides a method for using an LLM model to identify prompt words and generate corresponding product display suggestions. By segmenting and mapping the prompt words, the prompt words are converted into a numerical sequence form that the model can receive. After receiving the numerical sequence, the LLM model calculates the corresponding answer numerical sequence. Finally, the answer numerical sequence is converted into text form to generate the product display suggestions. This method realizes the generation of corresponding answers based on professional prompt words in the FMCG field, providing display suggestions for salespersons and improving their business visit efficiency.

[0031] Secondly, correspondingly, the present invention also provides a visitor display guidance suggestion generation system based on AIGC technology, including an acquisition module, a recognition module, a prompt word combination module, an answer generation module, and a training module;

[0032] The acquisition module is used to acquire image data of the store's merchandise display uploaded by the mobile terminal;

[0033] The recognition module is used to recognize the image data according to the preset merchandise display recognition model to obtain a summary text of the store's merchandise display;

[0034] The prompt word combination module is used to combine the summary text with the store's display agreement and the store's historical product sales report into prompt words in a preset prompt word format;

[0035] The answer generation module is used to input the prompt words into a preset LLM model, obtain product display suggestions corresponding to the store's product display, and transmit the product display suggestions back to the mobile terminal;

[0036] The training module is used to train the initial LLM model based on corpus data related to product display to obtain the LLM model.

[0037] Furthermore, the product display recognition model is obtained by modeling the products in the store based on a target detection algorithm.

[0038] In one possible implementation, the training module further includes an acquisition unit, a conversion unit, a word segmentation unit, and a training unit;

[0039] The acquisition unit is used to collect corpus data related to product display and construct a corpus database for training. The corpus data includes: historical product information, historical store information, historical display report summaries, historical display agreements, historical store sales data, and historical store product display suggestions.

[0040] The conversion unit is used to convert corpus data in the corpus database into prompt word data according to the prompt word format;

[0041] The word segmentation unit is used to segment and map the prompt word data using a word segmenter, converting the prompt word data into corresponding number sequences. The set of each number sequence is used as a fine-tuning instruction dataset, and the answer is manually labeled for each piece of data in the fine-tuning instruction dataset.

[0042] The training unit is used to train the initial LLM model based on the fine-tuning instruction dataset to obtain the LLM model.

[0043] Furthermore, the training unit is used to train the initial LLM model based on the fine-tuning instruction dataset to obtain the LLM model, specifically as follows:

[0044] Based on the fine-tuning instruction dataset, the initial LLM model is trained using supervised fine-tuning techniques to reduce the trainable parameters of the downstream tasks of the initial LLM model, thereby obtaining the first LLM model.

[0045] The fine-tuning instruction dataset is input into the first LLM model, and the initial machine answer corresponding to each data point in the fine-tuning instruction dataset is output. After the initial machine answer is manually scored, the initial machine answer is input into the initial reward model to obtain the initial machine score corresponding to each initial machine answer. The scoring result of the initial reward model is adjusted by adjusting the parameters of the initial reward model so that the scoring result of the initial reward model is close to the result of the manual scoring, thus obtaining the first reward model.

[0046] A reinforcement learning model is constructed by combining the first LLM model and the first reward model. The first LLM model is subjected to reinforcement learning through multiple iterations. In each iteration, the fine-tuning instruction dataset is input into the reinforcement learning model, and the first LLM model outputs the reinforcement learning answer corresponding to each data point in the fine-tuning instruction dataset. The first reward model scores each reinforcement learning answer. The reinforcement learning model updates the trainable parameters of the first LLM model based on the scoring results of the first reward model until the rate of change of the trainable parameters is lower than a preset threshold, at which point the iteration stops, and the LLM model is obtained.

[0047] In one possible implementation, the answer generation module is used to input the prompt words into a preset LLM model to obtain product display suggestions corresponding to the store's product display, and then transmit the product display suggestions back to the mobile terminal, specifically:

[0048] After obtaining the prompt word, the LLM model uses a word segmenter to segment and map the prompt word, converting the prompt word into a corresponding number sequence.

[0049] Calculate the corresponding sequence of numerical answers based on the given numerical sequence;

[0050] The numerical answer sequence is converted into product display suggestions corresponding to the store's product display using a preset answer format and then output.

[0051] The product display suggestions are transmitted back to the mobile terminal. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating an embodiment of a visitor display guidance suggestion generation method based on AIGC technology provided by the present invention.

[0053] Figure 2 This is a schematic diagram illustrating an application scenario of the visitor display guidance suggestion generation method based on AIGC technology provided by the present invention.

[0054] Figure 3 This is a schematic diagram illustrating the process of training an LLM model in one embodiment of a visitor display guidance suggestion generation method based on AIGC technology provided by the present invention.

[0055] Figure 4 This is a flowchart illustrating the specific process of training an LLM model in one embodiment of a visitor display guidance suggestion generation method based on AIGC technology provided by the present invention.

[0056] Figure 5 This is a schematic diagram illustrating the process of using LORA technology to perform preliminary training on an LLM model in one embodiment of a visitor display guidance suggestion generation method based on AIGC technology provided by the present invention.

[0057] Figure 6 : A schematic diagram of an embodiment of a visitor display guidance suggestion generation system based on AIGC technology provided by the present invention.

[0058] Figure 7 This is a schematic diagram of the training module in one embodiment of a visitor display guidance suggestion generation system based on AIGC technology provided by the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] It should be noted that the step numbers in this document are only for the convenience of explaining the specific embodiments and are not intended to limit the order in which the steps are performed.

[0061] Example 1:

[0062] like Figure 1 As shown, Embodiment 1 provides a method for generating visitor display guidance suggestions based on AIGC technology, including steps S1 to S4:

[0063] S1. Obtain image data of the store's merchandise display uploaded by the mobile terminal;

[0064] S2. Based on the preset merchandise display recognition model, the image data is recognized to obtain a summary text of the store's merchandise display;

[0065] S3. Combine the summary text with the store's display agreement and the store's historical product sales report into a prompt word using a preset prompt word format;

[0066] S4. Input the prompt words into a preset LLM model to obtain product display suggestions corresponding to the store's product display, and transmit the product display suggestions back to the mobile terminal; the LLM model is obtained by training an initial LLM model based on corpus data related to product display.

[0067] This invention provides a method for generating product display guidance suggestions based on AIGC technology. First, a product display recognition model is used to convert product display images taken by salespersons into text data. Then, the text data is input into an LLM model in a preset prompt word format to generate corresponding product display suggestions. This simplifies the entire process from display visit verification to providing display guidance suggestions in the FMCG industry, allowing salespersons to obtain corresponding product display suggestions simply by taking and uploading product display images, thus improving the efficiency of sales visits. Figure 2 This is a schematic diagram of an application scenario in an embodiment of the present invention.

[0068] In step S2, the collected image data is processed using a preset merchandise display recognition model to obtain the recognition result for each display image. The results are then summarized at the store level to generate a merchandise display summary report for the currently visited store. The merchandise display recognition model is obtained by modeling the merchandise in the store using an object detection algorithm, which is a deep learning-based Cascade Mask R-CNN object detection model.

[0069] This invention provides a modeling method for a product display recognition model. By using a target detection algorithm to model the products in the store, the product display recognition model can accurately identify various products in the image and convert the image data uploaded by the salesperson into text, which facilitates the subsequent summarization of the product recognition results.

[0070] In step S3, the process of combining the summary text with the store's display agreement and the store's historical sales reports into prompts using a preset prompt format specifically involves: constructing a semantic search knowledge base using embedding, for example, storing relevant content about store merchandise displays in a vector database by store dimension, and indexing relevant content about merchandise displays for the corresponding store based on the store's keywords or ID. The process also involves querying the vector database for merchandise display content related to the current store, such as the store's display agreement and historical sales reports, and combining this information with the display summary from the current store visit to form prompts.

[0071] In step S4, the LLM model is obtained by training the initial LLM model based on corpus data related to product display, including steps S401 to S404. A flowchart illustrating the training process of the LLM model is shown below. Figure 3 As shown:

[0072] S401. Collect corpus data related to product display and construct a corpus database for training. The corpus data includes: historical product information, historical store information, historical display report summaries, historical display agreements, historical store sales data, and historical store product display suggestions.

[0073] S402. Convert the corpus data in the corpus database into prompt word data according to the prompt word format;

[0074] S403. Use a word segmenter to segment and map the prompt word data, convert the prompt word data into corresponding number sequences, and use the set of number sequences as a fine-tuning instruction dataset, and manually annotate the answer for each data in the fine-tuning instruction dataset.

[0075] S404. Train the initial LLM model based on the fine-tuning instruction dataset to obtain the LLM model.

[0076] This invention provides a method for training an initial LLM model based on corpus data related to product display. The method involves collecting corpus data to construct a corpus database, and then converting the corpus database into a fine-tuning instruction dataset through a series of processing operations. The initial LLM model is then trained using this fine-tuning instruction dataset, enabling the trained LLM model to understand and generate professional knowledge in the FMCG field. It can generate corresponding answers based on professional prompts in the FMCG field, providing display suggestions to sales personnel and improving their business visit efficiency. Figure 4 This is a flowchart illustrating a specific process for training an LLM model in an embodiment of the present invention.

[0077] Specifically, in step S404, the initial LLM model is trained based on the fine-tuning instruction dataset to obtain the LLM model, as follows:

[0078] Based on the fine-tuning instruction dataset, the initial LLM model is trained using supervised fine-tuning techniques to reduce the trainable parameters for downstream tasks, thus obtaining the first LLM model. Specifically, the LLM model is trained using LORA (Low-Rank Adaptation) technology with the fine-tuning instruction dataset. By freezing the pre-trained model weights and injecting the trainable rank decomposition matrix into each layer of the Transformer architecture, the number of trainable parameters for downstream tasks is significantly reduced. Specifically: Assuming W represents the weight matrix in the LLM network layer, the network weight update matrix ΔW can generally be obtained through backpropagation, where the updated weights are the negative gradient of the loss function multiplied by the learning rate, i.e.:

[0079] ΔW=α(―ΔL W )

[0080] Where α is the preset learning rate, ―ΔL W Let ΔW be the negative gradient of the loss function, and ΔW be the updated value of the weight matrix. From this, the updated weights W can be obtained. ′ for:

[0081] W ′ =W+ΔW

[0082] The final model outputs the feature embedding h:

[0083] h=Wx+ΔWx

[0084] When fine-tuning a pre-trained LLM to adapt to a new task, its inherent dimensionality is very low. In reality, compared to the original model, only a small number of the fine-tuned weights are actually usable. LoRA (Local Relational Mapping) effectively represents or approximates the weight matrix in a lower-dimensional space while retaining most of its important information or structure. That is, it decomposes the new weight matrix adapted to the task into a lower-dimensional (smaller) matrix without losing too much important information. Specifically: assume ΔW is the weight matrix W∈R A×B The weight update matrix is ​​then decomposed into two smaller matrices: ΔW = W A W B W A ∈R A×r W B ∈R r×B Keeping the original weights W unchanged, only train the new matrix W. A and W B A flowchart illustrating the process of training an LLM model using LoRa technology is shown below. Figure 5 As shown.

[0085] The fine-tuning instruction dataset is input into the first LLM model, and the initial machine answer corresponding to each data point in the fine-tuning instruction dataset is output. After the initial machine answer is manually scored, the initial machine answer is input into the initial reward model to obtain the initial machine score corresponding to each initial machine answer. The scoring result of the initial reward model is adjusted by adjusting the parameters of the initial reward model so that the scoring result of the initial reward model is close to the result of the manual scoring, thus obtaining the first reward model.

[0086] Combining the first LLM model and the first reward model, a reinforcement learning model is constructed. The first LLM model is then subjected to reinforcement learning through multiple iterations. In each iteration, the fine-tuning instruction dataset is input into the reinforcement learning model. The first LLM model outputs the reinforcement learning answer corresponding to each data point in the fine-tuning instruction dataset, and the first reward model scores each reinforcement learning answer. Based on the scoring results of the first reward model, the reinforcement learning model updates the trainable parameters of the first LLM model until the rate of change of the trainable parameters falls below a preset threshold, at which point the iteration stops, thus obtaining the LLM model. Specifically, this involves considering the k different answers {y1,…,y} of the current policy π for a given query x. n Sampling is performed. Each answer y... i It is sent to the reward model to obtain a reward score r. RM To ensure the model doesn't deviate too far from the initial state π0, an empirically estimated KL penalty term is added, resulting in the following final reward function:

[0087] r = r RM ―λ KL D KL (π||π0)

[0088] Where λ KL It is the hyperparameter of the KL penalty, D KL It is the KL function. λ KL The value is typically set to 0.05. Input queries are deduplicated and sampled from the remaining fine-tuning instruction dataset. This ensures input diversity while preserving the model's answer preferences in both single-turn instruction and multi-turn dialogue scenarios.

[0089] This invention provides a method for training an initial LLM model based on a fine-tuning instruction dataset. First, supervised fine-tuning techniques are used to initially train the initial LLM model, reducing the trainable parameters for downstream tasks, simplifying training complexity, and shortening the time required to adjust subsequent trainable parameters, thus improving the overall efficiency of fine-tuning the initial LLM model. Second, after generating answers using the first LLM model, the answers are scored by both an initial reward model and a human. The machine scores are compared with the human scores, and by adjusting the parameters of the initial reward model, the machine scores are made closer to the human scores, enabling the initial reward model to correctly evaluate the answers generated by the first LLM model, thus obtaining the first reward model. In subsequent training, the first reward model can replace human scoring, significantly reducing the workload of manual scoring and improving the efficiency of training the initial LLM model. Finally, the first LLM model was combined with the first reward model to construct a reinforcement learning model. The first LLM model was then reinforced and trained, which further improved the first LLM model's ability to understand and generate professional knowledge in the FMCG field, resulting in the final LLM model that can be used to generate display suggestions in the FMCG field.

[0090] Furthermore, in step S4, the prompt words are input into a preset LLM model to obtain product display suggestions corresponding to the store's product display, and the product display suggestions are transmitted back to the mobile terminal, specifically as follows:

[0091] After obtaining the prompt word, the LLM model uses a word segmenter to segment and map the prompt word, converting the prompt word into a corresponding number sequence.

[0092] Calculate the corresponding sequence of numerical answers based on the given numerical sequence;

[0093] The numerical answer sequence is converted into product display suggestions corresponding to the store's product display using a preset answer format and then output.

[0094] The product display suggestions are transmitted back to the mobile terminal.

[0095] This invention provides a method for using an LLM model to identify prompt words and generate corresponding product display suggestions. By segmenting and mapping the prompt words, the prompt words are converted into a numerical sequence form that the model can receive. After receiving the numerical sequence, the LLM model calculates the corresponding answer numerical sequence. Finally, the answer numerical sequence is converted into text form to generate the product display suggestions. This method realizes the generation of corresponding answers based on professional prompt words in the FMCG field, providing display suggestions for salespersons and improving their business visit efficiency.

[0096] Example 2:

[0097] like Figure 6 The present invention provides a visitor display guidance suggestion generation system based on AIGC technology, including an acquisition module 10, a recognition module 20, a prompt word combination module 30, an answer generation module 40, and a training module 50.

[0098] The acquisition module 10 is used to acquire image data of the store's merchandise display uploaded by the mobile terminal;

[0099] The recognition module 20 is used to recognize the image data according to a preset merchandise display recognition model to obtain a summary text of the store's merchandise display;

[0100] The prompt word combination module 30 is used to combine the summary text with the store's display agreement and the store's historical product sales report into prompt words in a preset prompt word format;

[0101] The answer generation module 40 is used to input the prompt words into a preset LLM model, obtain product display suggestions corresponding to the store's product display, and transmit the product display suggestions back to the mobile terminal;

[0102] The training module 50 is used to train the initial LLM model based on corpus data related to product display to obtain the LLM model.

[0103] Furthermore, the product display recognition model is obtained by modeling the products in the store based on a target detection algorithm.

[0104] In one possible implementation, such as Figure 7 As shown, the training module 50 further includes an acquisition unit 501, a conversion unit 502, a word segmentation unit 503, and a training unit 504;

[0105] The acquisition unit 501 is used to collect corpus data related to product display and construct a corpus database for training. The corpus data includes: historical product information, historical store information, historical display report summary, historical display agreement, historical store sales data and historical store product display suggestions.

[0106] The conversion unit 502 is used to convert the corpus data in the corpus database into prompt word data according to the prompt word format;

[0107] The word segmentation unit 503 is used to segment and map the prompt word data using a word segmenter, convert the prompt word data into corresponding number sequences, and the set of the number sequences is used as a fine-tuning instruction dataset. The answer is manually labeled for each data in the fine-tuning instruction dataset.

[0108] The training unit 504 is used to train the initial LLM model based on the fine-tuning instruction dataset to obtain the LLM model.

[0109] Furthermore, the training unit is used to train the initial LLM model based on the fine-tuning instruction dataset to obtain the LLM model, specifically as follows:

[0110] Based on the fine-tuning instruction dataset, the initial LLM model is trained using supervised fine-tuning techniques to reduce the trainable parameters of the downstream tasks of the initial LLM model, thereby obtaining the first LLM model.

[0111] The fine-tuning instruction dataset is input into the first LLM model, and the initial machine answer corresponding to each data point in the fine-tuning instruction dataset is output. After the initial machine answer is manually scored, the initial machine answer is input into the initial reward model to obtain the initial machine score corresponding to each initial machine answer. The scoring result of the initial reward model is adjusted by adjusting the parameters of the initial reward model so that the scoring result of the initial reward model is close to the result of the manual scoring, thus obtaining the first reward model.

[0112] A reinforcement learning model is constructed by combining the first LLM model and the first reward model. The first LLM model is subjected to reinforcement learning through multiple iterations. In each iteration, the fine-tuning instruction dataset is input into the reinforcement learning model, and the first LLM model outputs the reinforcement learning answer corresponding to each data point in the fine-tuning instruction dataset. The first reward model scores each reinforcement learning answer. The reinforcement learning model updates the trainable parameters of the first LLM model based on the scoring results of the first reward model until the rate of change of the trainable parameters is lower than a preset threshold, at which point the iteration stops, and the LLM model is obtained.

[0113] In one possible implementation, the answer generation module 40 is used to input the prompt words into a preset LLM model to obtain product display suggestions corresponding to the store's product display, and then transmit the product display suggestions back to the mobile terminal, specifically:

[0114] After obtaining the prompt word, the LLM model uses a word segmenter to segment and map the prompt word, converting the prompt word into a corresponding number sequence.

[0115] Calculate the corresponding sequence of numerical answers based on the given numerical sequence;

[0116] The numerical answer sequence is converted into product display suggestions corresponding to the store's product display using a preset answer format and then output.

[0117] The product display suggestions are transmitted back to the mobile terminal.

[0118] For more detailed steps and working principles of this embodiment, please refer to the relevant description in Embodiment 1, but not limited to these descriptions.

[0119] This invention provides a product display guidance suggestion generation system based on AIGC technology. First, a product display recognition model is used to convert product display images taken by salespersons into text data. Then, the text data is input into an LLM model in a preset prompt word format to generate corresponding product display suggestions. This simplifies the entire process from display visit verification to providing display guidance suggestions in the FMCG industry, allowing salespersons to obtain corresponding product display suggestions simply by taking and uploading product display images, thus improving the efficiency of sales visits.

[0120] Regarding model training, supervised fine-tuning techniques are used to initially train the initial LLM model, reducing the trainable parameters for downstream tasks, simplifying training complexity, and shortening the time required to adjust subsequent trainable parameters, thus improving the overall efficiency of fine-tuning the initial LLM model. Secondly, after generating answers using the first LLM model, both the initial reward model and human reviewers score the answers. The machine scores are compared with human scores, and by adjusting the parameters of the initial reward model, the machine scores are made closer to human scores, enabling the initial reward model to correctly evaluate the answers generated by the first LLM model, thus obtaining the first reward model. In subsequent training, the first reward model can replace human review, significantly reducing the workload of manual review and improving the efficiency of training the initial LLM model. Finally, the first LLM model and the first reward model are combined to construct a reinforcement learning model, which further enhances the first LLM model's understanding and generation capabilities of professional knowledge in the FMCG field, resulting in the final LLM model, which can then be used to generate display suggestions in the FMCG field.

[0121] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A visit display guide suggestion generation method based on AIGC technology, characterized in that, The method comprises: acquiring image data of a product display of a store uploaded by a mobile terminal; identifying the image data according to a preset product display recognition model to obtain summary text of the product display; combining the summary text with a display agreement of the store and a historical product sales report of the store into a prompt word in a preset prompt word format; inputting the prompt word into a preset LLM model to obtain a product display suggestion corresponding to the product display of the store, and returning the product display suggestion to the mobile terminal; the LLM model is obtained by training an initial LLM model based on product display related corpus data, specifically: collecting product display related corpus data to construct a corpus database for training, wherein the corpus data includes historical product information, historical store information, historical display report summaries, historical display agreements, historical store sales data, and historical store product display suggestions; convert the corpus data in the corpus database into prompt word data according to the prompt word format; perform word segmentation and mapping processing on the prompt word data using a word segmenter, convert the prompt word data into respective corresponding digital sequences, and use the set of digital sequences as a fine-tuning instruction data set and manually annotate answers for each data in the fine-tuning instruction data set; train the initial LLM model based on the fine-tuning instruction data set to obtain the LLM model, specifically: train the initial LLM model based on the fine-tuning instruction data set through supervised fine-tuning technology, reduce the trainable parameters of the downstream tasks of the initial LLM model, and obtain a first LLM model; input the fine-tuning instruction data set into the first LLM model to output an initial machine answer corresponding to each data in the fine-tuning instruction data set, input each machine answer into an initial reward model after manually scoring each initial machine answer to obtain an initial machine score corresponding to each initial machine answer; adjust the scoring results of the initial reward model by adjusting the parameters of the initial reward model to make the scoring results of the initial reward model close to the results of the manual scoring, obtain a first reward model; combine the first LLM model and the first reward model to construct a reinforcement learning model, and perform reinforcement learning on the first LLM model through multiple iterations, in each iteration, input the fine-tuning instruction data set into the reinforcement learning model, output a reinforcement learning answer corresponding to each data in the fine-tuning instruction data set from the first LLM model, and score each reinforcement learning answer from the first reward model, the reinforcement learning model updates the trainable parameters of the first LLM model according to the scoring results of the first reward model until the change rate of the trainable parameters is lower than a preset threshold, stops iteration, and obtains the LLM model.

2. The visit display guide suggestion generation method based on the AIGC technology according to claim 1, wherein, The product display recognition model is obtained by modeling the products of the store according to a target detection algorithm.

3. The method of claim 1, wherein the method further comprises: determining a product category of the product; and determining a product category of the product based on the product category of the product and the product category of the product. The prompt word is input into a preset LLM model to obtain a commodity display suggestion corresponding to the store commodity display, and the commodity display suggestion is returned to the mobile terminal, specifically: After the LLM model obtains the prompt word, the prompt word is processed by using a word segmenter for segmentation and mapping, and the prompt word is converted into a corresponding digital sequence; According to the digital sequence, a corresponding digital answer sequence is calculated; The digital answer sequence is converted into a commodity display suggestion corresponding to the store commodity display in a preset answer format for output; The commodity display suggestion is returned to the mobile terminal.

4. A visit display guide suggestion generation system based on AIGC technology, characterized by, It comprises an acquisition module, an identification module, a prompt word combination module, an answer generation module and a training module; The acquisition module is used to acquire image data of a store commodity display uploaded by a mobile terminal; The identification module is used to identify the image data according to a preset commodity display identification model to obtain a summary text of the store commodity display; The prompt word combination module is used to combine the summary text, a display agreement of the store and a historical commodity sales report of the store into a prompt word in a preset prompt word format; The answer generation module is used to input the prompt word into a preset LLM model to obtain a commodity display suggestion corresponding to the store commodity display, and return the commodity display suggestion to the mobile terminal; The training module is used to train an initial LLM model based on commodity display related corpus data to obtain the LLM model; the training module further comprises an acquisition unit, a conversion unit, a word segmentation unit and a training unit; The acquisition unit is used to collect commodity display related corpus data to construct a corpus database for training, wherein the corpus data comprises historical commodity information, historical store information, historical display report summaries, historical display agreements, historical store sales data and historical store commodity display suggestions; The conversion unit is used to convert corpus data in the corpus database into prompt word data according to the prompt word format; The word segmentation unit is used to use a word segmenter to process the prompt word data for segmentation and mapping, and convert the prompt word data into each corresponding digital sequence, and the set of each digital sequence is used as a fine-tuning instruction data set, and each data in the fine-tuning instruction data set is manually labeled with an answer; The training unit is configured to train the initial LLM model based on the fine-tuning instruction dataset to obtain the LLM model, specifically: training the initial LLM model based on the fine-tuning instruction dataset by a supervised fine-tuning technique to reduce trainable parameters of a downstream task of the initial LLM model to obtain a first LLM model; inputting the fine-tuning instruction dataset into the first LLM model to output an initial machine answer corresponding to each piece of data in the fine-tuning instruction dataset, inputting each initial machine answer into an initial reward model after artificially scoring each initial machine answer to obtain an initial machine score corresponding to each initial machine answer; adjusting the scoring result of the initial reward model by adjusting the parameters of the initial reward model to make the scoring result of the initial reward model close to the result of the artificial scoring to obtain a first reward model; combining the first LLM model and the first reward model to construct a reinforcement learning model, and performing reinforcement learning on the first LLM model through multiple iterations, in each iteration, inputting the fine-tuning instruction dataset into the reinforcement learning model, outputting a reinforcement learning answer corresponding to each piece of data in the fine-tuning instruction dataset by the first LLM model, and scoring each reinforcement learning answer by the first reward model, the reinforcement learning model updates the trainable parameters of the first LLM model according to the scoring result of the first reward model until the change rate of the trainable parameters is lower than a preset threshold, stopping iteration, and obtaining the LLM model.

5. The visit display guide advice generating system based on the AIGC technology according to Claim 4, wherein The commodity display recognition model is obtained by modeling the commodities of the store according to a target detection algorithm.

6. The visit display guide advice generating system based on the AIGC technology according to claim 4, characterized by, The answer generation module is configured to input the prompt word into a preset LLM model to obtain a commodity display suggestion corresponding to the store commodity display, and transmit the commodity display suggestion back to the mobile terminal, specifically: After the LLM model obtains the prompt word, the LLM model uses a word segmenter to perform word segmentation and mapping processing on the prompt word, converts the prompt word into a corresponding digital sequence; According to the digital sequence, a corresponding digital answer sequence is calculated; The digital answer sequence is converted into a commodity display suggestion corresponding to the store commodity display in a preset answer format for output; The commodity display suggestion is transmitted back to the mobile terminal.

Images