AI live broadcast and intelligent new retail linkage system

Abstract

The invention discloses an AI live broadcast and intelligent new retail linkage system, and relates to the technical field of AI live broadcast and new retail linkage, and the system comprises a content perception module which collects a live broadcast room video stream in real time, extracts explanation audio-to-text, synchronously recognizes a commodity entity in a video, and forms commodity explanation data containing the explanation text and the commodity entity; the knowledge alignment module matches the serial numbers of the commodity entities and the standard information, and establishes an alignment relationship between explanation characters and serial number time to form a time sequence knowledge unit; the hotspot recognition module analyzes the change trend of the attention degree in combination with real-time comments, and recognizes hot commodities exceeding a threshold value; and the scene reconstruction module extracts a scene background and an anchor action when the hot commodity is displayed, and generates an associated virtual display environment. According to the method, synchronous association of explanation and commodity entities, time sequence popularity analysis and real scene virtual reconstruction are realized, and the accuracy and immersion of linkage of live broadcast and new retail are improved.

Description

Technical Field

[0001] This invention belongs to the field of AI live streaming and new retail linkage technology, specifically an AI live streaming and intelligent new retail linkage system. Background Technology

[0002] In existing live-streaming and retail integration scenarios, live-streaming content processing often separates audio and video analysis. After capturing the live-stream video stream, it frequently only extracts the host's audio and converts it into text to generate subtitles, or performs product recognition on non-real-time images alone. It fails to associate the audio narration with the actual products displayed in the video, making it difficult to establish a correspondence between the narration and the products. Real-time audience interaction analysis is mostly based on audio-text or independent comment data, failing to combine the temporal correlation between the narration and the products to track changes in audience attention to specific products. Furthermore, the construction of virtual display environments often relies on pre-designed generic templates, lacking a direct connection to the actual product display scenario during the live stream.

[0003] In conventional technologies, audio-to-text conversion does not include information about the actual product in the video, making it impossible to generate explanatory data with the product itself. Audience attention analysis does not consider the time alignment between the explanation and the product, making it difficult to accurately identify popular products. The virtual display environment is unrelated to the real-world context of the product display and the host's interactions during the live stream, failing to recreate the product's presentation during the live stream. This invention aims to solve the problems in existing live stream data processing where explanatory text cannot be synchronously linked to the actual product in the video to form product explanation data; to solve the problem of analyzing changes in audience attention to specific products and identifying trending products based on the time alignment between the explanation and the product; and to solve the problem of generating a linked virtual display environment based on the context of trending products being displayed in the live stream and the host's interactions. Summary of the Invention

[0004] This invention aims to solve at least one of the technical problems existing in the prior art; To this end, the present invention proposes an AI live streaming and intelligent new retail linkage system, comprising: The content awareness module collects video stream data from the live broadcast room in real time, extracts the host's audio explanation from the video stream data and converts it into text, and simultaneously identifies the product entities displayed in the video stream data to form product explanation data containing explanation text and product entities. The knowledge alignment module matches the product entities in the product description data with the standard product information in the product database to obtain the product number, and establishes a time alignment relationship between the description text and the product number to form a product knowledge unit with a time sequence. The hotspot identification module inputs the time-series product knowledge units into the interactive analysis model, combines them with the real-time audience comment data of the live broadcast room, analyzes the changing trend of the live broadcast audience's attention to specific products, and identifies products whose audience attention exceeds a preset threshold during the explanation period as hotspot products. The scene reconstruction module initiates a real-time scene reconstruction process based on the hot-selling products, extracts the scene background and anchor interaction actions when the hot-selling products are displayed from the video stream data, and generates a virtual display environment associated with the hot-selling products.

[0005] Furthermore, the step of matching the product entities in the product description data with standard product information in the product database to obtain product numbers, and establishing a time-aligned relationship between the description text and the product numbers to form a time-series product knowledge unit, includes: Image features are extracted from the product entities in the product description data to obtain product visual feature vectors; at the same time, natural language processing is performed on the description text in the product description data to extract key descriptive words. The product visual feature vector and the key descriptive words are simultaneously input into the product matching network; The product matching network performs similarity retrieval in the visual feature library of the product database based on the product visual feature vector, and performs semantic matching retrieval in the text description library of the product database based on the key descriptive words. The visual similarity retrieval results and semantic matching retrieval results are weighted and fused, and the standard product information with the highest fusion score is selected, and its product number is taken as the matching result. The time point at which the matching result is generated is recorded, and the explanatory text paragraph from the time point until the next product entity is identified is bound to the product number to form a product knowledge unit in the time sequence.

[0006] Furthermore, the step of inputting the time-series product knowledge units into the interactive analysis model, and combining it with the real-time audience comment data of the live stream, to analyze the changing trends of live stream viewers' attention to specific products includes: Using a preset time window as the period, the product knowledge unit of the time sequence is sliced, and the explanation content in each time window corresponds to one or more product numbers; Simultaneously capture real-time audience comment data within the same time window, analyze the real-time audience comment data, and extract comment topics containing product names, attributes, and purchase intentions; Establish a relationship graph between the set of product numbers and the set of comment topics within each time window, and calculate the weight of the explanation duration and the weight of the frequency of mention in comments for each product number within the time window; Based on the weight of the explanation duration and the weight of the frequency of mention in comments, calculate the instantaneous attention of each product number within each time window; Connect the instantaneous attention levels of the same product number within all time windows to form a curve showing how the product's attention changes over time, i.e., the trend of attention change.

[0007] Furthermore, identifying products whose audience attention exceeds a preset threshold during the explanation period as hot-selling products includes: For each product, iterate through its attention trend curve and identify all local peak points on the curve; Calculate the average attention level of the product attention trend curve from the start of the explanation to the current time; A dynamic threshold is set, which is determined by the average attention level and a preset base threshold. The instantaneous attention level of the local peak point is compared with the dynamic threshold. If the instantaneous attention level of the local peak point continues to exceed the dynamic threshold for a period of time that meets a preset requirement, the product is determined to be the hot product.

[0008] Furthermore, based on the trending product, the real-time scene reconstruction process is initiated, extracting the scene background and anchor interaction actions when the trending product is displayed from the video stream data, and generating a virtual display environment associated with the trending product, including: Locate the time period corresponding to the explanation cycle of the hot-selling product in the video stream data; Extract multiple frames of video footage containing the trending products within the specified time period, and separate the foreground layer containing the trending products and the background layer containing the background and the anchor. Semantic segmentation is performed on the background layer to identify replaceable common background elements and the anchor's image outline; Retrieve high-precision 3D product models that match the popular products from the 3D model library; A virtual scene is constructed, and the general background elements are stylized and transferred to serve as the environmental background. The anchor's image outline is transformed into a virtual guide image. The high-precision 3D product model and the virtual guide image are placed into the virtual scene, and the key interactive actions of the anchor during the time period are reproduced to form the virtual display environment.

[0009] Furthermore, the system also includes: The shopping guide script module activates the intelligent shopping guide dialogue engine, integrating the product information of the hot-selling products, the associated virtual display environment information, and the explanatory text corresponding to the hot-selling products to generate multi-round shopping guide dialogue scripts and response strategies for the hot-selling products, including: Establish a knowledge graph centered on the aforementioned hot products. The nodes of the knowledge graph include product attributes, selling points extracted from the explanatory text, unique interactive methods of the associated virtual display environment, and preset common customer questions and answers. Label each node in the knowledge graph with dialogue guidance weights and contextual dependencies; Based on the aforementioned dialogue guidance weights, multiple potential dialogue paths are constructed, ranging from basic product attributes to in-depth selling points. For each potential dialogue path, the pre-generated guiding questions, product information statements, and expected answers based on the common customer questions and answers are arranged in logical order to form a multi-round shopping guide dialogue script. Configure the response strategy for each of the multi-round shopping guide dialogue scripts. The response strategy defines the decision rules for whether to continue along the current path, switch to other paths, or jump to the transaction guidance stage when the audience gives different feedback.

[0010] Furthermore, the system also includes: The interactive guidance module displays the popular products through the virtual display environment and calls the multi-round shopping guide dialogue script and response strategy to conduct interactive Q&A with the audience entering the virtual display environment, guiding the audience to explore the product features; The behavior monitoring module monitors the interactive behavior of viewers in the virtual display environment, including virtual product trial operations, requests to switch between different display angles, and zooming in to view feature details, and records the duration of viewer dwell time and operation sequence, including: Interactive event tracking points are embedded in the virtual display environment, and these tracking points are bound to the interactive components, view controllers, and zoom controls of the product's 3D model. When a viewer triggers the aforementioned interactive event tracking point, the event type, trigger time, triggering object, and duration of the interaction are recorded. Starting from the moment the audience enters the virtual display environment, all triggering events are continuously recorded to form a raw operation log sorted by time. The original operation logs are cleaned and labeled to remove invalid operations, and a series of coherent operation combinations are labeled as meaningful interactive tasks to obtain the operation sequence. The total dwell time from entry to exit is then calculated.

[0011] Furthermore, the system also includes: The intention calculation module is used to quantify and calculate a personalized purchase intention index for the current viewer based on the dwell time, operation sequence, and the purchase intention explicitly expressed by the viewer during the interactive Q&A process. Specifically, this includes: A base score is set for the dwell time, and a nonlinear mapping is performed based on the different intervals to which the dwell time belongs to obtain a duration score; Analyzing the operation sequence, high weight is given to in-depth exploration operations that point to the core selling points, and low weight is given to casual browsing operations. The interaction depth score is obtained by counting the weighted number of each type of operation. By analyzing the audience's text or voice input during the interactive question-and-answer process through natural language understanding, affirmative purchase intention statements are identified, and each statement is given a corresponding intention score based on its degree of affirmation and specificity. The duration score, interaction depth score, and intention score are standardized and then weighted and summed according to a preset weight ratio to obtain the personalized purchase intention index.

[0012] Furthermore, the system also includes: The solution generation module is used to dynamically generate a customized transaction solution that includes exclusive discount information, inventory status information, and logistics forecast information in response to the personalized purchase intention index reaching the transaction trigger threshold. Specifically, it includes: Establish a multi-level transaction trigger line, with each level corresponding to different preferential rates and service priorities; When the personalized purchase intention index reaches a certain transaction trigger line for the first time, a coupon or discount scheme that matches the first-level transaction trigger line is selected from the discount strategy library as the exclusive discount information; Real-time inventory data of the popular products in the background warehouse management system can be queried to obtain the inventory status information; Based on the delivery address pre-set by the audience or estimated based on the IP address, the logistics interface is called to obtain the estimated delivery time and cost, thus forming the logistics estimation information; The exclusive discount information, the inventory status information, and the logistics forecast information are combined to generate a temporary unique transaction scheme identifier, which constitutes the customized transaction scheme.

[0013] Furthermore, the system also includes: The transaction synchronization module is used to present the customized transaction plan to the audience in the virtual display environment and open a one-click order channel. After capturing the audience's confirmation information, it synchronizes the order data, audience identity identifier, and the exclusive discount information to the new retail order fulfillment system, specifically including: In the interactive interface of the virtual display environment, detailed information about the customized transaction plan is displayed in a prominent pop-up window; A confirmation button is provided in the pop-up window interface. The confirmation button is associated with a one-click order instruction. When the viewer clicks it, it is considered as confirmation. Upon capturing a click event on the confirmation button, order data is encapsulated. The order data includes the product number of the popular product, the quantity purchased, the price in the customized transaction plan, the exclusive discount information, and the selected delivery option in the logistics forecast information. Obtain the identity identifier of the currently logged-in viewer and bind the order data to the viewer identity identifier; The bound complete order data package is pushed to the new retail order fulfillment system in real time through an encrypted interface, and the new retail order fulfillment system completes the inventory pre-positioning and payment process initialization operations.

[0014] Compared with the prior art, the beneficial effects of the present invention are: The system collects live stream video data in real time, extracts the host's audio narration from the same video stream and converts it into text, and simultaneously identifies the product entities displayed in the video stream, forming product description data that includes both narration text and product entities. By synchronously associating the audio narration content with the on-screen product entities within the same real-time video stream, the system directly binds the narration data to the product entities. This allows subsequent processing to clearly identify the specific product corresponding to a particular narration segment, changing the conventional approach of only obtaining audio text or individual product recognition results, and giving the product description information the specific product indicated by the on-screen product.

[0015] This method involves matching product entities in the product description data with product numbers from the standard product database. A time-series product knowledge unit is then created by aligning the description text with these product numbers. This time-series product knowledge unit is input into an interactive analysis model and combined with real-time audience comment data to analyze trends in audience attention to specific products. Products whose attention exceeds a preset threshold during the description period are identified as "hot" products. By combining the time alignment between description and product to analyze audience attention, this method can pinpoint specific products based on the temporal correlation between description and product, changing the conventional analysis based on data without time correlation and making the identification of hot products more closely aligned with audience reactions during product descriptions.

[0016] The process of real-time scene reconstruction based on trending products involves extracting the background and host interactions from the video stream data when these products are displayed, and generating a virtual display environment associated with them. This method of generating a virtual environment using the background and host interactions of trending products during live streaming as source material ensures that the virtual environment includes elements of the real-world product presentation during the live stream. This departs from the conventional approach of using generic templates to generate virtual scenes, making the virtual environment more closely resemble the actual product display during the live stream. Attached Figure Description

[0017] Figure 1 This is a sequence diagram of the AI ​​live streaming and intelligent new retail linkage system described in this invention; Figure 2 A flowchart for identifying popular products. Detailed Implementation

[0018] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] See Figure 1 The system comprises a content awareness module, a knowledge alignment module, a hotspot identification module, and a scene reconstruction module. The content awareness module receives real-time video stream data from the live broadcast room, processes it, extracts the anchor's audio explanations and converts them into text, and simultaneously uses computer vision technology to identify product entities appearing in the video. The converted text and the identified product entities are organized together into structured product explanation data. The knowledge alignment module receives the product explanation data, matches the product entities with pre-stored standard product information in the backend product database, assigns a unique product number to each identified product entity, and associates the converted explanation text with the corresponding product number based on its timestamp, forming a series of time-series-marked product knowledge units. The hotspot identification module receives these time-series-marked product knowledge units and inputs them, along with real-time audience comment data obtained from the live broadcast platform interface, into a pre-set interactive analysis model. This model analyzes the dynamic changes in audience attention to different products along the live broadcast timeline, identifies products whose audience attention exceeds a preset threshold within a certain explanation period, and marks them as hotspot products. The scene reconstruction module starts immediately after the hot product is identified. It locates the time segment in the original video stream data where the hot product is highlighted, extracts the background scene and the host's interactive actions within that time segment, and drives the graphics rendering engine based on this information to generate an interactive virtual display environment associated with the hot product.

[0020] In one embodiment of the present invention, the content awareness module collects video stream data from the live broadcast room in real time, with the video stream data input to the system at a rate of 30 frames per second. The content awareness module calls a speech recognition service to transcribe the audio explanatory data extracted from the video stream data in real time, generating a continuous audio explanatory text stream. Simultaneously, the module calls an object detection model to scan each frame of the video screen, identify and frame the product entities appearing in the screen. Each identified product entity, along with its timestamp and the corresponding audio explanatory text stream segment, is encapsulated into a structured product explanation data.

[0021] The knowledge alignment module receives product description data, which includes product images and explanatory text. First, it preprocesses and extracts features from the product images, using a convolutional neural network to extract a 1024-dimensional visual feature vector representing the product's appearance, color, and texture. Simultaneously, the module performs natural language processing on the associated explanatory text, using word segmentation, stop word removal, and keyword extraction algorithms to extract key descriptive terms such as "autumn new arrival," "pure cotton material," and "slim fit." These key descriptive terms are organized in a list. The product visual feature vector and the list of key descriptive terms are then simultaneously input into the product matching network. This network contains two parallel retrieval submodules. One submodule calculates the similarity between the product visual feature vector and a pre-stored database of tens of thousands of standard product visual features, using cosine similarity to obtain a visual retrieval candidate list sorted in descending order of similarity score. Another submodule performs semantic matching between the list of key descriptive terms and the text description library of the product database. Semantic matching is achieved by calculating the average similarity between word vectors or using a sentence encoder, generating a semantic matching candidate list. The product matching network performs a weighted fusion of the visual similarity score of each candidate product in the visual retrieval candidate list and the semantic matching score of the corresponding product in the semantic matching candidate list. The weighted fusion is calculated as follows: the final matching score equals the product of the visual similarity score and the weight coefficient w plus the product of the semantic matching score and the weight coefficient s. The weight coefficients w and s are preset adjustable parameters, and their sum is 1. The product matching network selects the standard product information with the highest final matching score and outputs its corresponding unique product number as the matching result. The system records the precise time point when the matching result is generated and binds all explanatory text stream segments generated during this period, from that time point until the content awareness module identifies the next different product entity or after a fixed time interval, to the aforementioned product number. The binding process adds product number and time range tags to this explanatory text paragraph, thus forming a product knowledge unit with a time sequence. The product knowledge unit is stored in the time sequence database for subsequent modules to call.

[0022] The hotspot identification module reads a time-series stream of product knowledge units from a time-series database. The module sets a fixed time window of 60 seconds, which slides in 10-second increments to slice the product knowledge unit stream. Within each time window, the system aggregates the set of all appearing product IDs and calculates the cumulative explanation time for each product ID within that time window. The explanation time weight is calculated based on the ratio of the cumulative explanation time to the total duration of the time window. The formula for calculating the explanation time weight is:

[0023] in: Indicates goods In the time window The weighting of the explanation time within the content Indicates goods In the time window The total duration of the explanation within the period, This represents a fixed length of 60 seconds within the time window. Simultaneously, the hotspot identification module captures real-time audience comment data within the same 60-second time window via the live streaming platform's open interface. After data cleaning, the real-time audience comment data is input into the topic analysis model. Based on clustering or keyword density methods, the model extracts comment topics containing product names, attributes, or purchase intentions, such as "Do you have size L for this dress?" or "I want to buy it." The system counts the number of times the product is mentioned in comment topics within the time window and calculates the frequency weight of each mention based on the ratio of this number to the total number of comment topics within the time window. The formula for calculating the frequency weight of product mentions is: in: Indicates goods In the time window The frequency weight of comments within the content Indicates time window Products mentioned The number of comment topics This represents the total number of comment topics within the time window t.

[0024] The hotspot identification module calculates an instantaneous attention level for each product ID i within each time window t. The calculation of instantaneous attention level combines the weight of the explanation duration and the frequency of mentions in comments, and the formula is as follows: in: Indicates goods In the time window Instantaneous attention, and These are preset weighting parameters used to adjust the relative importance of explanation duration and comment mentions on attention. The system ranks the instantaneous attention of the same product number across different time windows in chronological order. Connecting these points forms a curve reflecting the fluctuation of the product's popularity over time; this curve represents the trend of the product's popularity. In some embodiments, the weighting parameters... and The system can be trained and adjusted using historical live streaming data. The time window length and sliding step size are configurable system parameters. Optionally, for extracting comment topics, a combination of named entity recognition and sentiment analysis based on a pre-trained language model can be used to more accurately capture purchase intentions directly related to the product. The system will build a bipartite graph association model between product IDs and the set of comment topics, using graph algorithms to calculate more complex association weights, not just frequency ratios. Instantaneous attention can be understood. The calculation is the foundational data for subsequent identification of trending products.

[0025] In one embodiment of the present invention, the process of identifying trending products using an interactive analysis model is described. (See also...) Figure 2 For each product, the system iterates through its attention trend curve, identifying all local peaks. The system calculates the average attention level from the start of the product's explanation to the current time. A dynamic threshold is set, determined by the calculated average attention level and a preset base threshold. The model compares the instantaneous attention value of each identified local peak with this dynamic threshold. If the instantaneous attention value of a local peak exceeds the dynamic threshold for a preset duration, the product is determined to be a hot-selling product.

[0026] The scene reconstruction module initiates a real-time scene reconstruction process. The system locates the specific time period corresponding to the explanation cycle of the featured product in the video stream data. The system extracts multiple frames of video footage containing the featured product within this time period and uses image segmentation technology to separate the foreground layer containing the featured product and the background layer containing the background and the anchor's image. Semantic segmentation is performed on the background layer to identify replaceable common background elements and the anchor's image outline. The system retrieves a high-precision 3D product model matching the featured product from a pre-set 3D model library. The system constructs a virtual scene, using the identified common background elements after stylization transfer processing as the background of the virtual environment, and transforming the anchor's image outline into a virtual guide image. The system places the high-precision 3D product model and the virtual guide image into the virtual scene and drives the virtual guide image to reproduce the anchor's key interactive actions within the original time period, forming the final virtual display environment.

[0027] In its implementation, the interactive analysis model employs a peak detection algorithm to scan the entire attention trend curve, identifying all local peak points. Local peak points are those where the instantaneous attention value is higher than the corresponding values ​​in the previous and next time windows. The system calculates the arithmetic mean of all data points on the product attention trend curve from the start of the product explanation to the current analysis time, defining this as the average attention level of the product attention trend curve. The dynamic threshold is set based on the average attention level of the product attention trend curve and a preset base threshold, which is a fixed constant derived from historical live stream data. The formula for calculating the dynamic threshold is:

[0028] in: Indicates a dynamic threshold. This represents the average level of attention shown by the trend curve of product attention changes. This represents the preset base threshold. and This is the system's configured mixing coefficient, used to balance the influence weights of the average attention level and the base threshold. The sum of the mixing coefficients is usually set to 1. The model compares the instantaneous attention value of each identified local peak point with the calculated dynamic threshold. For comparison, the system presets a time length requirement; if the instantaneous attention value at a certain local peak point continuously exceeds a dynamic threshold... If the duration of a peak point reaches a preset time length requirement, the product corresponding to that local peak point is determined to be a hot product. In some embodiments, the peak detection algorithm can employ a zero-crossing detection method based on the first derivative or a sliding window comparison method. It can be understood that the preset time length requirement is to filter out short-lived interest peaks and ensure that the identified hot products have sustained attention. After a hot product is identified, the scene reconstruction module initiates a real-time scene reconstruction process. The system locates the specific time period corresponding to the explanation cycle of the hot product in the original video stream data. This time period begins when the first local peak point exceeding the dynamic threshold appears on the product's attention trend curve and ends when the instantaneous attention value falls back below the dynamic threshold. The system extracts multiple frames containing hot products within this time period at a fixed frame sampling interval. Each frame is processed using an instance segmentation neural network, which separates the image into a foreground layer containing the hot product and a background layer containing the background and the anchor's image. Semantic segmentation is performed on the separated background layer. The semantic segmentation model identifies replaceable common background elements in the background layer, such as walls, shelves, decorations, and the anchor's image outline. The system retrieves and calls up corresponding high-precision 3D product models from a pre-built 3D model library based on the product ID of popular products. These high-precision 3D product models include the product's geometric structure, material textures, and skeletal binding information. The instance segmentation neural network can adopt a Mask R-CNN-based architecture. Optionally, the semantic segmentation model can use DeepLabv3+ or other convolutional neural networks capable of pixel-level classification. The virtual scene construction step is executed by the graphics rendering engine. The graphics rendering engine first initializes a virtual 3D space as the basis of the virtual scene. It then processes the common background elements identified by semantic segmentation through a style transfer neural network. The style transfer neural network converts the visual style of the common background elements into a pre-defined virtual environment style, and the converted result serves as the environment background of the virtual scene. Simultaneously, the system inputs the identified anchor's image outline into a character generator. The character generator generates a virtual guide image with a similar body shape and clothing style based on the outline information. The virtual guide image is an animable 3D model. The graphics rendering engine places high-precision 3D product models and virtual guide avatars into a virtual scene. Based on the anchor's action sequence within the corresponding time period in the original video stream data, it drives the virtual guide avatar to reproduce the anchor's key interactive actions, including pointing at products, picking up products, and displaying product details. This reproduction process is achieved through keyframe animation or motion capture data mapping, ultimately forming an interactive virtual display environment associated with popular products. The style transfer neural network can adopt a CycleGAN or AdaIN architecture. The character generator can use generative adversarial network technology to synthesize 3D virtual avatars from contour images.

[0029] In one embodiment of the present invention, the shopping guide script module activates the intelligent shopping guide dialogue engine. The intelligent shopping guide dialogue engine receives the product ID of a popular product, the associated virtual display environment information, and the explanatory text paragraphs bound to that popular product from the knowledge alignment module. The intelligent shopping guide dialogue engine first queries the product database for complete product attribute information based on the product ID of the popular product. Product attribute information includes product name, price, specifications, and material composition. The intelligent shopping guide dialogue engine performs semantic analysis on the bound explanatory text paragraphs, extracting emphasized product selling points, such as "super waterproof performance" and "using new composite materials." Simultaneously, the intelligent shopping guide dialogue engine parses the virtual display environment information, identifying preset characteristic interaction methods within the virtual display environment, such as "360-degree rotation viewing" and "disassembly component demonstration." The intelligent shopping guide dialogue engine also loads a preset common customer Q&A knowledge base, which contains typical questions and standard answers for various types of products. The intelligent shopping guide dialogue engine constructs a knowledge graph centered on the popular product. The nodes of the knowledge graph include product attributes, selling points extracted from the explanatory text, characteristic interaction methods of the virtual display environment, and preset common customer Q&A. The system assigns a dialogue guidance weight to each node in the knowledge graph. This weight is a numerical value, indicating the priority at which the node is actively mentioned in a conversation. Basic product attributes such as price and specifications have lower dialogue guidance weights, while core selling points and unique interaction methods have higher weights. Nodes in the knowledge graph also have contextual dependencies, defining the logical connections between them. For example, the "product material" node can be connected to the "cleaning and maintenance" question-and-answer node.

[0030] Based on dialogue guidance weights, the intelligent shopping guide dialogue engine constructs multiple potential dialogue paths starting from basic product attribute nodes and gradually pointing to high-weight core selling point nodes. Each potential dialogue path is an ordered sequence of nodes. For each potential dialogue path, the intelligent shopping guide dialogue engine selects elements from a pre-generated statement template library. The engine then arranges pre-generated guiding questions, product information statements, and expected answers based on common customer questions and answers according to the logical order of the nodes in the potential dialogue path, forming a coherent multi-round shopping guide dialogue script. The system configures a response strategy for each multi-round shopping guide dialogue script. The response strategy is defined in the form of rules or decision trees. The response strategy specifies the rules for determining the next dialogue direction based on the text or voice feedback input by the audience when interacting with them based on the multi-round shopping guide dialogue script. These rules include delving deeper into the next node along the current potential dialogue path, switching to the starting node of another potential dialogue path, or directly jumping to the transaction guidance stage to inquire about purchase intentions. In some embodiments, the calculation of dialogue guidance weights can be based on the frequency of node mentions and the final conversion rate in historical dialogue data. The calculation formula can be:

[0031] in: This represents the dialogue guidance weight of node n. This indicates the frequency with which node n is mentioned in the historical conversation. The maximum frequency of mentions across all nodes is used for normalization. It is the statistical probability that node n will result in a purchase conversion after being discussed. It is a coefficient that balances frequency and conversion impact. It can be understood that constructing potential dialogue paths can be viewed as performing weighted path search on a knowledge graph.

[0032] The interactive guidance module showcases featured products to visitors through a virtual display environment. This environment provides a 3D product model viewing interface and text or voice chat functionality. The module invokes a multi-round sales guide dialogue script and response strategy generated for the current featured product, engaging in interactive Q&A with the visitor. Based on the dialogue script, the module proactively asks guiding questions or presents product information, and determines the next round of dialogue content based on the visitor's responses and the response strategy, thus guiding the visitor to explore product features. The behavior monitoring module synchronously monitors all visitor interactions within the virtual display environment. Interactive event tracking points are embedded in the rendering and interaction logic layers of the virtual environment. These event tracking points are predefined code hooks bound to interactive components, view controllers, and zoom controls of the product's 3D model. For example, interactive event tracking points are placed on detachable parts of the product, on the view rotation slider, and on the magnifying glass icon. When visitors trigger these interactive event tracking points through mouse clicks, touch, or voice commands, the behavior monitoring module records the event type, trigger time, trigger object, and interaction duration. The behavior monitoring module uses the unique identifier of the audience session as the key to record all interactive event tracking information triggered from the moment the audience enters the virtual display environment until they leave, forming a raw operation log sorted by timestamp.

[0033] The original operation logs are cleaned and labeled. The cleaning process removes short-lived, accidental triggers or repetitive operation records. The labeling process, based on predefined rules, identifies a series of coherent operations as meaningful interaction tasks. For example, rotating a product model three times consecutively to a specific angle and zooming in is labeled as a "detail inspection task." This results in a structured operation sequence, and the total dwell time from the viewer's entry into the virtual display environment to their departure is calculated. In some embodiments, the triggering of interaction event tracking can be based on browser click event listeners or collision detection in a game engine. Optionally, the labeling of interaction tasks can be accomplished using a classifier based on action sequence pattern recognition. It is understood that the original operation logs record the atomic operations of the viewer's interaction with the virtual display environment. Optionally, recording the duration of interaction is crucial for distinguishing between deep interaction and casual browsing. It is understood that the operation sequence and dwell time are fundamental data for quantifying viewer behavior.

[0034] In one embodiment of the present invention, the intention calculation module receives dwell time data, operation sequence data, and interactive question-and-answer process text or speech transcription data recorded by the interaction guidance module from the behavior monitoring module. The intention calculation module sets a base score for the dwell time, which is a fixed starting score. The system performs a non-linear mapping based on different preset intervals to which the dwell time belongs to obtain a duration score. The non-linear mapping is implemented through a piecewise function. The longer the dwell time, the gentler the gradient of the duration score growth. The specific segment intervals and corresponding score values ​​of the non-linear mapping are pre-configured by the system administrator according to the product category and scenario.

[0035] The intention calculation module analyzes the operation sequence, which is a series of meaningful interactive task records after cleaning and labeling. The system maintains an operation weight configuration table, assigning a weight value to each predefined interactive task type. For example, the "zoom in for details" operation is given a high weight value, the "quick view switching" operation a medium weight value, and the "page scrolling" operation a low weight value. The intention calculation module traverses the operation sequence, identifies the interactive task type corresponding to each operation record, and looks up its weight value from the operation weight configuration table. The intention calculation module calculates the weighted sum of the number of times each type of operation occurs to obtain the interaction depth score. The formula for calculating the interaction depth score is as follows: ,in Indicates the depth of interaction score. This indicates the total number of operation types defined in the system. This represents the preset weight value for the k-th operation type. This represents the number of times the k-th operation type appears in the current viewer's operation sequence. See Table 1 for a simplified operation weight configuration table.

[0036] Table 1: Operation Weight Configuration Table

[0037] It is understandable that the specific weight values ​​in the operation weight configuration table can be dynamically adjusted according to the product type.

[0038] The intention calculation module analyzes the audience's text or voice input during the interactive question-and-answer process using natural language understanding technology. Based on a pre-trained intention recognition and sentiment analysis module, the natural language understanding model identifies affirmative purchase intention statements in the audience's input. These statements include explicit expressions of purchase intention such as "This price is acceptable," "I want to place an order," and "Wrap it up for me." Each identified affirmative purchase intention statement receives a corresponding intention score based on its degree of affirmation and specificity. The degree of affirmation is quantified using confidence metrics from sentiment analysis, while specificity is assessed by whether the statement contains specific product attributes, quantities, or delivery information. The intention score combines the degree of affirmation score and the specificity score. In some embodiments, the natural language understanding model can be fine-tuned using a pre-trained model based on the Transformer architecture.

[0039] The intention calculation module standardizes the duration score, interaction depth score, and intention score using a Min-Max normalization method, mapping each score to a range of 0 to 1. The standardized duration score, interaction depth score, and intention score are then weighted and summed according to a pre-defined weighting ratio to obtain the final personalized purchase intention index. The formula for calculating the personalized purchase intention index is as follows:

[0040] in: Indicating a personalized purchase intention index, This represents the standardized duration score. This represents the standardized interaction depth score. This represents the standardized intention score. , , These are the preset weight coefficients corresponding to the three scores, and they satisfy... .

[0041] The solution generation module establishes a multi-level transaction trigger line, with each level corresponding to a personalized purchase intention index threshold and different discount levels and service priorities. For example, a three-level transaction trigger line can be established. , , ,in When the personalized purchase intention index The solution generation module is activated when a certain transaction trigger threshold is reached or exceeded for the first time. The module selects a coupon or discount scheme from the discount strategy library that matches the currently reached transaction trigger threshold as exclusive discount information. The discount strategy library pre-stores discount rules corresponding to different trigger threshold levels. The module also queries the real-time inventory data of popular products in the backend warehouse management system to obtain inventory status information, including the current available inventory quantity and warehouse location. Optionally, the module uses the delivery address pre-set in the viewer's profile, or, if not set, the potential delivery address estimated from the IP address of the network session, to call the application programming interface (API) of the logistics service provider to obtain estimated delivery time and cost, forming logistics forecast information. Obtaining logistics forecast information may require calling the APIs of multiple logistics service providers to obtain the best quote. The module then assembles the exclusive discount information, inventory status information, and logistics forecast information, generating a structured data packet containing fields such as discount amount, inventory quantity, estimated delivery time, and shipping cost. Simultaneously, the scheme generation module generates a temporary, unique transaction scheme identifier. This unique identifier consists of a product number, a timestamp, and a random number. This unique identifier, together with the aforementioned structured data packet, constitutes a customized transaction scheme for the current audience. It can be understood that the multi-level transaction trigger mechanism allows the system to provide differentiated promotional strategies based on the audience's different interest levels.

[0042] In one embodiment of the invention, the transaction synchronization module presents a customized transaction plan to the audience in a virtual display environment. The detailed information of the customized transaction plan is highlighted in a pop-up window within the interactive interface of the virtual display environment. A confirmation button is provided in the pop-up window, which is associated with a one-click order placement instruction; clicking this button by the audience signifies confirmation of the plan. Upon capturing the click event of the confirmation button, the system encapsulates the order data, which includes the product number of the popular item, the quantity purchased, the price in the customized transaction plan, exclusive discount information, and the selected delivery option from the logistics forecast information. The system obtains the identity of the currently logged-in audience and binds the order data to that audience identity. The bound complete order data package is then pushed to the new retail order fulfillment system in real time via an encrypted interface. The new retail order fulfillment system then completes inventory pre-positioning and payment process initialization operations.

[0043] In implementation, the pop-up interface uses high-contrast colors and eye-catching borders. The pop-up's internal area displays detailed information about the customized transaction plan in columns. This information includes the name and image of the featured product, the final price after applying exclusive discounts, the real-time inventory quantity in the inventory status information, and the estimated delivery time and shipping cost options in the logistics estimation information. A graphical confirmation button is provided at the bottom of the pop-up interface, labeled "Order Now" or similarly. Logically, this button is linked to a one-click order instruction. Clicking the confirmation button with a mouse or touchscreen is considered final confirmation of the currently displayed customized transaction plan. When the front-end event listener of the transaction synchronization module captures the click event of the confirmation button, the back-end service of the transaction synchronization module initiates the order data encapsulation process. This process collects and organizes multiple data fields, encapsulating them into a structured order data object. The order data object contains the following fields: the product number of the featured product, the quantity selected by the viewer, the final settlement price in the customized transaction plan, the coupon code or discount scheme identifier in the exclusive discount information, and the specific delivery option selected by the viewer from the multiple options in the logistics estimation information. The structured encapsulation process of order data objects can be represented as follows:

[0044] in: This represents the encapsulated order data object. Product numbers representing popular items. Represents the quantity purchased. Represents the final settlement price. This represents a unique discount information code. This represents the code for the selected delivery option.

[0045] The transaction synchronization module calls the identity authentication service to obtain the unique audience identifier of the currently logged-in audience in the system. The audience identifier is usually a user ID or account name. The transaction synchronization module will then encapsulate the order data object. The order data is bound to an audience identifier. This binding process involves adding a field named "UserID" to the order data object and filling it with the audience identifier's value, forming a complete order data package. In some embodiments, the order data object can be encapsulated using standard data exchange formats such as JSON or XML. The transaction synchronization module pushes the complete order data package bound to the audience identifier to the new retail order fulfillment system in real time via an encrypted network interface. The push process uses the HTTPS protocol and adds encryption to the data package at the application layer, ensuring the security of data transmission. Upon receiving the complete order data package, the new retail order fulfillment system immediately parses the data package content and then processes it according to the product number. and purchase quantity The system performs an inventory pre-holding operation, locking a corresponding quantity of goods from available inventory to prevent overselling. Simultaneously, the new retail order fulfillment system initializes the payment process, which includes generating a payment order number, calculating the amount due, and redirecting to the payment gateway or displaying a payment QR code. Inventory pre-holding and payment initialization are standard processing steps after the new retail order fulfillment system receives order data. In some embodiments, the encryption interface can use RSA or AES-based encryption algorithms to encrypt the order data packets. Optionally, after successful inventory pre-holding, the new retail order fulfillment system can return a confirmation receipt to the transaction synchronization module. The transaction synchronization module achieves seamless integration between the interaction in the virtual display environment and the actual backend transaction system.

[0046] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. An AI live streaming and smart new retail linkage system, characterized in that, include: The content awareness module collects video stream data from the live broadcast room in real time, extracts the host's audio explanation from the video stream data and converts it into text, and simultaneously identifies the product entities displayed in the video stream data to form product explanation data containing explanation text and product entities. The knowledge alignment module matches the product entities in the product description data with the standard product information in the product database to obtain the product number, and establishes a time alignment relationship between the description text and the product number to form a product knowledge unit with a time sequence. The hotspot identification module inputs the time-series product knowledge units into the interactive analysis model, combines them with the real-time audience comment data of the live broadcast room, analyzes the changing trend of the live broadcast audience's attention to specific products, and identifies products whose audience attention exceeds a preset threshold during the explanation period as hotspot products. The scene reconstruction module initiates a real-time scene reconstruction process based on the hot-selling products, extracts the scene background and anchor interaction actions when the hot-selling products are displayed from the video stream data, and generates a virtual display environment associated with the hot-selling products.

2. The AI ​​live streaming and intelligent new retail linkage system according to claim 1, characterized in that, The step of matching the product entities in the product description data with standard product information in the product database to obtain product numbers, and establishing a time-aligned relationship between the description text and the product numbers to form a time-series product knowledge unit, includes: Image features are extracted from the product entities in the product description data to obtain product visual feature vectors; at the same time, natural language processing is performed on the description text in the product description data to extract key descriptive words. The product visual feature vector and the key descriptive words are simultaneously input into the product matching network; The product matching network performs similarity retrieval in the visual feature library of the product database based on the product visual feature vector, and performs semantic matching retrieval in the text description library of the product database based on the key descriptive words. The visual similarity retrieval results and semantic matching retrieval results are weighted and fused, and the standard product information with the highest fusion score is selected, and its product number is taken as the matching result. The time point at which the matching result is generated is recorded, and the explanatory text paragraph from the time point until the next product entity is identified is bound to the product number to form a product knowledge unit in the time sequence.

3. The AI ​​live streaming and intelligent new retail linkage system according to claim 1, characterized in that, The step of inputting the time-series product knowledge units into the interactive analysis model, and combining it with the real-time audience comment data of the live broadcast room, to analyze the changing trends of live broadcast audience attention to specific products includes: Using a preset time window as the period, the product knowledge unit of the time sequence is sliced, and the explanation content in each time window corresponds to one or more product numbers; Simultaneously capture real-time audience comment data within the same time window, analyze the real-time audience comment data, and extract comment topics containing product names, attributes, and purchase intentions; Establish a relationship graph between the set of product numbers and the set of comment topics within each time window, and calculate the weight of the explanation duration and the weight of the frequency of mention in comments for each product number within the time window; Based on the weight of the explanation duration and the weight of the frequency of mention in comments, calculate the instantaneous attention of each product number within each time window; Connect the instantaneous attention levels of the same product number within all time windows to form a curve showing how the product's attention changes over time, i.e., the trend of attention change.

4. The AI ​​live streaming and intelligent new retail linkage system according to claim 3, characterized in that, The identification of products whose audience attention exceeds a preset threshold during the explanation period as hot-selling products includes: For each product, iterate through its attention trend curve and identify all local peak points on the curve; Calculate the average attention level of the product attention trend curve from the start of the explanation to the current time; A dynamic threshold is set, which is determined by the average attention level and a preset base threshold. The instantaneous attention level of the local peak point is compared with the dynamic threshold. If the instantaneous attention level of the local peak point continues to exceed the dynamic threshold for a period of time that meets a preset requirement, the product is determined to be the hot product.

5. The AI ​​live streaming and intelligent new retail linkage system according to claim 1, characterized in that, Based on the trending product, the real-time scene reconstruction process is initiated, extracting the scene background and anchor interaction actions when the trending product is displayed from the video stream data, and generating a virtual display environment associated with the trending product, including: Locate the time period corresponding to the explanation cycle of the hot-selling product in the video stream data; Extract multiple frames of video footage containing the trending products within the specified time period, and separate the foreground layer containing the trending products and the background layer containing the background and the anchor. Semantic segmentation is performed on the background layer to identify replaceable common background elements and the anchor's image outline; Retrieve high-precision 3D product models that match the popular products from the 3D model library; A virtual scene is constructed, and the general background elements are stylized and transferred to serve as the environmental background. The anchor's image outline is transformed into a virtual guide image. The high-precision 3D product model and the virtual guide image are placed into the virtual scene, and the key interactive actions of the anchor during the time period are reproduced to form the virtual display environment.

6. The AI ​​live streaming and intelligent new retail linkage system according to claim 1, characterized in that, The system also includes: The shopping guide script module activates the intelligent shopping guide dialogue engine, integrating the product information of the hot-selling products, the associated virtual display environment information, and the explanatory text corresponding to the hot-selling products to generate multi-round shopping guide dialogue scripts and response strategies for the hot-selling products, including: Establish a knowledge graph centered on the aforementioned hot products. The nodes of the knowledge graph include product attributes, selling points extracted from the explanatory text, unique interactive methods of the associated virtual display environment, and preset common customer questions and answers. Label each node in the knowledge graph with dialogue guidance weights and contextual dependencies; Based on the aforementioned dialogue guidance weights, multiple potential dialogue paths are constructed, ranging from basic product attributes to in-depth selling points. For each potential dialogue path, the pre-generated guiding questions, product information statements, and expected answers based on the common customer questions and answers are arranged in logical order to form a multi-round shopping guide dialogue script. Configure the response strategy for each of the multi-round shopping guide dialogue scripts. The response strategy defines the decision rules for whether to continue along the current path, switch to other paths, or jump to the transaction guidance stage when the audience gives different feedback.

7. The AI ​​live streaming and intelligent new retail linkage system according to claim 1, characterized in that, The system also includes: The interactive guidance module displays the popular products through the virtual display environment and calls the multi-round shopping guide dialogue script and response strategy to conduct interactive Q&A with the audience entering the virtual display environment, guiding the audience to explore the product features; The behavior monitoring module monitors the interactive behavior of viewers in the virtual display environment, including virtual product trial operations, requests to switch between different display angles, and zooming in to view feature details, and records the duration of viewer dwell time and operation sequence, including: Interactive event tracking points are embedded in the virtual display environment, and these tracking points are bound to the interactive components, view controllers, and zoom controls of the product's 3D model. When a viewer triggers the aforementioned interactive event tracking point, the event type, trigger time, triggering object, and duration of the interaction are recorded. Starting from the moment the audience enters the virtual display environment, all triggering events are continuously recorded to form a raw operation log sorted by time. The original operation logs are cleaned and labeled to remove invalid operations, and a series of coherent operation combinations are labeled as meaningful interactive tasks to obtain the operation sequence. The total dwell time from entry to exit is then calculated.

8. The AI ​​live streaming and intelligent new retail linkage system according to claim 7, characterized in that, The system also includes: The intention calculation module is used to quantify and calculate a personalized purchase intention index for the current viewer based on the dwell time, operation sequence, and the purchase intention explicitly expressed by the viewer during the interactive Q&A process. Specifically, this includes: A base score is set for the dwell time, and a nonlinear mapping is performed based on the different intervals to which the dwell time belongs to obtain a duration score; Analyzing the operation sequence, high weight is given to in-depth exploration operations that point to the core selling points, and low weight is given to casual browsing operations. The interaction depth score is obtained by counting the weighted number of each type of operation. By analyzing the audience's text or voice input during the interactive question-and-answer process through natural language understanding, affirmative purchase intention statements are identified, and each statement is given a corresponding intention score based on its degree of affirmation and specificity. The duration score, interaction depth score, and intention score are standardized and then weighted and summed according to a preset weight ratio to obtain the personalized purchase intention index.

9. The AI ​​live streaming and intelligent new retail linkage system according to claim 8, characterized in that, The system also includes: The solution generation module is used to dynamically generate a customized transaction solution that includes exclusive discount information, inventory status information, and logistics forecast information in response to the personalized purchase intention index reaching the transaction trigger threshold. Specifically, it includes: Establish a multi-level transaction trigger line, with each level corresponding to different preferential rates and service priorities; When the personalized purchase intention index reaches a certain transaction trigger line for the first time, a coupon or discount scheme that matches the first-level transaction trigger line is selected from the discount strategy library as the exclusive discount information; Real-time inventory data of the popular products in the background warehouse management system can be queried to obtain the inventory status information; Based on the delivery address pre-set by the audience or estimated based on the IP address, the logistics interface is called to obtain the estimated delivery time and cost, thus forming the logistics estimation information; The exclusive discount information, the inventory status information, and the logistics forecast information are combined to generate a temporary unique transaction scheme identifier, which constitutes the customized transaction scheme.

10. The AI ​​live streaming and intelligent new retail linkage system according to claim 9, characterized in that, The system also includes: The transaction synchronization module is used to present the customized transaction plan to the audience in the virtual display environment and open a one-click order channel. After capturing the audience's confirmation information, it synchronizes the order data, audience identity identifier, and the exclusive discount information to the new retail order fulfillment system, specifically including: In the interactive interface of the virtual display environment, detailed information about the customized transaction plan is displayed in a prominent pop-up window; A confirmation button is provided in the pop-up window interface. The confirmation button is associated with a one-click order instruction. When the viewer clicks it, it is considered as confirmation. Upon capturing a click event on the confirmation button, order data is encapsulated. The order data includes the product number of the popular product, the quantity purchased, the price in the customized transaction plan, the exclusive discount information, and the selected delivery option in the logistics forecast information. Obtain the identity identifier of the currently logged-in viewer and bind the order data to the viewer identity identifier; The bound complete order data package is pushed to the new retail order fulfillment system in real time through an encrypted interface, and the new retail order fulfillment system completes the inventory pre-positioning and payment process initialization operations.

Images