A social shopping method and system based on virtual avatar synchronous rendering and real-time audio and video communication

Abstract

This invention relates to the fields of internet e-commerce and computer graphics processing technology, specifically a social shopping guide method and system based on virtual avatar synchronous rendering and real-time audio and video communication. The invention establishes a global state machine on the client side to persistently store the coordinates, actions, and appearance data of the virtual avatar; it achieves unobstructed following by superimposing offset vectors on touch coordinates, and uses interpolation algorithms and boundary detection to achieve smooth animation; it divides the logical space based on AOI (Area of ​​Interest) regions, and uses incremental synchronization + dead reckoning to reduce bandwidth and device load for multiple users on the same screen; it establishes a hierarchical voice communication channel distinguishing between ordinary users and operation specialists, automatically synchronizing the user's product browsing context during shopping guide calls, and combining AS (Autopilot) speech recognition to enable voice commands to directly trigger front-end transaction UI interactions. This invention solves the shortcomings of existing e-commerce virtual avatar page resets, finger occlusion, lag during multi-user synchronization, low efficiency of text communication, and stiff shopping guide promotions, creating a continuous and immersive cloud shopping social scene.

Description

Technical Field

[0001] This invention relates to the fields of internet e-commerce and computer graphics processing technology, specifically to a social shopping guide method and system based on virtual avatar synchronous rendering and real-time audio and video communication. Background Technology

[0002] Current mainstream mobile e-commerce platforms use static images and text, short videos, and pop-up live streams as product display methods. Online shopping is isolated and lacks the immersive social shopping experience of offline malls. Existing e-commerce solutions that incorporate virtual digital avatars and voice customer service suffer from four core technical flaws:

[0003] The virtual avatar interaction is fragmented, resulting in a lack of a continuous shopping experience. Existing e-commerce virtual avatars can only be rendered on a single page, and the model is directly destroyed and reset when jumping from the product list to the store or details page. Traditional solutions directly bind the virtual avatar to the finger touch coordinates, and the finger occludes the model during operation, resulting in a poor visual experience. There is no independent global state storage structure, and the avatar's position, clothing, and actions cannot be retained across pages, making it impossible to form a coherent cloud shopping experience.

[0004] Multi-user simultaneous screen synchronization incurs huge overhead, causing lag and overheating on low-end phones. Traditional multi-user virtual avatar synchronization uses full data packet broadcasting, which causes a surge in network bandwidth and client rendering pressure as the number of online users increases. Furthermore, the lack of a trajectory compensation mechanism for network jitter results in teleportation and screen shaking when remote users move, and low-end phones are prone to memory overflow and crashes.

[0005] The interactive format of the shopping guide is too simple, resulting in low conversion efficiency. Regular customer service only provides text consultation channels, and the cost of communicating complex discount rules such as order combining, discounts, and shipping fees in text is extremely high. Promotions rely on pop-ups and banner pushes, which are very easy to arouse user resentment. There is no distinction between the voice permissions of ordinary users and operation specialists, making it impossible to realize group order communication among users, operation-wide broadcasting, and targeted and precise shopping guidance.

[0006] Voice communication is completely disconnected from product business. Traditional voice customer service only realizes simple voice conversation and cannot automatically capture the SKU currently viewed by the user or the context of the activity; voice dialogue cannot be linked to coupons or bundled purchase interfaces, and voice communication cannot directly trigger transaction operations, resulting in a break in the business link from sales guide communication to order placement. Summary of the Invention

[0007] (a) Technical problems to be solved

[0008] To address the shortcomings of existing technologies, this invention provides a social shopping guide method and system based on virtual avatar synchronous rendering and real-time audio and video communication.

[0009] (II) Technical Solution

[0010] To achieve the above objectives, the present invention provides the following technical solution: a social shopping guide method based on virtual avatar synchronous rendering and real-time audio and video communication, comprising the following steps:

[0011] Step 1, Virtual Avatar Initialization and Global State Persistence: Load the custom virtual avatar 3D model in the client rendering engine and create an independent global state machine in memory; the global state machine is dedicated to storing the virtual avatar's real-time coordinates, standing / walking action state, hairstyle / clothing / handheld prop appearance configuration data. The lifecycle of the state machine is independent of any product page, and the state data is not cleared when the page is destroyed.

[0012] Step 2, Unobstructed Touch Coordinate Calculation and Boundary Collision Handling: Capture screen touch swipe events in real time and extract the absolute coordinates of finger touch; superimpose a preset fixed relative offset vector (e.g., 30px from the upper right of the screen) to calculate the target coordinates of the virtual avatar, avoiding finger occlusion of the model; if the target coordinates exceed the preset UI visible boundary, trigger edge snapping or bounce animation to ensure that the avatar is always displayed within the screen.

[0013] Step 3, Smooth Motion Rendering and Network Latency Compensation: Calculate the difference between the avatar's current coordinates and the target coordinates using a fixed rendering frame rate of 60FPS. Generate a continuous and smooth motion trajectory using linear interpolation (Lerp) or Bézier curve algorithm. To address network latency, a dead reckoning method is introduced. The client predicts the motion path locally based on the avatar's speed and direction in the previous frame. After receiving the server's actual coordinate data, the path is gradually smoothed to eliminate instantaneous jitter.

[0014] Step 4: Lightweight multi-user synchronization based on AOI (Area of ​​Interest): The e-commerce platform is divided into independent logical spaces based on individual stores, product detail pages, and LBS (Location Based on Location) locations. After a user enters a space, they send a subscription request to the state synchronization server. The server only sends incremental change data of other user avatars within that space (only pushes data when location or action changes). When a user leaves the logical space, the subscription is automatically canceled, and the client clears the local cache of external avatar data, reducing bandwidth and rendering overhead.

[0015] Step 5: Based on dual-role RTC real-time voice communication scheduling, establish a two-level signaling routing mechanism for ordinary users and operation specialists: ① Calls initiated between ordinary users: establish a P2P or server relay voice channel, and broadcast the call visual icons of both parties to users in the same space; ② Global broadcast by operation specialists: establish a one-to-many voice stream and push shopping guide voice to all users in the current logical space; ③ Targeted shopping guide by operation specialists: initiate one-to-one exclusive voice calls for users who stay for a long time.

[0016] Step 6: Intelligent Linkage Between Voice Shopping Guide and Product Business Context. After establishing a one-to-one voice channel with the user, the system automatically captures structured contextual data such as the user's currently viewed SKU, store discounts, and delivery fee reductions and pushes it to the operations client. During the call, the system uses ASR voice recognition to parse the operations script, identifies discount keywords, and automatically pops up coupon and bundle recommendation overlays on the user's front end. Voice directly triggers the transaction UI interaction.

[0017] Furthermore, the virtual avatar adopts a modular dress-up design, with headwear, tops, bottoms, and props loaded independently. All appearance configurations are written into the global state machine, and the appearance remains unchanged after crossing pages or logging in again with the account.

[0018] A social shopping guide system based on virtual avatar synchronous rendering and real-time audio and video communication

[0019] The four local modules on the client side are used to execute the above social shopping guide methods:

[0020] Client-side rendering module: Loads virtual avatar models, listens for and captures screen touch events, performs coordinate offset calculations, interpolation smoothing animations, boundary collision rendering, and local AS speech recognition;

[0021] Global State Management Module: Built-in global state machine, unified reading and writing of avatar coordinates, actions, and appearance data, and state reading and reuse when switching pages to achieve seamless cross-page transition;

[0022] State synchronization server: Divides the AOI logical space, handles client subscription / unsubscription requests, and only distributes avatar incremental change data;

[0023] Audio and video communication server: Implements hierarchical role signaling routing, establishes three types of voice channels: P2P, global broadcast, and one-to-one, and connects with the product business system to synchronize page context.

[0024] (III) Beneficial Effects

[0025] Compared with existing technologies, this invention provides a social shopping guide method and system based on virtual avatar synchronous rendering and real-time audio and video communication, which has the following beneficial effects:

[0026] This social shopping guide method and system based on virtual avatar synchronous rendering and real-time audio and video communication stores all avatar data through an independent global state machine, so the image is not reset when the page jumps; the touch coordinate superposition offset vector completely solves the problem of finger occlusion; the interpolation smoothing algorithm eliminates sliding teleportation, and the boundary collision prevents the model from sliding off the screen, which greatly improves the comfort of operation and extends the user's stay time in the APP;

[0027] Independent logical spaces are split by store / details page, only incremental avatar data that changes are synchronized, and invalid data is automatically cleaned up; local prediction trajectory compensation is used when there is network jitter, without the need to broadcast all user information, which significantly reduces network bandwidth usage, and even low-end mobile phones can smoothly enable multiple people to shop on the same screen, solving the problems of lag, overheating and crashing.

[0028] Differentiate communication permissions between regular users and operations specialists: users can negotiate group orders and verify sizes via voice; operations specialists support full-site voice promotions and targeted one-on-one shopping guidance, which is less resistant to pop-up ads and proactively guides users to high-priced items, thereby increasing exposure and conversion.

[0029] One-on-one calls automatically synchronize users' browsing of product and promotional information, making operational communication more targeted; ASR analyzes voice keywords in real time and automatically pops up the corresponding coupon interface, saving users the step of manually searching for activities, greatly reducing the cost of complex promotional communication, and improving order conversion rate;

[0030] Virtual avatars are modularized and can be freely customized by users. Configurations are stored in a unified global state machine, and the avatar is not lost when crossing pages or logging back into the account, achieving a personalized look for each user and enhancing the social atmosphere of online shopping. Detailed Implementation

[0031] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0032] Example 1: Complete Execution Process of Mobile Social Shopping Guide:

[0033] Step 1, Virtual Avatar Initialization:

[0034] When a user opens an e-commerce app, the client-side rendering engine loads modular virtual avatar 3D resources; a global state machine is created in an independent area of ​​memory, and the coordinates, actions, and clothing blank data are initialized and stored. The lifecycle of the state machine is not limited by the lifecycle of the product page.

[0035] Step 2, Unobstructed Touch Tracking and Boundary Control:

[0036] The client monitors screen swipe touches in real time to obtain the absolute coordinates (Xtouch, Ytouch) of the touch point; it overlays preset offset vectors ΔX=30px and ΔY=30px to calculate the target rendering coordinates Xtarget=Xtouch+ΔX and Ytarget=Ytouch+ΔY; it checks whether the target coordinates exceed the safe UI boundaries of the screen (top, bottom, left, right). If the distance exceeds the boundary by a small amount, it performs edge snapping; if it exceeds the threshold, it plays a bounce animation to ensure that the avatar is always in the visible area.

[0037] Step 3, Smooth Movement and Network Latency Compensation:

[0038] Set a fixed rendering frame rate of 60FPS. Calculate the difference between the current position and the target coordinates in each frame and use Bézier curves to generate continuous motion animation. The client locally caches the previous frame to represent the movement speed and direction. When the server data arrives late, the dead reckoning method is used to predict the coordinates of the intermediate frames. After the server receives the actual data packets, a gradual correction method is used to smoothly correct them without abrupt jumps.

[0039] Step 4: Lightweight AOI multi-user synchronization:

[0040] The backend divides each product detail page and independent store into an independent AOI logical space; when a user enters a product page, the client sends a space subscription command to the status synchronization server; the server only pushes incremental change data of other users in that space (data packets are only sent when the location or action changes); when a user exits the page, an unsubscribe request is automatically sent, and the client clears the external avatar cache to reduce rendering pressure.

[0041] Step 5: Hierarchical voice communication scheduling:

[0042] User interaction scenario: When a user clicks on another user's virtual avatar to initiate a call, the server establishes a P2P or trunk voice channel, and all users in the same space display the call icon between the two parties, which can be used for group buying and inquiring about sizes.

[0043] Full-domain broadcast scenario: When an operating account holds high-level signaling privileges, clicking the broadcast control will push a shopping guide voice message to all users in the current space, and users can mute it with one click;

[0044] The targeted shopping guide system automatically identifies users who linger on a single product for more than 30 seconds, and the operations team initiates a one-on-one exclusive voice message.

[0045] Step 6: Voice-product context linkage:

[0046] The moment a one-on-one voice channel is established, the server automatically captures structured data such as the user's currently viewed SKUs, shopping cart amount, discounts, and shipping rules, and pushes it to the operations client. During the call, the client's local ASR continuously parses the operations voice, recognizes keywords such as "¥30 off for orders over ¥200" and "free shipping", and automatically pops up coupon redemption and add-on recommendations on the user's front end, with voice directly driving the transaction interaction.

[0047] Step 7: Seamless cross-page transition

[0048] When a user navigates from the product list to the details page, the current page rendering instance is destroyed, but the coordinates, appearance, and action data within the global state machine are fully preserved. After the new page loads, the rendering module reads the state machine data, automatically avoids core areas such as the product main image and the purchase button, and reconstructs the virtual avatar, resulting in uninterrupted visual experience and a continuous shopping experience.

[0049] Example 2: A social shopping guide system based on virtual avatar synchronous rendering and real-time audio and video communication, comprising the following modules:

[0050] Client-side rendering module: responsible for loading virtual avatar models, capturing touch events, calculating coordinate offsets, rendering linear / Bezier interpolation animations, rendering boundary bounce animations, and local AS speech recognition;

[0051] Global State Management Module: Independent in-memory global state machine, unified reading and writing of avatar coordinates, actions, and clothing configurations, and reuse of state data when switching pages;

[0052] State synchronization server: Divides the AOI logical space, handles subscription / unsubscription, filters static and unchanged avatar data, and only sends incremental data packets;

[0053] Audio and video communication server: Differentiates between user / operation roles to complete signaling scheduling, establishes three types of voice channels, and connects to the product and service synchronization page context.

[0054] In this embodiment, the computer-readable storage medium can be any program storage carrier such as a USB flash drive, hard disk, mobile phone ROM, or cloud storage, and the medium stores an executable program; when the program is executed by a mobile terminal processor such as a mobile phone or tablet, all the method steps of Embodiment 1 are implemented.

[0055] In this document, the term "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The term "embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment, nor does it specifically limit its independence or connection with other embodiments. In principle, in this application, as long as there are no technical contradictions or conflicts, the technical features mentioned in each embodiment can be combined in any way to form corresponding implementable technical solutions.

[0056] Unless otherwise defined, the technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the use of related terms herein is merely for the purpose of describing particular embodiments and is not intended to limit this application.

[0057] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention.

Claims

1. A social shopping guide method based on virtual avatar synchronous rendering and real-time audio and video communication, characterized in that, include: Step 1: Load the current user's virtual avatar model in the client rendering engine and establish an independent global state machine. The global state machine is independent of the lifecycle of each business page and is used to record the real-time coordinates, action status and appearance configuration data of the virtual avatar. Step 2: Capture the user's touch and swipe events on the screen in real time, extract the touch point coordinates and combine them with a preset relative offset vector to calculate the target coordinates of the virtual avatar; Step 3: Use a smooth interpolation algorithm to drive the virtual avatar to move towards the target coordinates with an animation trajectory, and when the client crosses different business pages, use the global state machine to achieve seamless transition of the virtual avatar's position and state; Step 4: Based on the region of interest algorithm, obtain the virtual avatar data of other online users in the current business space through the state synchronization server, and perform screen rendering on the client. Step 5: In response to role-based real-time audio and video communication requests, establish corresponding voice communication channels to enable real-time voice-guided shopping interactions between users or between operators and users.

2. The social shopping guide method based on virtual avatar synchronous rendering and real-time audio and video communication according to claim 1, characterized in that, The step of extracting the touch point coordinates and combining them with a preset relative offset vector to calculate the target coordinates of the virtual avatar specifically includes: Get the absolute coordinates of the user's touch swipe events on the screen; The absolute coordinates are superimposed with a preset relative offset to generate the target coordinates of the virtual avatar, so as to avoid the virtual avatar being obscured by the user's finger; When the target coordinates exceed the preset screen UI boundary, the boundary collision detection logic is triggered, and the virtual avatar is snapped to the edge or a bounce animation is executed.

3. The social shopping guide method based on virtual avatar synchronous rendering and real-time audio and video communication according to claim 1, characterized in that, The step of using a smooth interpolation algorithm to drive the virtual avatar to move toward the target coordinates along an animated trajectory specifically includes: The difference between the current position of the virtual avatar and the target coordinates is calculated at a preset frame rate; Using a linear interpolation algorithm or a Bézier curve algorithm, a smooth movement trajectory of the virtual avatar in the current frame is generated based on the difference. When network latency causes the virtual avatar's position to deviate, the dead reckoning method is used to predict the movement trajectory locally based on the speed and direction of the previous moment, and then a smooth correction is made after receiving the actual data from the server.

4. The social shopping guide method based on virtual avatar synchronous rendering and real-time audio and video communication according to claim 1, characterized in that, The interest region-based algorithm obtains virtual avatar data of other online users within the current business space through a state synchronization server, specifically including: The shopping platform is divided into multiple logical spaces based on stores, product detail pages, or geographical locations; When a user enters a certain logical space, a space subscription request is sent to the state synchronization server; The state synchronization server only sends incremental data of virtual avatars of other online users within the logical space to the client; When the user leaves the logical space, the subscription is canceled and the corresponding avatar data in local memory is cleared.

5. The social shopping guide method based on virtual avatar synchronous rendering and real-time audio and video communication according to claim 1, characterized in that, The establishment of a corresponding voice communication channel in response to a role-based real-time audio and video communication request specifically includes: Establish a role-based signaling routing control mechanism, whereby the roles include ordinary users and operations specialists; When a communication request is received from a regular user, a point-to-point or server relay voice communication channel is established, and the call visual status is broadcast to users in the same space. When a communication request is received from an operations specialist, a broadcast voice channel for all users or a targeted one-on-one voice shopping guide channel for a specific user is established according to the operations specialist's instructions.

6. The social shopping guide method based on virtual avatar synchronous rendering and real-time audio and video communication according to claim 5, characterized in that, After establishing a targeted one-on-one voice shopping guide channel for specific users, the following is also included: Automatically obtain the context data of the business page currently being viewed by the specific user, including the currently viewed product SKU, store discount activities, and delivery fee reduction rules; The context data is pushed as structured information to the operations specialist's client. During voice communication, the voice commands of the operations specialist are parsed using voice recognition technology, and the corresponding transaction link UI interaction is triggered on the client of the specific user.

7. A social shopping guide system based on virtual avatar synchronous rendering and real-time audio and video communication, used to execute the social shopping guide method based on virtual avatar synchronous rendering and real-time audio and video communication as described in any one of claims 1-6, characterized in that, include: The client-side rendering module is used to load the virtual avatar model, capture touch swipe events, and drive the virtual avatar movement using a smooth interpolation algorithm; The global state management module is used to record the real-time coordinates, action status, and appearance configuration data of the virtual avatar, and to achieve seamless state transitions when the page jumps. A state synchronization server is used to distribute incremental virtual avatar data of other online users in the current logical space to the client based on the region of interest algorithm. The audio and video communication server is used to respond to role-based real-time audio and video communication requests and establish corresponding voice communication channels to enable real-time voice-guided shopping interactions between users or between operators and users.

8. A social shopping guide system based on virtual avatar synchronous rendering and real-time audio and video communication according to claim 7, characterized in that, A computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the social shopping guide method based on virtual avatar synchronous rendering and real-time audio and video communication as described in any one of claims 1 to 6.