Virtual e-commerce customer service interaction method, electronic device and computer readable storage medium

Abstract

The application provides a virtual e-commerce customer service interaction method, an electronic device and a computer readable storage medium, which are used for providing an interaction function by using a virtual e-commerce customer service. The method comprises the following steps: displaying a virtual e-commerce customer service interaction interface by using a terminal device; acquiring input information of a user by using the terminal device; playing one or more preset interaction videos of the virtual e-commerce customer service by using the terminal device based on the input information, wherein the preset interaction videos are used for collecting user demand information; and playing one or more recommended interaction videos of the virtual e-commerce customer service by using the terminal device based on the user demand information, wherein the recommended interaction videos are used for introducing recommended products. The application can save the communication cost of e-commerce customer service and users, improve the accuracy of product recommendation of e-commerce customer service, improve the service quality, improve the satisfaction and loyalty of users, and reduce the operation cost.

Description

Technical Field

[0001] This application relates to the technical fields of virtual humans and artificial intelligence, and in particular to virtual e-commerce customer service interaction methods, electronic devices, and computer-readable storage media. Background Technology

[0002] Virtual objects include virtual humans, virtual animals, and virtual cartoon characters. Virtual humans, in particular, are anthropomorphic figures constructed using CG technology and operating in code form, possessing various interactive capabilities such as language communication, facial expressions, and gesture demonstrations. Virtual human technology has rapidly developed in the field of artificial intelligence and has been applied in many technological areas, including film, media, games, finance, cultural tourism, education, and healthcare. One of the most common applications is virtual customer service, which uses virtual humans to handle customer inquiries and complaints in place of human customer service personnel. Virtual customer service can provide 24-hour service and can automatically translate according to customer needs and language.

[0003] Existing human customer service is costly, and during peak hours, waiting times are too long, leading to user churn or dissatisfaction. Furthermore, existing e-commerce customer service cannot understand user input, and customer service robots provide unnatural, mechanical, template-based, and impersonal responses.

[0004] Based on this, this application provides a virtual e-commerce customer service interaction method, an electronic device, and a computer-readable storage medium to improve related technologies. Summary of the Invention

[0005] The purpose of this application is to provide virtual e-commerce customer service interaction methods, electronic devices, and computer-readable storage media to save communication costs between e-commerce customer service and users, improve the accuracy of product recommendations by e-commerce customer service, enhance service quality, increase user satisfaction and loyalty, and reduce operating costs.

[0006] The objective of this application is achieved through the following technical solution:

[0007] Firstly, this application provides a virtual e-commerce customer service interaction method for providing interactive functions using virtual e-commerce customer service, the method comprising:

[0008] The virtual e-commerce customer service interaction interface is displayed using a terminal device;

[0009] The terminal device is used to obtain user input information, which includes one or more of first text information, voice information, and / or image information;

[0010] Based on the input information, the terminal device is used to play one or more preset interactive videos of the virtual e-commerce customer service, the preset interactive videos being used to collect user demand information;

[0011] Based on the user demand information, the terminal device is used to play one or more recommended interactive videos of the virtual e-commerce customer service, which are used to introduce recommended products.

[0012] The beneficial effects of this technical solution are as follows: Users access the virtual e-commerce customer service interactive interface through a terminal device within an application with virtual e-commerce customer service capabilities. They input one or more of the following: text information, voice information, and / or image information, to express their needs. Based on the user's input, the virtual e-commerce customer service can play one or more preset interactive videos on the terminal device to help users express their needs more accurately and collect user demand information. Based on the collected demand information, the virtual e-commerce customer service can play one or more recommended interactive videos to introduce recommended products that meet the user's needs and guide the user to make a purchase. On the one hand, by providing a virtual e-commerce customer service interface, users can more conveniently obtain the information they need without having to queue for human customer service during peak hours, which helps improve user satisfaction and loyalty. On the other hand, by transforming the way e-commerce recommends products, virtual e-commerce customer service can collect user needs information in a short time by using pre-set interactive videos based on user input, and then provide corresponding product recommendations. This saves communication costs and time between e-commerce customer service and users, conserves customer service resources, and allows for simple and quick understanding of user needs, improving overall service efficiency. Furthermore, by introducing recommended products through interactive videos based on accurately and targeted user needs, the product recommendations are more objective and free from the subjective emotions of human customer service. Interactive videos also solve problems such as unnatural, mechanical, templated, and impersonal customer service robot presentations, enhancing user trust in recommended products and increasing their desire to purchase, thus improving the service quality of virtual e-commerce customer service. Finally, virtual e-commerce customer service can provide uninterrupted 24 / 7 service without incurring additional costs, reducing e-commerce operating costs.

[0013] In some optional implementations, the method further includes:

[0014] Use the terminal device to obtain at least two comparison products specified by the user;

[0015] Based on at least two of the comparison products, obtain product information for at least two of the comparison products;

[0016] Based on at least two pieces of product information, obtain at least two differences between the comparative products;

[0017] The difference information is processed by speech synthesis to obtain synthesized speech based on the difference information.

[0018] The terminal device is used to play the synthesized voice of the difference information using the virtual e-commerce customer service.

[0019] The beneficial effects of this technical solution are as follows: Users can select and input at least two products for comparison via a terminal device to inquire about their advantages, disadvantages, and differences with virtual e-commerce customer service. Based on the user's input of at least two products, the virtual e-commerce customer service obtains product information (such as price, functions, specifications, etc.), analyzes and compares the differences between these products, extracts the difference information, and synthesizes it into a synthesized speech. This synthesized speech is then played on the terminal device by the virtual e-commerce customer service, allowing users to more intuitively understand the differences between the products. On the one hand, users can understand the differences between any two or more products through virtual e-commerce customer service, thus better choosing the products that suit them and enhancing user satisfaction. On the other hand, generating natural language from the differences through speech synthesis technology and then playing the synthesized speech by the virtual e-commerce customer service solves the problem of mechanical and templated responses from robot customer service, making it easier for users to understand the differences between the products. Users can more accurately choose the products that suit them, thereby increasing product sales. In addition, virtual e-commerce customer service can solve users' product comparison problems in a short time, improving the response efficiency of e-commerce customer service and reducing time and resource costs.

[0020] In some optional implementations, obtaining difference information between at least two of the comparison products based on the product information includes:

[0021] The product information is input into the difference detection model to obtain difference information for at least two of the comparison products;

[0022] The training process of the difference detection model includes:

[0023] Obtain a training set, which includes multiple sets of training data. Each set of training data includes at least two sample product information and labeled data of the difference information corresponding to at least two sample product information.

[0024] For each set of training data, perform the following processing:

[0025] Input at least two sample product information from the training data into a preset deep learning model to obtain predicted data of the difference information corresponding to at least two sample product information.

[0026] The model parameters of the deep learning model are updated based on the predicted data and labeled data of the difference information corresponding to at least two sample product information.

[0027] The system checks whether the preset training termination condition is met; if so, the trained deep learning model is used as the difference detection model; if not, the system continues to train the deep learning model using the next set of training data.

[0028] The beneficial effects of this technical solution are as follows: by comparing the differences between products through deep learning algorithms, the differences between at least two products provided by the user can be identified more quickly and accurately; and by automatically identifying product differences through virtual e-commerce customer service, the burden and cost of e-commerce customer service can be reduced.

[0029] In some optional implementations, the step of playing one or more preset interactive videos of the virtual e-commerce customer service using the terminal device based on the input information includes:

[0030] When the input information includes the first text information, keywords from the first text information are extracted. Based on the keywords of the first text information, one or more preset interactive videos matching the keywords of the first text information are searched from the question-and-answer database matched by the virtual e-commerce customer service. The terminal device is then used to play one or more preset interactive videos matching the keywords of the first text information; and / or,

[0031] When the input information includes the voice information, the voice information is recognized, converted into second text information, keywords are extracted from the second text information, and based on the keywords, one or more preset interactive videos matching the keywords in the question-and-answer database are searched. The terminal device is then used to play one or more preset interactive videos matching the keywords in the second text information; and / or,

[0032] When the input information includes the image information, the image information is identified, the image information is converted into third text information, keywords of the third text information are extracted, and based on the keywords of the third text information, one or more preset interactive videos that match the keywords of the third text information are searched from the question-and-answer database. The terminal device is then used to play one or more preset interactive videos that match the keywords of the third text information.

[0033] The beneficial effects of this technical solution are as follows: User input information can include one or more of the following: text information, voice information, and image information. The system recognizes the voice or image information, converts it into text information, extracts keywords corresponding to each text element, and searches for matching preset interactive videos in a question-and-answer database based on these keywords for playback. On the one hand, it enriches the forms of user input information, allowing users greater freedom to interact with virtual e-commerce customer service in multiple ways. On the other hand, by automatically matching relevant preset interactive videos with key content extracted from the input information, virtual e-commerce customer service can quickly respond to user questions or other input information, improving the interaction efficiency of e-commerce customer service. Furthermore, through preset interactive video services, targeted interaction with users is conducted to collect user needs information for better user service, enhancing the user experience. In addition, automatically matching preset interactive videos with virtual e-commerce customer service reduces the burden and cost of e-commerce customer service.

[0034] In some optional implementations, the step of playing one or more recommended interactive videos of the virtual e-commerce customer service using the terminal device based on the user demand information includes:

[0035] From the question-and-answer database matched by the virtual e-commerce customer service, obtain one or more recommended interactive videos that match the user's needs; or,

[0036] Obtain the recommended text and virtual human actions corresponding to the user's demand information; drive the virtual e-commerce customer service's facial expressions and lip movements according to the recommended text, and drive the virtual e-commerce customer service's actions according to the virtual human actions, so as to generate the recommended interactive video.

[0037] The beneficial effects of this technical solution are as follows: Based on the user's input demand information, one or more recommended interactive videos matching the user's demand information are obtained from the question and answer database matched by the virtual e-commerce customer service to display relevant recommended products to the user. By automatically matching relevant recommended interactive videos, the virtual e-commerce customer service can quickly respond to user needs and improve interaction efficiency; it can also generate recommended text and virtual human actions based on user demand information, drive the virtual e-commerce customer service's facial expressions and lip movements and add actions to generate recommended interactive videos, provide more customized recommended interactive services, respond to user needs, solve the problem of mechanical and templated robot customer service replies, help users better understand recommended products, and reduce customer service costs.

[0038] In some optional implementations, the method further includes:

[0039] The terminal device is used to obtain feedback information from the user, and the feedback information is used to evaluate the accuracy of the recommended interactive video.

[0040] Based on the feedback information, update the question-and-answer database; and / or update the recommended text.

[0041] The beneficial effects of this technical solution are as follows: It utilizes terminal devices to obtain user feedback on recommended interactive videos, such as whether the videos match user needs and the accuracy of the recommendations. Based on this feedback, the matching question-and-answer database and recommended text are updated to improve matching accuracy. On the one hand, by collecting and analyzing user feedback, the question-and-answer database and recommended text of the virtual e-commerce customer service can be adjusted in a timely manner, improving recommendation accuracy and better meeting user needs. Continuously optimized virtual e-commerce customer service provides more accurate and customized interactive services, enhancing user experience and satisfaction, thereby increasing user loyalty and purchasing power. On the other hand, by collecting and analyzing user feedback, it is also possible to better understand user needs and provide e-commerce with more precise market operation strategies.

[0042] In some optional implementations, the method further includes:

[0043] The terminal device is used to conduct video interaction with the user using the virtual e-commerce customer service;

[0044] During the video interaction, the user's facial information is obtained;

[0045] Based on the portrait information, obtain the user's current facial expression information and / or action information;

[0046] Based on the facial expression information and / or the action information, obtain the user's emotional information;

[0047] Based on the emotional information, it is determined whether the user is satisfied with the recommended product.

[0048] The beneficial effects of this technical solution are as follows: By using terminal devices to interact with users via video through virtual e-commerce customer service, user facial information can be obtained, and the user's current facial expressions and actions can be analyzed to determine the user's emotional state, thereby assessing the user's satisfaction with recommended products; By monitoring user emotional information in real time, virtual e-commerce customer service can better understand user emotions, indirectly infer user needs, adjust service strategies in a timely manner, and improve service quality; Customized services enhance user experience and satisfaction.

[0049] In some optional implementations, the method further includes:

[0050] When it is detected that the user is satisfied with the recommended product, continue playing a recommended interactive video that is similar to the recommended interactive video;

[0051] When it is detected that the user is not satisfied with the recommended product, the recommended interactive video is replaced.

[0052] The beneficial effects of this technical solution are as follows: when user satisfaction with the recommended product is detected, a video similar to the recommended interactive video will continue to be played; when user dissatisfaction is detected, the recommended interactive video will be adjusted in a timely manner to improve user experience, meet user needs, and increase product sales; and by continuously optimizing and improving customized services, user loyalty will be increased, and the competitive advantage of e-commerce will be expanded.

[0053] Secondly, this application provides that the electronic device includes a memory and at least one processor, the memory storing a computer program, and the at least one processor being configured to execute the computer program to perform the following steps:

[0054] The virtual e-commerce customer service interaction interface is displayed using a terminal device;

[0055] The terminal device is used to obtain user input information, which includes one or more of first text information, voice information, and / or image information;

[0056] Based on the input information, the terminal device is used to play one or more preset interactive videos of the virtual e-commerce customer service, the preset interactive videos being used to collect user demand information;

[0057] Based on the user demand information, the terminal device is used to play one or more recommended interactive videos of the virtual e-commerce customer service, which are used to introduce recommended products.

[0058] In some alternative implementations, the at least one processor is further configured to perform the following steps when executing the computer program:

[0059] Use the terminal device to obtain at least two comparison products specified by the user;

[0060] Based on at least two of the comparison products, obtain product information for at least two of the comparison products;

[0061] Based on at least two pieces of product information, obtain at least two differences between the comparative products;

[0062] The difference information is processed by speech synthesis to obtain synthesized speech based on the difference information.

[0063] The terminal device is used to play the synthesized voice of the difference information using the virtual e-commerce customer service.

[0064] In some alternative implementations, the at least one processor is further configured to, when executing the computer program, acquire difference information of at least two of the comparative products based on the product information in the following manner:

[0065] The product information is input into the difference detection model to obtain difference information for at least two of the comparison products;

[0066] The training process of the difference detection model includes:

[0067] Obtain a training set, which includes multiple sets of training data. Each set of training data includes at least two sample product information and labeled data of the difference information corresponding to at least two sample product information.

[0068] For each set of training data, perform the following processing:

[0069] Input at least two sample product information from the training data into a preset deep learning model to obtain predicted data of the difference information corresponding to at least two sample product information.

[0070] The model parameters of the deep learning model are updated based on the predicted data and labeled data of the difference information corresponding to at least two sample product information.

[0071] The system checks whether the preset training termination condition is met; if so, the trained deep learning model is used as the difference detection model; if not, the system continues to train the deep learning model using the next set of training data.

[0072] In some alternative implementations, the at least one processor is further configured to, when executing the computer program, play one or more preset interactive videos of the virtual e-commerce customer service using the terminal device based on the input information:

[0073] When the input information includes the first text information, keywords from the first text information are extracted. Based on the keywords of the first text information, one or more preset interactive videos matching the keywords of the first text information are searched from the question-and-answer database matched by the virtual e-commerce customer service. The terminal device is then used to play one or more preset interactive videos matching the keywords of the first text information; and / or,

[0074] When the input information includes the voice information, the voice information is recognized, converted into second text information, keywords are extracted from the second text information, and based on the keywords, one or more preset interactive videos matching the keywords in the question-and-answer database are searched. The terminal device is then used to play one or more preset interactive videos matching the keywords in the second text information; and / or,

[0075] When the input information includes the image information, the image information is identified, the image information is converted into third text information, keywords of the third text information are extracted, and based on the keywords of the third text information, one or more preset interactive videos that match the keywords of the third text information are searched from the question-and-answer database. The terminal device is then used to play one or more preset interactive videos that match the keywords of the third text information.

[0076] In some alternative implementations, the at least one processor is further configured to, when executing the computer program, play one or more recommended interactive videos of the virtual e-commerce customer service using the terminal device based on the user demand information:

[0077] From the question-and-answer database matched by the virtual e-commerce customer service, obtain one or more recommended interactive videos that match the user's needs; or,

[0078] Obtain the recommended text and virtual human actions corresponding to the user's demand information; drive the virtual e-commerce customer service's facial expressions and lip movements according to the recommended text, and drive the virtual e-commerce customer service's actions according to the virtual human actions, so as to generate the recommended interactive video.

[0079] In some alternative implementations, the at least one processor is further configured to perform the following steps when executing the computer program:

[0080] The terminal device is used to obtain feedback information from the user, and the feedback information is used to evaluate the accuracy of the recommended interactive video.

[0081] Based on the feedback information, update the question-and-answer database; and / or update the recommended text.

[0082] In some alternative implementations, the at least one processor is further configured to perform the following steps when executing the computer program:

[0083] The terminal device is used to conduct video interaction with the user using the virtual e-commerce customer service;

[0084] During the video interaction, the user's facial information is obtained;

[0085] Based on the portrait information, obtain the user's current facial expression information and / or action information;

[0086] Based on the facial expression information and / or the action information, obtain the user's emotional information;

[0087] Based on the emotional information, it is determined whether the user is satisfied with the recommended product.

[0088] In some alternative implementations, the at least one processor is further configured to perform the following steps when executing the computer program:

[0089] When it is detected that the user is satisfied with the recommended product, continue playing a recommended interactive video that is similar to the recommended interactive video;

[0090] When it is detected that the user is not satisfied with the recommended product, the recommended interactive video is replaced.

[0091] Thirdly, this application provides a computer-readable storage medium storing a computer program that is implemented when executed by at least one processor.

[0092] The steps of any of the above methods or the functions of any of the above electronic devices. Attached Figure Description

[0093] The present application will be further described below with reference to the accompanying drawings and embodiments.

[0094] Figure 1 This is a flowchart illustrating a virtual e-commerce customer service interaction method provided in an embodiment of this application;

[0095] Figure 2 This is a flowchart illustrating another virtual e-commerce customer service interaction method provided in an embodiment of this application;

[0096] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0097] Figure 4 This is a schematic diagram of the structure of a program product for implementing a virtual e-commerce customer service interaction method, provided in an embodiment of this application. Detailed Implementation

[0098] The present application will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0099] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, a and b and c, where a, b, and c can be single or multiple. It is worth noting that "at least one" can also be interpreted as "one or more".

[0100] It should also be noted that, in the embodiments of this application, the words "exemplary" or "for example" are used to indicate that they are examples, illustrations, or descriptions. Any implementation or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other implementations or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0101] The technical field and related terms of the embodiments of this application are briefly described below.

[0102] Virtual objects include virtual humans, virtual animals, and virtual cartoon characters. Virtual humans, in particular, are anthropomorphic figures constructed using CG technology and operating in code form, possessing various interactive capabilities such as language communication, facial expressions, and action demonstrations. Virtual human technology has rapidly developed in the field of artificial intelligence and has been applied in many technological areas, including film, media, games, finance, culture and tourism, education, and healthcare. It allows for the customization of virtual hosts, virtual anchors, virtual idols, virtual customer service representatives, virtual lawyers, virtual financial advisors, virtual doctors, virtual tour guides, and virtual assistants, and can also generate videos with a single click from text or audio. Among virtual humans, service-oriented virtual humans primarily function to replace real-life services and provide daily companionship; they are virtualizations of real-world service roles. Their industrial value mainly lies in reducing costs in existing service industries, thereby improving efficiency and reducing costs in the existing market.

[0103] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities. AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0104] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. A computer program can learn experience E given a certain type of task T and a performance metric P. If its performance on task T can be measured by P, it improves with experience E. Machine learning specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence.

[0105] Deep learning is a special type of machine learning that learns to represent the world using nested hierarchical concepts, achieving tremendous functionality and flexibility. Each concept is defined as being associated with a simpler one, while more abstract representations are computed in a less abstract manner. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by demonstration.

[0106] Virtual object interaction applications provide interactive functionality using virtual objects. These virtual objects can simulate human communication and behavior and interact with users. Such software (referring to virtual human interaction applications) is typically driven by artificial intelligence and natural language processing technologies and can interact with users through text, voice, images, forms, and other methods.

[0107] (Method Implementation Examples)

[0108] See Figure 1 , Figure 1 This is a flowchart illustrating a virtual e-commerce customer service interaction method provided in an embodiment of this application.

[0109] This application provides a virtual e-commerce customer service interaction method, used to provide interactive functions using virtual e-commerce customer service, the method including:

[0110] Step S101: Display the virtual e-commerce customer service interaction interface using a terminal device;

[0111] Step S102: Use the terminal device to obtain the user's input information, the input information including one or more of first text information, voice information and / or image information;

[0112] Step S103: Based on the input information, use the terminal device to play one or more preset interactive videos of the virtual e-commerce customer service, the preset interactive videos being used to collect user demand information;

[0113] Step S104: Based on the user demand information, use the terminal device to play one or more recommended interactive videos of the virtual e-commerce customer service, the recommended interactive videos being used to introduce recommended products.

[0114] Therefore, users access the virtual e-commerce customer service interactive interface through a terminal device within an application with virtual e-commerce customer service capabilities. They express their needs by inputting one or more of the following: text, voice, and / or image information. Based on the user's input, the virtual e-commerce customer service can play one or more preset interactive videos on the terminal device to help users express their needs more accurately and collect user demand information. Based on the collected demand information, the virtual e-commerce customer service can play one or more recommended interactive videos to introduce recommended products that meet the user's needs and guide the user to make a purchase. On the one hand, by providing a virtual e-commerce customer service interface, users can more conveniently obtain the information they need without having to queue for human customer service during peak hours, which helps improve user satisfaction and loyalty. On the other hand, by transforming the way e-commerce recommends products, virtual e-commerce customer service can collect user needs information in a short time by using pre-set interactive videos based on user input, and then provide corresponding product recommendations. This saves communication costs and time between e-commerce customer service and users, conserves customer service resources, and allows for simple and quick understanding of user needs, improving overall service efficiency. Furthermore, by introducing recommended products through interactive videos based on accurately and targeted user needs, the product recommendations are more objective and free from the subjective emotions of human customer service. Interactive videos also solve problems such as unnatural, mechanical, templated, and impersonal customer service robot presentations, enhancing user trust in recommended products and increasing their desire to purchase, thus improving the service quality of virtual e-commerce customer service. Finally, virtual e-commerce customer service can provide uninterrupted 24 / 7 service without incurring additional costs, reducing e-commerce operating costs.

[0115] The virtual e-commerce customer service interaction method can run on electronic devices. The electronic devices and the (user-used) terminal devices can be independent, or they can be integrated. When the electronic devices and terminal devices are independent, the electronic devices can be computers, servers (including cloud servers), or other devices with computing capabilities. This application does not limit the terminal device; it can be, for example, a mobile phone, tablet computer, laptop computer, desktop computer, smart wearable device, or other smart terminal device with a display screen and speakers. Alternatively, the terminal device can be a workstation or console with a display screen and speakers. The display screen can be a touch screen or a non-touch screen.

[0116] The user uses a terminal device (such as a smartphone, computer, etc.) to send an access request to establish a communication connection with the server. The server can run one or more applications to provide the interactive functions of a virtual e-commerce customer service (these applications can be written in one or more programming languages, such as Java, Python, Node.js, etc., and can utilize various frameworks and libraries to implement various functions, such as natural language processing, speech-to-text, image recognition, etc.). During the interaction between the user and the virtual e-commerce customer service, the terminal device will play the interactive video of the virtual e-commerce customer service. The preset interactive video and recommended interactive video of the virtual e-commerce customer service need to be stored on one or more servers, and these servers can use cloud storage or self-owned storage, etc. to store the videos.

[0117] The embodiments of this application can be applied to virtual object interaction applications, e-commerce platform applications, etc. The embodiments of this application do not limit the interactive content of the virtual e-commerce customer service. For example, it can be product recommendation, purchase guidance, after-sales service, logistics query, delivery tracking, question answering, social interaction, etc. The embodiments of this application do not limit the recommended products of the virtual e-commerce customer service. For example, they can be food, daily necessities, clothing, home furnishings, cars, or commercial housing, etc.

[0118] The virtual e-commerce customer service interaction interface is, for example, a graphical user interface. The virtual e-commerce customer service interaction interface can include a user input box where the user can input text, pictures, or send voice, etc. The interface can also include a virtual object (virtual e-commerce customer service), and the virtual e-commerce customer service can be one or more of a dynamic virtual person, virtual animal, and virtual cartoon image. As an example, the virtual e-commerce customer service is the virtual person "JING" (Chinese name: Jing). The interface can also include a video window where the user's video image can be displayed when the user has a video interaction with the virtual e-commerce customer service. The interface can also include a product recommendation area where the virtual e-commerce customer service can display recommended products to the user, and the recommended products can be accompanied by product information such as pictures, prices, and evaluation scores. The interface can also include a common questions area that can display common questions related to products, orders, after-sales, etc. and their solutions to facilitate users to quickly obtain information. The interface can also include a dialog box to display the conversation records between the virtual e-commerce customer service and the user to facilitate the user to view the historical communication content. The interface can also include operation buttons, such as a send button, cancel button, initiate voice button, initiate video button, end button, follow button, subscribe button, form interaction button, etc.

[0119] The preset interactive videos can include greeting videos, which can be used to welcome users and guide them into the conversation when the virtual e-commerce customer service begins interacting with them, such as, "Hello, welcome to our store! I'm your virtual customer service assistant. We recently launched a product called 'Smartwatch,' which is powerful and comes in various styles. Are you interested in learning more?"; they can also include user needs survey videos, where the virtual e-commerce customer service can initiate surveys or ask questions to inquire about the user's needs, such as the user's intended use of the product, purchase purpose, price budget, preferred color, etc., such as, "What is the main purpose of your watch purchase? Is it for fitness or daily commuting? If convenient, could you tell me your budget and style preferences?"; and they can also include personalized recommendation videos, based on... The system uses user needs information or historical data to recommend products that match the user's needs and inquires whether personalized recommendations include the user's requirements or whether to continue recommending products. It can also include frequently asked questions (FAQ) videos, allowing users to quickly select questions and obtain answers, such as choosing payment, shipping, or delivery methods. Furthermore, it can include consultation and explanation videos, answering user inquiries about products, such as product features, specifications, and uses. Order tracking videos allow users to check order status and logistics information through the digital human virtual e-commerce customer service. After-sales service videos help users resolve after-sales issues, such as returns, exchanges, and maintenance. It can also include user feedback collection videos, such as user satisfaction surveys and product reviews, to optimize service quality. Finally, it can include a closing message to thank the user at the end of the conversation and remind them to contact customer service if they have any questions, such as, "Thank you very much for choosing our product. If you have any further questions or suggestions, please don't hesitate to contact our customer service team. We will be happy to assist you."

[0120] The recommended interactive videos can provide users with recommendation information. These videos can include product recommendation videos, which recommend suitable products based on user needs and historical information. For example, "I see you previously purchased a smart bracelet. Would you like our newly launched multi-functional fitness bracelet? It can monitor heart rate, steps, and other data, and supports multiple sports modes, making it suitable for more fitness enthusiasts." Or, "We have a limited-time offer: purchase a 'phone + earphone' set at a discounted price. What do you think of this offer?" or "If you purchased our coffee machine, we highly recommend trying our specialty coffee beans. They have a unique taste and rich aroma that you'll never forget." Virtual e-commerce customer service representatives can make corresponding facial expressions and gestures based on product characteristics, such as smiling and looking up, to attract user attention. They can also provide product images or videos to help users better understand product details and usage effects. Recommended interactive videos can also include shopping guide videos, guiding users through the shopping process, including browsing products, adding items to the cart, and placing an order. Furthermore, recommended interactive videos can include personalized service videos, providing personalized purchase suggestions based on user purchase preferences and other data.

[0121] Collecting user needs through forms requires configuring the forms in the virtual e-commerce customer service configuration backend. The form data generated by users will be stored in a database or other data storage system. Relational or non-relational databases such as MySQL and MongoDB can be used for data storage, or cloud storage or other technologies can be used for data storage and management. After receiving the form data submitted by the user, the electronic device can analyze the form data locally or on the backend server.

[0122] In one specific embodiment, user A opens an e-commerce platform application and clicks the virtual e-commerce customer service button to enter the virtual e-commerce customer service interactive interface. The interactive interface includes a text chat box, a voice recognition button, and an image upload button. In the interface, the user can enter the text message "I want to buy a pair of white sneakers." Based on user A's input, the interactive interface plays a preset interactive video corresponding to the virtual e-commerce customer service representative. The virtual e-commerce customer service representative might say, "What is the main purpose of your purchase of these sneakers? Is it for fitness or daily commuting? If convenient, could you tell me your size, style preference, and budget?", accompanied by a smiling emoticon and questioning gestures. User A's input requirements might be, for example, "daily commuting, size 42, simple style." Based on user A's input, the interactive interface plays one or more recommended sneaker interactive videos corresponding to the virtual e-commerce customer service representative, including sneaker product display videos or special offer videos related to the user's needs. User A can make a purchase decision based on these recommended videos or continue interacting with the virtual e-commerce customer service representative in the input box to seek more advice and assistance.

[0123] In one specific embodiment, user Xiaoli wants to buy a dress. Xiaoli uses her smartphone (terminal device) to access an e-commerce company's mobile application, which provides a virtual e-commerce customer service interaction function. Xiaoli's smartphone sends an access request to the e-commerce company's server. After establishing a communication connection, Xiaoli's smartphone displays the virtual e-commerce customer service interaction interface, and Xiaoli interacts with the virtual e-commerce customer service representative.

[0124] Xiaoli enters "dress" in the user input box on the activity interface. Based on this input, the interactive interface plays a pre-set interactive video of a virtual e-commerce customer service representative to welcome the user and introduce the latest dress styles, outputting the voice message, "Hello, welcome to our women's clothing section! I am your virtual customer service assistant. We recently launched a series of dresses called 'Spring Blossoms,' with various styles and comfortable materials. Are you interested in learning more?" Xiaoli enters "Okay, recommend some dresses for me." Based on the user's input, the system extracts the keyword "dress" and plays pre-set interactive videos related to the keyword "dress" from the virtual e-commerce customer service's question and answer database. For example, the pre-set interactive video content might be, "What length of dress would you like to buy? Long or short? What color or print do you prefer?" Xiaoli enters "long, brown, vintage style" in the user input box. Based on Xiaoli's needs, the system retrieves one or more recommended interactive videos of dresses matching the keywords "long, brown, retro style" from the virtual e-commerce customer service's question and answer database. Alternatively, the virtual e-commerce customer service can recommend dresses relevant to the user's needs, for example, playing a recommended interactive video saying, "Based on your needs, I suggest you try this 'retro floral dress.' It's made of lightweight fabric, comes in a variety of colors, and is perfect for spring," accompanied by a twirling motion while lifting the dress. At the end of the conversation, a pre-set interactive video of the virtual e-commerce customer service closing remarks is played, such as, "Thank you so much for choosing our dress. If you have any further questions or suggestions, please feel free to contact us. We will be happy to assist you."

[0125] See Figure 2 , Figure 2 This is a flowchart illustrating another virtual e-commerce customer service interaction method provided in an embodiment of this application.

[0126] In some optional embodiments, the method further includes:

[0127] Step S105: Use the terminal device to obtain at least two comparison products specified by the user;

[0128] Step S106: Based on at least two of the comparison products, obtain product information for at least two of the comparison products;

[0129] Step S107: Based on at least two pieces of product information, obtain at least two differences between the comparative products;

[0130] Step S108: Perform speech synthesis on the difference information to obtain synthesized speech of the difference information;

[0131] Step S109: Use the terminal device to play the synthesized voice of the difference information using the virtual e-commerce customer service.

[0132] Therefore, users can select and input at least two products for comparison via their terminal devices to inquire about their advantages, disadvantages, and differences with virtual e-commerce customer service. Based on the user's input of at least two products, the virtual e-commerce customer service obtains product information (such as price, functions, specifications, etc.), analyzes and compares the differences between these products, extracts the difference information, and synthesizes it into a synthesized speech. This synthesized speech is then played on the terminal device by the virtual e-commerce customer service, allowing users to more intuitively understand the differences between the products. On the one hand, users can understand the differences between any two or more products through virtual e-commerce customer service, thus better choosing the products that suit them and enhancing user satisfaction. On the other hand, generating natural language from the differences through speech synthesis technology and then playing the synthesized speech by the virtual e-commerce customer service solves the problem of mechanical and templated responses from robot customer service, making it easier for users to understand the differences between the products. Users can more accurately choose the products that suit them, thereby increasing product sales. In addition, virtual e-commerce customer service can solve users' product comparison problems in a short time, improving the response efficiency of e-commerce customer service and reducing time and resource costs.

[0133] In one specific embodiment, at least two of the comparative products are, for example, high-top shoes A1 and low-top shoes A2; at least two of the product information are, for example, "product details of high-top shoes A1, such as material leather, sole height 2.5cm, and upper height 25cm" and "product details of low-top shoes A1, such as material fabric, sole height 2.5cm, and upper height 15cm".

[0134] In some optional embodiments, the difference information of at least two of the comparative products may include product application information and product selection information provided based on the difference information; the difference information of at least two of the comparative products may be, for example, "The differences between the two pairs of shoes you are comparing are as follows: 1. Different materials: High-top shoes A1 are made of leather, while low-top shoes A1 are made of fabric; 2. Different shoe heights: The shoe height of high-top shoes A1 is 10 cm higher than that of low-top shoes A1; In some sports, the shoe height of high-top shoes can better protect your ankle, but in daily commuting, high-top shoes may restrict your ankle movement. The above information is for reference only."

[0135] In one specific embodiment, when purchasing a mobile phone, user B wants to understand the differences between two phones. User B opens their terminal device (such as a smartphone), enters the virtual e-commerce customer service interaction interface in the e-commerce platform application, and enters the names of two phones, C and D. Based on the user's input, the system retrieves product information for these two phones from the database, such as price, screen size, and camera resolution. Based on this information, the system calculates and lists the differences between the two phones, such as which phone has a larger screen or a clearer camera. This difference information is then converted into synthesized speech. The virtual e-commerce customer service outputs the synthesized speech detailing these differences and further explains the advantages and disadvantages of each difference to the user. For example, "Phone C has a larger screen, making it more suitable for watching videos and playing games; however, due to the larger screen, it may consume more battery life," to help user B make a decision based on the information provided by the virtual e-commerce customer service.

[0136] In some optional embodiments, obtaining difference information between at least two of the comparison products based on the product information includes:

[0137] The product information is input into the difference detection model to obtain difference information for at least two of the comparison products;

[0138] The training process of the difference detection model includes:

[0139] Obtain a training set, which includes multiple sets of training data. Each set of training data includes at least two sample product information and labeled data of the difference information corresponding to at least two sample product information.

[0140] For each set of training data, perform the following processing:

[0141] Input at least two sample product information from the training data into a preset deep learning model to obtain predicted data of the difference information corresponding to at least two sample product information.

[0142] The model parameters of the deep learning model are updated based on the predicted data and labeled data of the difference information corresponding to at least two sample product information.

[0143] The system checks whether the preset training termination condition is met; if so, the trained deep learning model is used as the difference detection model; if not, the system continues to train the deep learning model using the next set of training data.

[0144] Therefore, by using deep learning algorithms to compare product differences, it is possible to identify the differences between at least two products provided by the user more quickly and accurately; and by using virtual e-commerce customer service to automatically identify product differences, the burden and cost of e-commerce customer service can be reduced.

[0145] In some optional embodiments, the step of playing one or more preset interactive videos of the virtual e-commerce customer service using the terminal device based on the input information includes:

[0146] When the input information includes the first text information, keywords from the first text information are extracted. Based on the keywords of the first text information, one or more preset interactive videos matching the keywords of the first text information are searched from the question-and-answer database matched by the virtual e-commerce customer service. The terminal device is then used to play one or more preset interactive videos matching the keywords of the first text information; and / or,

[0147] When the input information includes the voice information, the voice information is recognized, converted into second text information, keywords are extracted from the second text information, and based on the keywords, one or more preset interactive videos matching the keywords in the question-and-answer database are searched. The terminal device is then used to play one or more preset interactive videos matching the keywords in the second text information; and / or,

[0148] When the input information includes the image information, the image information is identified, the image information is converted into third text information, keywords of the third text information are extracted, and based on the keywords of the third text information, one or more preset interactive videos that match the keywords of the third text information are searched from the question-and-answer database. The terminal device is then used to play one or more preset interactive videos that match the keywords of the third text information.

[0149] Therefore, user input can include one or more of the following: text, voice, and image information. The system identifies the voice or image information, converts it into text, extracts keywords corresponding to each text segment, and then searches a question-and-answer database for matching pre-set interactive videos based on these keywords. On one hand, this enriches the forms of user input, giving users more freedom to interact with virtual e-commerce customer service in various ways. On the other hand, by automatically matching relevant pre-set interactive videos with key content extracted from the input, virtual e-commerce customer service can quickly respond to user questions or other input, improving the efficiency of e-commerce customer service interactions. Furthermore, the pre-set interactive video service allows for targeted interaction with users to collect user needs information, leading to better user service and enhanced user experience. Finally, the automatic matching of pre-set interactive videos by virtual e-commerce customer service reduces the burden and cost of e-commerce customer service.

[0150] In one specific embodiment, a user H is searching for a new mobile phone. They upload a picture of the phone to the virtual e-commerce customer service interaction interface and input the voice message, "Introduce the phone in the picture; it has 6GB of RAM and 256GB of internal storage." to describe the phone's configuration information. The system recognizes the phone image, converts it into text information, and extracts keywords, such as "XX phone 18pro". It also recognizes the voice message and converts it into the text information "6GB RAM, 256GB internal storage". Based on "XX phone 18pro" and "6GB RAM, 256GB internal storage", the system searches for and plays a relevant preset interactive video from the virtual e-commerce customer service's question and answer database.

[0151] In some optional embodiments, the step of playing one or more recommended interactive videos of the virtual e-commerce customer service using the terminal device based on the user demand information includes:

[0152] From the question-and-answer database matched by the virtual e-commerce customer service, obtain one or more recommended interactive videos that match the user's needs; or,

[0153] Obtain the recommended text and virtual human actions corresponding to the user's demand information; drive the virtual e-commerce customer service's facial expressions and lip movements according to the recommended text, and drive the virtual e-commerce customer service's actions according to the virtual human actions, so as to generate the recommended interactive video.

[0154] Therefore, based on the user's input needs, one or more recommended interactive videos matching the user's needs are retrieved from the question-and-answer database of the virtual e-commerce customer service to display relevant recommended products to the user. By automatically matching relevant recommended interactive videos, the virtual e-commerce customer service can quickly respond to user needs and improve interaction efficiency. It can also generate recommended text and virtual human actions based on user needs, drive the virtual e-commerce customer service's facial expressions and lip movements, and add actions to generate recommended interactive videos, providing more customized recommended interactive services, responding to user needs, solving the problem of mechanical and templated robot customer service replies, helping users better understand recommended products, and reducing customer service costs.

[0155] In one specific embodiment, a user F enters "I want to buy a pair of long men's jeans, size L" in the input box of the virtual e-commerce customer service interface. Based on the input information of user F, the system searches for and plays recommended interactive videos that match "men's jeans" in the existing question and answer database of the virtual e-commerce customer service.

[0156] In one specific embodiment, a user G enters "I want to buy a pair of long, loose-fitting men's jeans" into the input box of the virtual e-commerce customer service interface. Based on user G's input, the system retrieves recommended men's loose-fitting jeans from the database, extracts information such as images, prices, colors, sizes, features, and styling tips, and generates corresponding recommendation text. Simultaneously, it matches corresponding actions for the virtual e-commerce customer service representative. Synthetic speech is generated based on the recommendation text and integrated with the virtual human's facial expressions, lip movements, and actions to create a recommended interactive video showcasing product images, explaining product features, and providing purchase suggestions. Users can use this interactive video to better understand the product and its features and make a purchase decision.

[0157] In some optional embodiments, the method further includes:

[0158] The terminal device is used to obtain feedback information from the user, and the feedback information is used to evaluate the accuracy of the recommended interactive video.

[0159] Based on the feedback information, update the question-and-answer database; and / or update the recommended text.

[0160] Therefore, by utilizing terminal devices to obtain user feedback on recommended interactive videos, such as whether they match user needs and the accuracy of the recommendations, the matching question-and-answer database and recommendation text can be updated based on user feedback to improve matching accuracy. On the one hand, by collecting and analyzing user feedback, the question-and-answer database and recommendation text of virtual e-commerce customer service can be adjusted in a timely manner to improve recommendation accuracy, better meet user needs, and continuously optimize virtual e-commerce customer service to provide more accurate and customized interactive services, enhance user experience and satisfaction, thereby increasing user loyalty and purchasing power. On the other hand, by collecting and analyzing user feedback, it is also possible to better understand user needs and provide e-commerce with more precise market operation strategies.

[0161] In some optional embodiments, obtaining user feedback can be achieved by providing a visible rating or comment function when the user watches a recommended interactive video, allowing the user to rate and provide feedback on the recommended interactive video. For example, the user can give a score of 1-10 on a 10-point scale and provide comments and suggestions about the recommended interactive video in the input box. Alternatively, emails, SMS messages, or other push notifications can be sent to the user to request feedback and evaluation. For example, when a user purchases a product, the system can automatically send an SMS asking if the user would like to provide feedback. Natural language processing technology can also be used to automatically identify and analyze user feedback and associate it with the corresponding recommended interactive video. For example, if the user requests more product details, the application can associate this feedback with the corresponding recommended interactive video and update the question-and-answer database and recommendation text accordingly. Social media platforms can also be used to collect user feedback. For example, natural language processing technology can be used to search for posts and comments related to the recommended interactive video on social media and integrate them into the feedback database.

[0162] In some optional embodiments, the feedback information can be rating feedback (e.g., a rating on a scale of 1-10, or 1-100), evaluation level feedback (levels of A, B, C, or D), text evaluation feedback, and voice evaluation feedback, etc.; for example, it can be feedback that the product information in the recommended interactive video is not detailed or in-depth enough, such as wanting to see more product features, specifications, and materials; it can be feedback that there are errors or inaccuracies in the recommended interactive video, such as incorrect recommended prices or mismatched keywords; it can also be feedback that the user is satisfied with the recommended content of the interactive video and provides positive evaluation and encouragement to provide positive feedback and prove the effectiveness of the recommended interactive video; it can be feedback that the user has a negative experience and dissatisfaction with the recommended interactive video, such as finding the recommended interactive video too long or too simple, or pointing out that some information is missing or not clear enough; the feedback information page can be suggestions or requests from users related to the recommended interactive video, such as needing more interactive functions or customizing recommendation options, etc.

[0163] In some alternative embodiments, feedback information can be stored in a database, and data mining and machine learning techniques can be used to automatically classify and analyze user feedback. For example, by analyzing a large amount of feedback information, users' general opinions and preferences on recommended interactive videos can be determined, and the question-and-answer database and recommendation text can be updated accordingly.

[0164] In some optional embodiments, the method further includes:

[0165] The terminal device is used to conduct video interaction with the user using the virtual e-commerce customer service;

[0166] During the video interaction, the user's facial information is obtained;

[0167] Based on the portrait information, obtain the user's current facial expression information and / or action information;

[0168] Based on the facial expression information and / or the action information, obtain the user's emotional information;

[0169] Based on the emotional information, it is determined whether the user is satisfied with the recommended product.

[0170] Therefore, by using terminal devices to interact with users via video through virtual e-commerce customer service, user facial information can be obtained. This information can be used to analyze the user's current facial expressions and actions to assess the user's emotional state and thus detect the user's satisfaction with recommended products. By monitoring user emotional information in real time, virtual e-commerce customer service can better understand user emotions, indirectly infer user needs, adjust service strategies in a timely manner, and improve service quality. Customized services can enhance user experience and satisfaction.

[0171] In some optional embodiments, obtaining the user's facial image information can be achieved by activating the camera of the terminal device or an external camera to capture the user's facial image; or by capturing the user's facial image information from a dynamic video. Based on the facial image information, obtaining the user's current expression information and / or action information can be achieved using facial expression recognition technology to detect the movement and changes of the user's facial muscles and infer the user's current expression state. For example, when the user smiles, the upward turn of the corners of the mouth and the curvature of the eyes can be detected. Alternatively, computer vision technology can be used to detect the user's body posture and actions, such as the user shaking their head or moving their fingers. By analyzing the user's facial expression and action information, the user's emotional state can be inferred. For example, if the user is detected to have an upward turn of the corners of their mouth and a curvature of their eyes indicating a happy expression and positive action, the user is inferred to be in a pleasant mood; if the user is detected to be shaking their head or pointing their thumb downwards, the user is inferred to be in a depressed mood; if the user is detected to be frowning, the user is inferred to be possibly in a dissatisfied mood.

[0172] In one specific embodiment, the user engages in video communication with a virtual e-commerce customer service representative. During the video interaction, the user's mobile phone camera is activated to capture the user's facial image to obtain the user's portrait information. Computer vision technology is then used to identify and analyze the user's current expressions and actions, such as detecting whether the user is smiling or clapping. Based on these expressions and actions, the user's emotional state is inferred to be happy. The emotional information is then used to assess the user's satisfaction with the recommended products.

[0173] In some optional embodiments, the method further includes:

[0174] When it is detected that the user is satisfied with the recommended product, continue playing a recommended interactive video that is similar to the recommended interactive video;

[0175] When it is detected that the user is not satisfied with the recommended product, the recommended interactive video is replaced.

[0176] Therefore, when user satisfaction with the recommended product is detected, a video similar to the recommended interactive video will continue to be played; when user dissatisfaction is detected, the recommended interactive video will be adjusted in a timely manner to improve user experience, meet user needs, and increase product sales. By continuously optimizing and improving customized services, user loyalty can be increased and the competitive advantage of e-commerce can be expanded.

[0177] In some optional embodiments, users interact with virtual e-commerce customer service via video to learn about recommended mobile phones. When it is detected that the user is satisfied with the currently recommended mobile phone product Z1, the video continues to play interactive videos recommending similar mobile phone products (same brand, same price, or same configuration). Alternatively, if it is detected that the user shows positive feelings about a certain feature of the phone, similar interactive videos can continue to play, introducing other related features, functions, or advantages. If it is detected that the user is dissatisfied with the recommended product, the interactive video can be directly changed to provide more relevant information or solutions, improving the user's shopping experience. For example, if it is detected that the user is not interested in the storage capacity of the phone as described in the interactive video, the video can automatically switch to an interactive video introducing the phone's camera pixels or display screen, showcasing other features of the phone to meet the user's needs.

[0178] In one specific embodiment, this application also provides a virtual e-commerce customer service interaction method, used to provide interactive functions using virtual e-commerce customer service. The method includes: displaying the virtual e-commerce customer service interaction interface using a terminal device; obtaining user input information using the terminal device, the input information including one or more of first text information, voice information, and / or image information; when the input information includes the first text information, extracting keywords from the first text information; based on the keywords of the first text information, searching for one or more preset interactive videos matching the keywords of the first text information from a question-and-answer database matched by the virtual e-commerce customer service; and playing one or more preset interactive videos matching the keywords of the first text information using the terminal device. This information includes: keyword matching of preset interactive videos; and / or when the input information includes the voice information, recognizing the voice information, converting the voice information into second text information, extracting keywords from the second text information, and based on the keywords of the second text information, searching for one or more preset interactive videos matching the keywords of the second text information from the question-and-answer database, and playing one or more preset interactive videos matching the keywords of the second text information using the terminal device; and / or when the input information includes the image information, recognizing the image information, converting the image information into third text information, extracting keywords from the third text information, and based on the keywords of the third .... Search the database for one or more preset interactive videos that match the keywords of the third text information, and play one or more preset interactive videos that match the keywords of the third text information using the terminal device; retrieve one or more recommended interactive videos that match the user's needs from the question and answer database matched by the virtual e-commerce customer service; or, retrieve the recommended text and virtual human actions corresponding to the user's needs; drive the virtual e-commerce customer service's facial expressions and lip movements according to the recommended text, and drive the virtual e-commerce customer service's actions according to the virtual human actions to generate the recommended interactive video; retrieve at least two comparison products specified by the user using the terminal device; and retrieve at least two... The comparison involves: providing product information for at least two of the products; inputting the product information into a difference detection model to obtain difference information for at least two of the comparison products; performing speech synthesis on the difference information to obtain synthesized speech; using the terminal device to play the synthesized speech of the difference information via the virtual e-commerce customer service; using the terminal device to obtain feedback information from the user, which is used to evaluate the recommendation accuracy of the interactive video; updating the question-and-answer database based on the feedback information; and / or updating the recommendation text; using the terminal device to conduct video interaction with the user via the virtual e-commerce customer service; and obtaining the user's facial image information during the video interaction.Based on the facial image information, obtain the user's current facial expression information and / or action information; based on the facial expression information and / or the action information, obtain the user's emotional information; based on the emotional information, detect whether the user is satisfied with the recommended product; when the user is satisfied with the recommended product, continue playing a recommended interactive video similar to the recommended interactive video; when the user is dissatisfied with the recommended product, replace the recommended interactive video.

[0179] (Equipment Example)

[0180] This application provides an electronic device whose specific embodiments are consistent with the embodiments described in the above method embodiments and the technical effects achieved, and some contents will not be repeated.

[0181] The electronic device includes a memory and at least one processor, the memory storing a computer program, and the at least one processor being configured to execute the computer program to perform the following steps:

[0182] The virtual e-commerce customer service interaction interface is displayed using a terminal device;

[0183] The terminal device is used to obtain user input information, which includes one or more of first text information, voice information, and / or image information;

[0184] Based on the input information, the terminal device is used to play one or more preset interactive videos of the virtual e-commerce customer service, the preset interactive videos being used to collect user demand information;

[0185] Based on the user demand information, the terminal device is used to play one or more recommended interactive videos of the virtual e-commerce customer service, which are used to introduce recommended products.

[0186] In some alternative implementations, the at least one processor is further configured to perform the following steps when executing the computer program:

[0187] Use the terminal device to obtain at least two comparison products specified by the user;

[0188] Based on at least two of the comparison products, obtain product information for at least two of the comparison products;

[0189] Based on at least two pieces of product information, obtain at least two differences between the comparative products;

[0190] The difference information is processed by speech synthesis to obtain synthesized speech based on the difference information.

[0191] The terminal device is used to play the synthesized voice of the difference information using the virtual e-commerce customer service.

[0192] In some alternative implementations, the at least one processor is further configured to, when executing the computer program, acquire difference information of at least two of the comparative products based on the product information in the following manner:

[0193] The product information is input into the difference detection model to obtain difference information for at least two of the comparison products;

[0194] The training process of the difference detection model includes:

[0195] Obtain a training set, which includes multiple sets of training data. Each set of training data includes at least two sample product information and labeled data of the difference information corresponding to at least two sample product information.

[0196] For each set of training data, perform the following processing:

[0197] Input at least two sample product information from the training data into a preset deep learning model to obtain predicted data of the difference information corresponding to at least two sample product information.

[0198] The model parameters of the deep learning model are updated based on the predicted data and labeled data of the difference information corresponding to at least two sample product information.

[0199] The system checks whether the preset training termination condition is met; if so, the trained deep learning model is used as the difference detection model; if not, the system continues to train the deep learning model using the next set of training data.

[0200] When deep learning is used to train various models, by designing and establishing an appropriate number of neural computing nodes and a multi-layered computational hierarchy, and by selecting suitable input and output layers, a pre-defined deep learning model (i.e., the initial model corresponding to each model) can be obtained. Through the learning and optimization of deep learning models, a functional relationship from input to output can be established. Although it is not possible to find a 100% accurate functional relationship between input and output, it can approximate the real-world correlation as closely as possible. The models trained in this way can obtain corresponding output data based on input data, have a wide range of applications, and provide high accuracy and reliability in calculation results.

[0201] The embodiments of this application do not limit the training process of each model. For example, it may adopt a supervised learning training method, a semi-supervised learning training method, or an unsupervised learning training method.

[0202] When using supervised or semi-supervised learning training methods, this application embodiment does not limit the method of obtaining labeled data. For example, manual labeling, automatic labeling, or semi-automatic labeling can be used. When sample data is collected during real-world interactions, real data can be obtained from historical data as labeled data through keyword extraction.

[0203] The embodiments of this application do not limit the training termination conditions during the training process of each model. For example, it may be that the number of training times reaches a preset number (the preset number of times is, for example, 1 time, 3 times, 10 times, 100 times, 1000 times, 10000 times, etc.), or it may be that the training data in the training set has completed one or more training sessions, or it may be that the total loss value obtained in this training is not greater than the preset loss value.

[0204] In some alternative implementations, the at least one processor is further configured to, when executing the computer program, play one or more preset interactive videos of the virtual e-commerce customer service using the terminal device based on the input information:

[0205] When the input information includes the first text information, keywords from the first text information are extracted. Based on the keywords of the first text information, one or more preset interactive videos matching the keywords of the first text information are searched from the question-and-answer database matched by the virtual e-commerce customer service. The terminal device is then used to play one or more preset interactive videos matching the keywords of the first text information; and / or,

[0206] When the input information includes the voice information, the voice information is recognized, converted into second text information, keywords are extracted from the second text information, and based on the keywords, one or more preset interactive videos matching the keywords in the question-and-answer database are searched. The terminal device is then used to play one or more preset interactive videos matching the keywords in the second text information; and / or,

[0207] When the input information includes the image information, the image information is identified, the image information is converted into third text information, keywords of the third text information are extracted, and based on the keywords of the third text information, one or more preset interactive videos that match the keywords of the third text information are searched from the question-and-answer database. The terminal device is then used to play one or more preset interactive videos that match the keywords of the third text information.

[0208] In some alternative implementations, the at least one processor is further configured to, when executing the computer program, play one or more recommended interactive videos of the virtual e-commerce customer service using the terminal device based on the user demand information:

[0209] From the question-and-answer database matched by the virtual e-commerce customer service, obtain one or more recommended interactive videos that match the user's needs; or,

[0210] Obtain the recommended text and virtual human actions corresponding to the user's demand information; drive the virtual e-commerce customer service's facial expressions and lip movements according to the recommended text, and drive the virtual e-commerce customer service's actions according to the virtual human actions, so as to generate the recommended interactive video.

[0211] In some alternative implementations, the at least one processor is further configured to perform the following steps when executing the computer program:

[0212] The terminal device is used to obtain feedback information from the user, and the feedback information is used to evaluate the accuracy of the recommended interactive video.

[0213] Based on the feedback information, update the question-and-answer database; and / or update the recommended text.

[0214] In some alternative implementations, the at least one processor is further configured to perform the following steps when executing the computer program:

[0215] The terminal device is used to conduct video interaction with the user using the virtual e-commerce customer service;

[0216] During the video interaction, the user's facial information is obtained;

[0217] Based on the portrait information, obtain the user's current facial expression information and / or action information;

[0218] Based on the facial expression information and / or the action information, obtain the user's emotional information;

[0219] Based on the emotional information, it is determined whether the user is satisfied with the recommended product.

[0220] In some alternative implementations, the at least one processor is further configured to perform the following steps when executing the computer program:

[0221] When it is detected that the user is satisfied with the recommended product, continue playing a recommended interactive video that is similar to the recommended interactive video;

[0222] When it is detected that the user is not satisfied with the recommended product, the recommended interactive video is replaced.

[0223] See Figure 3 , Figure 3 A structural block diagram of an electronic device provided in an embodiment of this application is shown.

[0224] Electronic device 10 may include, for example, at least one memory 11, at least one processor 12, and a bus 13 connecting different platform systems.

[0225] The memory 11 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 111 and / or cache memory 112, and may further include read-only memory (ROM) 113.

[0226] The memory 11 also stores a computer program, which can be executed by the processor 12 to enable the processor 12 to implement the steps of any of the above methods.

[0227] The memory 11 may also include a utility 114 having at least one program module 115, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0228] Accordingly, processor 12 can execute the aforementioned computer program, and can also execute utility 114.

[0229] The processor 12 may employ one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0230] Bus 13 can represent one or more types of bus structures, including a memory bus or memory controller, peripheral bus, graphics acceleration port, processor, or a local bus using any bus structure with multiple bus structures.

[0231] Electronic device 10 can also communicate with one or more external devices, such as keyboards, pointing devices, Bluetooth devices, etc., and with one or more devices capable of interacting with it, and / or with any device that enables it to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed through input / output interface 14. Furthermore, electronic device 10 can communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 15. Network adapter 15 can communicate with other modules of electronic device 10 via bus 13. It should be understood that, although not shown in the figures, in practical applications, other hardware and / or software modules can be used in conjunction with electronic device 10, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0232] (Media Example)

[0233] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of any of the above methods or the functions of any of the above devices. The specific embodiments and the technical effects achieved are consistent with those described in the above method embodiments, and some contents will not be repeated.

[0234] See Figure 4 , Figure 4 A schematic diagram of the structure of a program product provided in an embodiment of this application is shown.

[0235] The program product is used to implement any of the methods described above. The program product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In embodiments of this application, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device. The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0236] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium capable of sending, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, or any suitable combination thereof. Program code for performing operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code may be executed entirely on a user computing device, partially on a user device, as a standalone software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to user computing devices via any type of network, including local area networks (LANs) or wide area networks (WANs), or they can be connected to external computing devices (e.g., via the Internet using an Internet service provider).

[0237] This application describes the invention from the perspectives of purpose, performance, progress, and novelty, and it meets the functional enhancement and use requirements emphasized by the Patent Law. The above description and drawings are merely preferred embodiments of this application and are not intended to limit this application. Therefore, all structures, devices, features, etc., that are similar to or identical to those of this application, i.e., all equivalent substitutions or modifications made in accordance with the scope of this patent application, shall fall within the scope of protection of this patent application.

Claims

1. A virtual e-commerce customer service interaction method, characterized in that, The method for providing interactive functions using virtual e-commerce customer service includes: The virtual e-commerce customer service interaction interface is displayed using a terminal device; The terminal device is used to obtain user input information, which includes one or more of first text information, voice information, and / or image information; Based on the input information, the terminal device is used to play one or more preset interactive videos of the virtual e-commerce customer service. The preset interactive videos are used to collect user demand information by asking questions to the user. The user demand information includes at least one of product usage scenarios, purchase purpose, price budget, and specification preferences. Based on the user demand information, the terminal device is used to play one or more recommended interactive videos of the virtual e-commerce customer service. Playing the recommended interactive videos includes: obtaining the recommended text and virtual human actions corresponding to the user demand information, driving the virtual e-commerce customer service's facial expressions and lip movements according to the recommended text, and driving the virtual e-commerce customer service's actions according to the virtual human actions, so as to generate the recommended interactive videos. During the playback of the recommended interactive video, the terminal device uses the virtual e-commerce customer service to interact with the user via video. During the video interaction, the user's body posture and movements are detected; Based on the user's current action information, identify the user's real-time emotions; The playback strategy for the recommended interactive videos is adjusted based on the real-time sentiment.

2. The virtual e-commerce customer service interaction method according to claim 1, characterized in that, The method further includes: Use the terminal device to obtain at least two comparison products specified by the user; Based on at least two of the comparison products, obtain product information for at least two of the comparison products; Based on at least two pieces of product information, obtain at least two differences between the comparative products; The difference information is processed by speech synthesis to obtain synthesized speech based on the difference information. The terminal device is used to play the synthesized voice of the difference information using the virtual e-commerce customer service.

3. The virtual e-commerce customer service interaction method according to claim 2, characterized in that, The step of obtaining difference information for at least two of the comparison products based on the product information includes: The product information is input into the difference detection model to obtain difference information for at least two of the comparison products; The training process of the difference detection model includes: Obtain a training set, which includes multiple sets of training data. Each set of training data includes at least two sample product information and labeled data of the difference information corresponding to at least two sample product information. For each set of training data, perform the following processing: Input at least two sample product information from the training data into a preset deep learning model to obtain predicted data of the difference information corresponding to at least two sample product information. The model parameters of the deep learning model are updated based on the predicted data and labeled data of the difference information corresponding to at least two sample product information. The system checks whether the preset training termination condition is met; if so, the trained deep learning model is used as the difference detection model; if not, the system continues to train the deep learning model using the next set of training data.

4. The virtual e-commerce customer service interaction method according to claim 1, characterized in that, The step of playing one or more preset interactive videos of the virtual e-commerce customer service using the terminal device based on the input information includes: When the input information includes the first text information, keywords from the first text information are extracted. Based on the keywords of the first text information, one or more preset interactive videos matching the keywords of the first text information are searched from the question-and-answer database matched by the virtual e-commerce customer service. The terminal device is then used to play one or more preset interactive videos matching the keywords of the first text information; and / or, When the input information includes the voice information, the voice information is recognized, converted into second text information, keywords are extracted from the second text information, and based on the keywords, one or more preset interactive videos matching the keywords in the question-and-answer database are searched. The terminal device is then used to play one or more preset interactive videos matching the keywords in the second text information; and / or, When the input information includes the image information, the image information is identified, converted into third text information, keywords of the third text information are extracted, and based on the keywords of the third text information, one or more preset interactive videos matching the keywords of the third text information are searched from the question-and-answer database. The terminal device is then used to play one or more preset interactive videos matching the keywords of the third text information.

5. The virtual e-commerce customer service interaction method according to claim 1, characterized in that, The method further includes: The terminal device is used to obtain feedback information from the user, and the feedback information is used to evaluate the accuracy of the recommended interactive video. Based on the feedback information, update the question-and-answer database; and / or update the recommended text.

6. The virtual e-commerce customer service interaction method according to claim 1, characterized in that, The terminal device utilizes the virtual e-commerce customer service to interact with users via video, and also includes: During the video interaction, the user's facial information is obtained; Based on the portrait information, obtain the user's current facial expression information and action information; Based on the facial expression information and the action information, the user's emotional information is obtained; Based on the emotional information, it is determined whether the user is satisfied with the recommended product.

7. The virtual e-commerce customer service interaction method according to claim 6, characterized in that, The method further includes: When it is detected that the user is satisfied with the recommended product, continue playing a recommended interactive video that is similar to the recommended interactive video; When it is detected that the user is not satisfied with the recommended product, the recommended interactive video is replaced.

8. An electronic device, characterized in that, The electronic device includes a memory and at least one processor, the memory storing a computer program, and the at least one processor being configured to execute the computer program to perform the following steps: Use terminal devices to display a virtual e-commerce customer service interaction interface; The terminal device is used to obtain user input information, which includes one or more of first text information, voice information, and / or image information; Based on the input information, the terminal device is used to play one or more preset interactive videos of the virtual e-commerce customer service. The preset interactive videos are used to collect user demand information by asking questions to the user. The user demand information includes at least one of product usage scenarios, purchase purpose, price budget, and specification preferences. Based on the user demand information, the terminal device is used to play one or more recommended interactive videos of the virtual e-commerce customer service. Playing the recommended interactive videos includes: obtaining the recommended text and virtual human actions corresponding to the user demand information, driving the virtual e-commerce customer service's facial expressions and lip movements according to the recommended text, and driving the virtual e-commerce customer service's actions according to the virtual human actions, so as to generate the recommended interactive videos. During the playback of the recommended interactive video, the terminal device uses the virtual e-commerce customer service to interact with the user via video. During the video interaction, the user's body posture and movements are detected; Based on the user's current action information, identify the user's real-time emotions; The playback strategy for the recommended interactive videos is adjusted based on the real-time sentiment.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by at least one processor, implements the steps of the method of any one of claims 1-7 or the function of the electronic device of claim 8.

Images