Language interaction interface-based interaction method for end-cloud model combination and computing equipment

Abstract

The invention provides an interaction method based on a language interaction interface and used for end-cloud model combination and computing equipment. The method comprises the following steps: calling an end-side intelligent assistant in response to a user request; judging a user intention through an end side model; according to the user intention, different interaction results are generated according to the intention recognition result; wherein if it is judged that the user intention is the first type of intention, the end side model directly processes the user intention and returns a result to complete interaction; and if it is judged that the user intention is a second type intention, entering a cloud side large model for processing, and returning a result to an end side to complete interaction. According to the technical scheme, the user interaction boundary can be weakened, the operation process is more flexible, the user interaction cost is reduced, and the interaction delay time of a large model is shortened.

Description

Technical Field

[0001] The present invention relates to the field of e-commerce, and in particular to an interaction method and computing device for combining a terminal and a cloud model based on a language interaction interface. Background Art

[0002] The technology of improving the user experience and efficiency on e-commerce platforms through large pre-trained language models has become the mainstream of e-commerce platform personalization functions. This type of technology mainly relies on training natural language understanding capabilities based on large language models (LLMs), combined with powerful text generation and multimodal processing capabilities. Through understanding the context, the user's true intention is captured, and the conversation history is maintained, making the conversation more coherent and natural.

[0003] Although the current traditional graphical page interaction (GUI interaction) interaction method provides users with an intuitive operating interface, it also has some limitations and shortcomings in some aspects. For example, traditional GUIs often use fixed layouts, and complex GUIs require a period of learning to use them proficiently. Traditional GUI design often focuses more on click interaction on mobile devices and mouse and keyboard operation on personal computers. It lacks flexibility and displays information in one window, which is not convenient for users who need to view multiple views or multitask at the same time.

[0004] To this end, a technical solution is needed that can weaken the boundaries of user interaction, make the operation process more flexible, reduce user interaction costs, and reduce the interaction delay time of large models. Summary of the invention

[0005] The present invention aims to provide an interaction method and computing device based on a language interaction interface for the combination of end-cloud models, which can weaken the user interaction boundaries, make the operation process more flexible, reduce the user interaction cost, and reduce the interaction delay time of large models.

[0006] According to one aspect of the present invention, there is provided an interaction method for combining a terminal and a cloud model based on a language interaction interface, the interaction method comprising:

[0007] Respond to user requests and invoke the client-side intelligent assistant;

[0008] Determine user intent through the client-side model.

[0009] According to the user intention, generating different interaction results for the intention recognition result;

[0010] If it is determined that the user intention is the first type of intention, the terminal side model directly processes and returns the result to complete the interaction;

[0011] If it is determined that the user intention is the second type of intention, it will enter the cloud-side large model for processing, and then return the result to the terminal side to complete the interaction.

[0012] According to some embodiments, the responding to the user request to invoke the end-side intelligent assistant is accomplished by voice and / or clicking on an interactive entrance of the end-side intelligent assistant;

[0013] The user interacts with the terminal-side intelligent assistant in the form of voice or text;

[0014] The end-side model converts and analyzes the voice or text content so that the end-side intelligent assistant can capture the user's intention.

[0015] According to some embodiments, the responding to the user request to wake up the terminal-side intelligent assistant is a global voice wake-up, and the terminal-side intelligent assistant is woken up by a voice wake-up program on any page;

[0016] A transparent page is popped up on the current page to prompt the user that the terminal-side intelligent assistant has been invoked.

[0017] According to some embodiments, the client-side model directly processes and returns a result, including:

[0018] For the first type of intent, if the end-side model determines that the user's intention is achieved by interacting with the end-side intelligent assistant, it enters the interactive chat page; if it determines that the user's intention is achieved by jumping to the corresponding page, the corresponding page is directly called up through routing navigation.

[0019] According to some embodiments, entering the cloud-side big model for processing and then returning the result to the client side to complete the interaction includes:

[0020] For the second type of intent, the user voice request conversion content is brought into the chat interface of the terminal-side intelligent assistant as a query word and displayed in the chat interface after obtaining the feedback result through the cloud-side big model.

[0021] According to some embodiments, the interaction method further comprises:

[0022] The terminal-side intelligent assistant actively generates push messages based on the user traffic data using the terminal-side model and the cloud-side big model, and pushes them to the user through recommendation tags and / or windows.

[0023] According to some embodiments, the device-side intelligent assistant actively generates a push message based on the user movement data using the device-side model and the cloud-side big model, including:

[0024] In response to the user entering a predetermined page, acquiring a behavior sequence through an application interface, the behavior sequence including the user movement line data;

[0025] The behavior sequence is used as an input of the end-side model to perform end-side model reasoning, and an intent classification corresponding to a standard operating procedure is obtained as a reasoning result;

[0026] Sending the intent classification and page context information to the cloud-side big model to obtain a personalized push message;

[0027] The personalized push message is pushed to the user through a pop-up recommendation tag and / or window.

[0028] According to some embodiments, the interaction method further comprises:

[0029] Collect user behaviors, including user page browsing, click and slide behaviors;

[0030] Process the collected user behaviors into behavior sequences and provide corresponding application interfaces for calling.

[0031] According to some embodiments, the behavior sequence is a piece of text used to describe the user behavior within a period of time.

[0032] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the interaction method as described above is implemented.

[0033] According to another aspect of the present invention, there is provided a computing device, comprising:

[0034] Processor; and

[0035] A memory stores a computer program, and when the computer program is executed by the processor, the interaction method as described in any one of the above items is implemented.

[0036] According to an embodiment of the present invention, an interactive method for combining end-cloud models based on a language interactive interface is provided. The end-side intelligent assistant is called up by responding to user requests through voice or clicking on the interactive entrance of the intelligent assistant. The user's intention is judged by the end-side model. If the user's intention is judged to be the first type of intention, it is directly processed and the result is returned. If the user's intention is judged to be the second type of intention, it enters the cloud-side large model for processing, and then returns the result to the end side to complete the interaction. The present invention can achieve the purpose of weakening the user interaction boundary by being able to call up the intelligent assistant on any page, making the operation process more flexible, reducing the user interaction cost, solving the user's intention through the end-cloud combination, and reducing the large model interaction delay time.

[0037] According to some embodiments, for simple queries or instructions, the end side can directly process and quickly return results without waiting for network transmission time, achieving low-latency response, thereby providing a faster user experience.

[0038] According to some embodiments, simple requests can be processed on the device without uploading to the cloud, which reduces the transmission of user data on the Internet and helps protect user privacy. Users can experience a smoother and more natural interaction experience because even complex requests can be answered in a timely and high-quality manner, providing a good user experience.

[0039] According to some embodiments, processing simple tasks on the end side can reduce the load on the cloud server, so that cloud resources can be more effectively used to process complex tasks that require higher computing power. The present invention allows flexible adjustment of which tasks are processed on the end side and which tasks are sent to the cloud for processing according to different user intentions. As technology develops, new functions and services can be added more easily, with higher flexibility and scalability.

[0040] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below.

[0042] Figure 1 A flow chart of an interaction method for combining end-cloud models based on a language interaction interface according to an example embodiment is shown.

[0043] Figure 2 A schematic diagram of a product architecture according to an example embodiment is shown.

[0044] Figure 3a A schematic diagram of a simple user interaction flow according to an example embodiment is shown.

[0045] Figure 3b A schematic diagram of a user interaction flow according to an example embodiment is shown.

[0046] Figure 4a A schematic diagram showing initialization of a mobile terminal interactive effect implementation solution according to an example embodiment.

[0047] Figure 4b A schematic diagram of user behavior collection according to a mobile terminal interaction effect implementation scheme according to an example embodiment is shown.

[0048] Figure 4c A schematic diagram of a global wake-up link for implementing a mobile terminal interaction effect according to an example embodiment is shown.

[0049] Figure 4d The present invention illustrates the process of actively delivering content based on the terminal-cloud model combination according to the mobile terminal interactive effect implementation solution of the exemplary embodiment.

[0050] Figure 5 A schematic diagram of a terminal-side intelligent message push delivery model according to an example embodiment is shown.

[0051] Figure 6 A schematic diagram showing a terminal-side model architecture according to an example embodiment is shown.

[0052] Figure 7 A block diagram of a computing device is shown according to an exemplary embodiment. DETAILED DESCRIPTION

[0053] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0054] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0055] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0056] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0057] It should be understood that although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another component. Therefore, the first component discussed below can be referred to as the second component without departing from the teachings of the present inventive concept. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more.

[0058] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0059] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present invention, and therefore cannot be used to limit the protection scope of the present invention.

[0060] The rise of the digital economy, the e-commerce industry is a part of the digital economy, using information technology to achieve online transactions of goods and services, greatly facilitating consumers' shopping experience. With the popularization of the Internet and mobile Internet, the digital economy has become a new driving force for global economic growth. Consumers' increasing demand for personalization, immediacy and interactivity has prompted e-commerce companies to explore new marketing methods.

[0061] Currently, the artificial intelligence assistants on e-commerce platforms optimize dialogue strategies by simulating user behavioral feedback during human-computer interaction, transferring knowledge learned from large-scale data sets to specific tasks, such as product recommendation systems, and combining multiple types of data such as text, images, and sounds to improve understanding and generation capabilities.

[0062] The smart assistants on the current mainstream e-commerce platforms have some shortcomings. For example, in the entire shopping chain, the artificial intelligence assistant (AI assistant) cannot be called up anytime and anywhere, and can only be entered through regular entrances, with high learning costs; it can only complete highly standardized interactive behaviors, which is highly restrictive; the smart assistant content return delay is high, and each request requires the cloud-side model to return, which has poor timeliness.

[0063] To this end, the present invention proposes an interaction method based on a language interaction interface for end-cloud model integration, which can weaken the user interaction boundary, make the operation process more flexible, reduce the user interaction cost, and reduce the interaction delay time of large models.

[0064] Before describing the embodiments of the present invention, some terms or concepts involved in the embodiments of the present invention are explained.

[0065] End-cloud integration: The end-side model and the cloud-side large model work together to process user requests.

[0066] Terminal model: an intelligent model (AI model) deployed on the user terminal.

[0067] LUI: Language-based user interface.

[0068] Tips: Recommended tag, which carries the message push that the smart assistant actively generates based on user behavior and other information.

[0069] Window: (Recommended) Window, same as Tips, with a larger interactive area and more information.

[0070] Chat: The main interface of the smart assistant chat, which supports users to interact with the smart assistant.

[0071] SOP: Standard Operating Procedure, which means "standard operating procedure" or "standard operating procedure" in Chinese.

[0072] Exemplary embodiments of the present invention are described below with reference to the accompanying drawings.

[0073] Figure 1 A flow chart of an interaction method for combining end-cloud models based on a language interaction interface according to an example embodiment is shown.

[0074] The current mainstream e-commerce platform smart assistants have some shortcomings, such as using conventional user graphical interface (GUI) interaction, providing proprietary dialogue pages, and completing the interactive closed loop in the form of questions and answers. In the entire shopping chain, the artificial intelligence assistant (AI assistant) cannot be aroused anytime and anywhere, and can only be entered through conventional entrances, with high learning costs; it can only complete highly standardized interactive behaviors, which are highly restrictive; there is basically no AI interactive experience outside of specific interactive areas, and it is impossible to achieve full-process AI accompaniment for user shopping; the intelligent assistant content return delay is high, and each request requires the cloud side model to return, which has poor timeliness. In response to the above problems, the present invention provides an interactive method based on a language interactive interface for the combination of end-cloud models, which can weaken the user interaction boundary, make the operation process more flexible, reduce the user interaction cost, and reduce the interaction delay time of large models.

[0075] join Figure 1 In S101, the terminal-side intelligent assistant is invoked in response to a user request.

[0076] According to some embodiments, the end-side intelligent assistant is invoked by voice and / or clicking on the interaction entrance of the end-side intelligent assistant; the user interacts with the end-side intelligent assistant in voice or text form; the end-side model converts and analyzes the voice or text content so that the end-side intelligent assistant can capture the user's intention.

[0077] Voice wake-up is a global voice wake-up. The end-side smart assistant is woken up through the voice wake-up program on any page; a transparent page pops up on the current page through route navigation to prompt the user that the end-side smart assistant has been woken up. The global voice wake-up function requires the user to grant microphone permission and turn on the global voice function to wake up anytime and anywhere.

[0078] Route navigation refers to the page jump within the APP, such as jumping from the home page to the product details page. Here, it means jumping to a transparent page (without jump animation) after the voice call of the current page, and displaying information such as "I am here" on the new page. The reason for this is that the information of voice interaction can be easily reused in any page through an independent transparent page.

[0079] According to some embodiments, the user can activate the smart assistant through voice commands or by clicking a specific button on the device (such as an icon or physical button on the phone screen). This method allows the user to easily start the smart assistant and start interacting with it. When the user interacts with the smart assistant through voice, the end-side model of the smart assistant converts the voice signal into text and analyzes the text through natural language processing technology to understand the user's intentions. The user can choose to communicate with the smart assistant through voice or text input. This provides flexibility, and users can choose the most appropriate interaction method according to their preferences or current environment.

[0080] In S103, the user intention is determined by using the terminal-side model.

[0081] According to some embodiments, the end-side model is not independent of the application (APP), but can be integrated into the application to serve its own application. The end-side refers to the operation of the model on the user device, not the cloud side. However, the operation of the model will give priority to the features provided by the system for different systems, such as using the built-in reasoning engine of the Hongmeng system and the reasoning acceleration hardware for reasoning acceleration on Hongmeng, which is similar to Android or Apple's iOS system. Since the model and deployment framework are integrated in the APP, the APP can call the capabilities provided by the model when necessary.

[0082] According to an example embodiment, the end-side model classifies the user intent, for example, into a first type of intent that the end-side can directly respond to and a second type of intent that the end-side cannot directly respond to, the former having a corresponding standard operating procedure (SOP) and the latter not. For the first type of intent, the end-side model can directly process it, and for the second type of intent, the end-side model cannot directly process it and needs to be processed by a large cloud-side model.

[0083] According to some embodiments, if the terminal-side model determines that the user's intention is achieved by interacting with the terminal-side intelligent assistant, it enters the interactive chat page; if it determines that the user's intention is achieved by jumping to the corresponding page, the corresponding page is directly called up through the routing navigation.

[0084] In S105, different interaction results are generated according to the intention recognition result according to the user intention.

[0085] According to some embodiments, if the user intention is determined to be the first type of intention, the terminal side model directly processes and returns the result to complete the interaction; if the user intention is determined to be the second type of intention, the cloud side large model is entered for processing, and then the result is returned to the terminal side to complete the interaction.

[0086] The end-side model can quickly process simple requests without communicating with the cloud. The end-side model can convert the user's voice input into a text-based voice model in real time. It can be combined with the user's real-time route data to make judgments and classify the content pushed by the smart assistant. The cloud-side big model infers the user's intention online and generates results. The end-cloud combination is that the user's voice input is converted into text by the end-side model, and then the text query is inferred online by the cloud-side big model. Combined with the intelligent capabilities of the end-side, such as voice, text, and image processing capabilities, user input can be processed by the end capabilities first and then enter the cloud for intent classification to produce results.

[0087] For the first type of intent, or simple intent, if the end-side model determines that the user's intent is achieved through interaction with the end-side intelligent assistant, it enters the interactive chat page; if it determines that the user's intent is achieved by jumping to the corresponding page, the corresponding page is directly called up through routing navigation.

[0088] For the second type of intent, or complex intent, the user voice request conversion content is brought into the chat interface of the terminal-side intelligent assistant as a query word and displayed in the chat interface after obtaining feedback results through the cloud-side big model.

[0089] Simple user intentions are processed through the client-side model. For example, if a user asks "I want to see the current logistics status of my order", the user can be redirected directly to the corresponding result page.

[0090] When the client-side model cannot solve complex intent problems, the cloud-side model is used to process them. For example, if a request is “help me pick out a dress for someone’s wedding next week”, the intent recognition module must perform intent routing before the large model can infer and produce results.

[0091] According to some embodiments, the intelligent assistant is online 24 / 7 and provides real-time services to quickly solve user problems and help guide user shopping decisions. It provides relevant and personalized product recommendations based on the user's query history and personal preferences. It automatically generates product descriptions and review summaries to increase the information content and attractiveness of product pages, and automatically analyzes user feedback to help merchants improve products and services.

[0092] The difference from some current assistant-type interactions (including voice assistants) is that at the interaction level, the general assistant-type tools on the market that are mounted on large models lack global voice awakening and dialogue capabilities, and require operations at designated entrances to initiate voice interactions. Some tools that can be voice awakened are basically within the corresponding application domain and cannot conduct general conversations. Secondly, the common voice assistant-type tools on the market can only answer some general questions or conversations, and can only construct professional question-and-answer assistants for a certain scenario through the creation of various intelligent bodies (agent bodies). For vertical tracks such as e-commerce, these assistant tools on the market cannot meet user needs.

[0093] Figure 2 A schematic diagram of a product architecture according to an example embodiment is shown.

[0094] This solution innovatively adds a language interaction interface based on the LUI large model, reshapes the user's purchasing experience in e-commerce applications, and also pioneers efficient collaboration between the end and the cloud, significantly reducing the round-trip delay of the large model on the cloud side and improving the user experience.

[0095] The core architecture of the present invention can be divided into three layers, namely, the resource layer, the application layer and the interaction layer.

[0096] A variety of end-side models are used in the resource layer to support user interaction, such as voice-to-text, text extraction, and image processing. At the same time, a dedicated end-side model is trained to control the delivery strategy of the recommendation tags generated by AI in real time. The end-side AI model is customized and optimized for e-commerce scenarios, and can respond to users' initial query requests immediately, reduce round-trip latency in the cloud, and improve user experience; the cloud-side large model, as the core of deep learning and complex decision-making, is responsible for handling complex user needs, such as personalized recommendations, in-depth product analysis, etc., and ensures efficient execution of key tasks and continuous learning optimization through the end-cloud combination strategy.

[0097] The application layer provides users with artificial intelligence (AI Agent) capabilities in a variety of e-commerce scenarios. It judges user intent through the end-side model or the end-cloud combined model capabilities according to different user input methods, and provides services and returns results according to the SOP corresponding to the intent. According to some embodiments, the SOPs involved include but are not limited to: finding the same product SOP, comparing products SOP, merchant search SOP, product matching recommendation SOP, list recommendation SOP, general knowledge Q&A SOP, product search SOP, procurement suggestion SOP, product information Q&A SOP, product evaluation SOP, product price SOP, product service SOP, product price trend SOP, store new product SOP, product selection SOP, product quality SOP, direct answer SOP.

[0098] In terms of interaction mode, the interactive layer innovatively applies the language user interface (LUI) based on the large model, and uses the end-cloud combined model to understand user intentions. Users can interact with the application in a natural conversational manner through voice, text or mixed input to perform operations such as product search, price inquiry, purchase suggestion consultation, and order inquiry, which greatly improves the intuitiveness and convenience of interaction.

[0099] Figure 3a A schematic diagram of a simple user interaction flow according to an example embodiment is shown.

[0100] The present invention combines LUI interaction with the end-cloud model, innovatively uses LUI interaction in e-commerce track APPs, and utilizes the cloud-integrated large model capabilities to first capture and parse user requests on the end side to preliminarily determine user intent. A diversion mode is implemented in which the end-side model directly processes and returns results for simple intents, and complex intents enter the cloud-side large model for processing. Users can use voice input to call up the smart assistant on any page in the APP, input purchasing needs through natural language anytime and anywhere, and obtain feedback from the smart assistant with low latency after voice content conversion and analysis by the end-side model; some complex intents are processed by the cloud-side large model and returned in the smart assistant dialogue window, thereby improving the shopping experience and increasing shopping fun. The interaction process of LUI is shown in the figure below. Different interaction processes are generated for different interaction methods and intent recognition results based on user input.

[0101] The user interaction process is as follows: the user initiates a request and puts forward a purchasing demand in the form of voice or text; LUI layer analysis, the application end-side model capability captures and parses the user request, and preliminarily determines the type of intent; the end-side responds immediately, for simple requests, the end-side model directly processes and quickly returns the result; complex requests are transferred to the cloud: if in-depth processing is required, the request is securely encrypted and transmitted to the large model service center; cloud-based processing and learning, the large model performs personalized analysis, recommendations or decisions based on the request; result feedback and execution, the processing results are transmitted back to the client through a secure channel, and the user interface displays and executes the corresponding operations.

[0102] The LUI used in the present invention allows the user's operation mode to gradually transform from a standard state to a non-standard state, making the entire process more flexible, weakening the application's capability boundary, and greatly reducing the user interaction cost. By combining the end and the cloud, on the basis of ensuring the quality of capabilities, some interactive scenarios can omit the return delay to improve the user experience. Active content generation and delivery, instantly obtain the analysis message of the intelligent assistant based on the real-time movement and user situation on the core shopping page, help users make shopping decisions, discover more shopping interests, and provide instant shopping guarantees.

[0103] Figure 3bA schematic diagram of a user interaction flow when a push message is actively generated according to an example embodiment is shown.

[0104] Based on the end-cloud integrated AI capability, the present invention designs an e-commerce smart assistant active content generation and delivery strategy to achieve end-cloud integrated AI active content generation and delivery, as will be described in detail later. Track user movements under the end-side model capability, generate product recommendations, product comparisons, evaluation summaries, product selection, after-sales service and other content based on the user's real-time situation, and actively push them to users through tips and window interaction modes to help users make shopping decisions, discover more shopping interests, and provide instant shopping guarantees. At the same time, for complex scenarios, users can directly use the content of Tips and windows as query words to bring into the smart assistant chat interface, and obtain more and more accurate feedback through the cloud-side large model.

[0105] Through the above architecture and process, the present invention achieves higher interaction efficiency and better personalized experience through an intelligent model system that combines end and cloud, setting a new benchmark for the e-commerce field.

[0106] Figure 4a A schematic diagram showing initialization of a mobile terminal interactive effect implementation solution according to an example embodiment.

[0107] For the global voice wake-up model, after the App is started and enters the homepage, if the user has not authorized it, a tip for enabling global voice wake-up will pop up. If authorized, the model will be automatically loaded and audio collection will be started to continuously identify the preset keywords. For the user tip active prompt model, the loading of the model will be triggered after entering the App homepage. If the model is not on the local disk or the model version number has changed, the model will be downloaded and verified before loading.

[0108] Initialization at app startup occurs when the user interface and interaction logic are built at the entry point. It mainly includes the following two parts: one is to initialize the voice wake-up SDK with user authorization and consent, and the other is to initialize the terminal model.

[0109] Figure 4b A schematic diagram of user behavior collection according to a mobile terminal interaction effect implementation scheme according to an example embodiment is shown.

[0110] User behavior collection and processing mainly includes the collection of user page browsing behavior and click and slide behavior. After collection, they will be processed into behavior sequences and provide corresponding API interfaces for other modules to call. A behavior sequence is a piece of text that describes the user's behavior in the APP over a period of time, for example: #Travel: 1688 homepage, stay time: 15.92s##Travel: search results page, search term: moisturizing lotion, stay time: 15.92s#Travel: product details page, product category: moisturizing lotion, stay time: 11.84s......

[0111] Figure 4c A schematic diagram of a global wake-up link for implementing a mobile terminal interaction effect according to an example embodiment is shown.

[0112] The user can wake up the AI ​​Agent with global voice. When the user says the name of the AI ​​Agent on any page, it can be recognized by the voice wake-up program. At this time, a transparent page (Entry Component) pops up through the route navigation, prompting the user "I am here". The user then says his intention, and the microphone obtains his audio input and converts it into text. At this time, after the intention classification of the end-side model, it can be determined that if the user's intention can be satisfied by the AI ​​Agent, the user will enter the AI ​​Agent chat page. If the user wants to go to a specific page, the corresponding page will be directly called up through the route navigation, avoiding the complex steps in the traditional GUI interaction method where the user must exit to the homepage before entering other pages.

[0113] Figure 4d The present invention illustrates the process of actively delivering content based on the terminal-cloud model combination according to the mobile terminal interactive effect implementation solution of the exemplary embodiment.

[0114] join Figure 4d According to some embodiments, the terminal-side intelligent assistant actively generates push messages based on user movement data using the terminal-side model and the cloud-side big model, and pushes them to the user through recommendation tags and / or windows.

[0115] Tips and windows are both forms of content generated by smart assistants, and the difference lies in the ability to carry content. Tips are more of a display of short information or content prompts to guide users to click to view more content, while windows can carry more information and display more content. For example, in the new news, new products of the stores you follow will be generated and pushed based on the traffic data. First, a tip will pop up like "New products in the store you follow, come and see xxxx". When the user confirms that the tips information is meaningful to the needs of the current scenario, he will click on the tips, and then the window will present the specific new products.

[0116] According to some embodiments, as described above, user behaviors are collected, including the collection of user page browsing behaviors, click and slide behaviors. After collection, the user behaviors are processed into behavior sequences, and corresponding interfaces are provided for other modules to call. The behavior sequence can be a piece of text used to describe user behaviors over a period of time.

[0117] According to some embodiments, in response to a user entering a predetermined page, the end-side intelligent assistant obtains a behavior sequence through an application interface, and the behavior sequence includes the user movement line data. Then, the behavior sequence is used as the input of the end-side model to perform end-side model reasoning, and obtain the intent classification corresponding to the standard operating procedure as the reasoning result. For example, after the user enters the target page (home page, product details page, etc.), the end-side model calculation will be triggered. Model calculation is to obtain the user behavior sequence through the interface provided in the user behavior collection and processing part above, as the input of the end-side model, perform end-side model reasoning, and obtain the reasoning result. The reasoning result is the most likely user intention.

[0118] After that, the intent classification and page context information are sent to the cloud-side big model to obtain a personalized push message. Then, the personalized push message is pushed to the user through a pop-up recommendation tag and / or window. For example, the intent ID and context information are sent up to the cloud-side big model for processing and the result is returned, and a tip is displayed on the current page to guide the user to click to enter the smart assistant chat mode, and make personalized recommendations or in-depth product analysis based on the user's intent.

[0119] For example, after a user opens the app, he enters the favorites page, and then enters the details page of a certain product from the favorites. At this time, the user's movement path will be input into the client-side intent routing model, and the model will generate an intent ID for the product price trend SOP. Then the client will send the intent ID representing the product price trend SOP, as well as the offer ID (quotation ID) and category information corresponding to the product to the server-side cloud-side big model. The cloud-side big model generates a personalized tip based on the product title, category and other information, as well as the information of the behavioral intention: for example, the product price has hit a new low, come and have a look~, and finally returns it to the client for display.

[0120] Figure 5 A schematic diagram of a terminal-side intelligent message push delivery model according to an example embodiment is shown.

[0121] Figure 5 A closed-loop diagram showing the iterative improvement of the online Tips model and the route data optimization, including online data collection, constructing tips model training data through prompts, optimizing route data collection, and updating the model again. This process can be repeated online through multiple closed-loop iterations until the desired effect is achieved.

[0122] In the present invention, the offline effect is judged based on the indicators on the training set and the validation set. The combined indicators on the training set and the validation set can help us judge the quality of the current data on the movement route. In the cold start phase, the user movement route data is the movement route data of the existing users on the application. There are problems of incomplete movement routes and low quality. The model trained at this time is only the initial version when it is launched. On the online side, the real and complete movement route data will be used to retrain and update the online model version. Since the user's movement route will change in real time, the movement route Tips model on the online side will be updated at the daily level to fully learn the user's latest movement route behavior sequence pattern. Finally, according to indicators such as the click-through rate of the agent intelligent tool, the corresponding movement route data behind the agent with a low click-through rate will be optimized in a targeted manner. The new supplementary movement route data will be reflected in the subsequent model training, and the new model will make the agent tool with a low click-through rate predict better, thereby improving its click-through rate. Low agent tool click-through rate indicator-optimization of movement route data-model upgrade-low agent tool click-through rate improvement, forming a feedback closed loop.

[0123] In the implementation of the traffic flow combined with the end-side model to deliver recommended content, the core of the end-side intelligent Tips delivery model is to quickly and intelligently predict the user's next desired action based on the user's historical traffic flow information, and guide the user to interact deeply with Yuanbao in the form of tips at the foreground.

[0124] The implementation of the end-side tips model mainly includes the following aspects:

[0125] Data flywheel: The core of the data side is to structure the user's recent movement information. According to the capabilities of each agent, the user movement information is redesigned. In the data cold start phase, according to the required user movement information, the current cloud point information is filtered and selected. The movement information mainly includes the type of page currently entered, the page function description, the length of time the page stays, and the user search terms and product category information associated with the current page. Relying on the powerful language understanding and logical reasoning capabilities of the current large language model (LLM), the open source qwen2-72b model is used to intelligently label the structured user movement data, and the prompt word is used to complete the classification task of the user movement data to the agent tool that the next user may use. This process can be iterated online periodically, and the movement data is adjusted based on the model effect to form a data closed loop.

[0126] The qwen2-72b model is a large language model with 72 billion parameters trained and open-sourced by the Alibaba team. The model supports completing user-specified task requirements through dialogue, such as role-playing, code generation, logic analysis, intent recognition, and other tasks. In the present invention, the qwen2-72b large model is used to help us clean and build a training data set for training the end-side Tips recognition small model, thereby avoiding the expensive cost of manually annotating data.

[0127] Prompt is the language for people to interact with the big model. Based on prompt, the big model can complete the task requirements specified by humans. In this invention, we require the big model to infer the agent tool that the user is most likely to use next based on the user's recent behavior route data, so as to construct the training sample of the small model on the end side. The agent tool is the category to be used by the Tips small model on the end side. There are currently 12 types of agent tools: product search, product comparison, list recommendation, recommended merchants, product price, same product, consulting product details, product service, product evaluation, product matching, general information assistant, and product selling point.

[0128] prompt example:

[0129] <task> Based on the user's recent behavior, infer the tool that the user is most likely to use next from the list of tools provided.< / task>

[0130] <Tool List>

[0131] Recommended merchants: This tool can help recommend merchants, factories, and stores on e-commerce platforms such as Taobao and Tmall (you can only recommend online stores and factories).

[0132] Same products: This tool can help users search for the same or similar products of a certain product.

[0133] Product Search: This tool can be used to search for products with certain services, functions, materials, specifications, or a certain category.

[0134] Product matching: This tool can help users solve problems such as how to wear and match products.

[0135] Product selection: This tool can help users analyze how to select individual products and similar issues (including product selection and product purchasing skills), and give suggestions on how to select products.

[0136] Product comparison: This tool can help users select and compare multiple products, and then provide a detailed comparison result.

[0137] Product Reviews: This tool can help users view product reviews, product reputation and other similar information.

[0138] Merchandise Services: This tool is used when users need to view the service information supported by a product or merchant (including shipping, customization, drop - shipping, etc.).

[0139] Merchandise Price: This tool is used when users inquire about the price information of a product, including but not limited to discounts, promotions, competitive prices, wholesale prices, etc.

[0140] Merchandise Price Trend: This tool helps users view the historical price changes of a product and determine whether the current product is at a low - price point. Note that this tool can only be used to judge price trends.

[0141] Merchandise Selling Points: This tool helps users view the highlights of a product, summarize purchasing advice information, and determine whether the current product is worth buying.

[0142] Consult Product Details: This tool helps users view detailed information related to a product, such as minimum order quantity, shipping speed, size specifications, color and material, style, attributes, and other similar issues.

[0143] List Recommendation: This tool helps users analyze business opportunity information such as the popularity, sales volume, and trends of a certain product in the market, and recommends product lists that are hot - selling, have soaring popularity, or have been on the list for a long time on platforms such as Taobao and Douyin to users.

[0144] General Information Assistant: Helps users answer general knowledge questions in the e - commerce field, general knowledge questions in non - e - commerce fields, membership service questions, after - sales service questions, logistics questions, order questions, code questions, math questions, logical reasoning, location - based questions, and questions that require online search.

[0145] < / Tool List>

[0146] <Requirements>

[0147] 1. After analyzing that the user has the intention to search for more similar products, use the 'Product Search' tool.

[0148] 2. After analyzing that the user has the intention to be interested in a certain type of product, use the 'List Recommendation' tool.

[0149] < / Requirements>

[0150] <User Movement Data>

[0151] {user_track}

[0152] < / User Movement Data>

[0153] <Output Format>

[0154] <analyze> Based on the user's flow, guess the tool that the user is most likely to use next from the provided tool list.< / analyze>

[0155] <tool> The name of the tool you choose (strictly from the list of tools provided above)< / tool>

[0156] < / Output format>

[0157] Figure 6 A schematic diagram showing an edge - side model architecture according to an exemplary embodiment.

[0158] An edge model refers to a machine - learning model that runs on user devices (such as smartphones, tablets, and smart home devices, etc.) or edge servers (servers close to the data source). Compared with cloud - side models, edge models need to meet some specific requirements, mainly considering the following two points. One is low latency, ensuring that the inference time of the model on edge devices is as short as possible. The other is resource efficiency. Since the RAM cache and storage space of edge devices are limited, the model should significantly reduce the memory and storage requirements during operation, and control the number of parameters and the size of activation layers.

[0159] The Tips model is essentially an agent - tool classification model. Considering some characteristics of edge models, the ALBERT model is adopted as the base model and fine - tuned using the "user movement path, agent tool" offline labeled data. ALBERT (A Lite BERT) is an improved BERT variant proposed by the Google AI team. Its main goal is to reduce the parameter scale to improve the model's efficiency while maintaining or enhancing the model's performance. The key innovation of ALBERT during the pre - training process lies in optimizing parameter sharing and factorization to significantly reduce the model parameters without significantly sacrificing performance.

[0160] During the model fine - tuning training, the open - source transformers framework is used for rapid training, and after the model training is completed, the model parameters are exported in the onnx format to facilitate the deployment of the model on the system. Transformers is an open - source third - party library that supports data loading and rapid model fine - tuning training. It contains a built - in model collection, and the albert model is just one model implemented using this framework.

[0161] The edge - side model deployment framework is implemented by abstracting multiple inference engines, and then running a set of inference codes on multiple different inference frameworks, so as to achieve an optimal solution for performance and package size by leveraging the inference engines provided by different vendors. At the same time, multiple data pre - processing post - processors are developed and built to be compatible with the inferences of different AI tasks.

[0162] The performance and package size optimization of the present invention utilize the built - in inference engines of different vendor systems to exert the hardware performance, achieving the minimum dependence, the smallest package size, and optimized performance. The present invention can be cross - platform and adapted to different operating systems, including IOS, Android, and Harmony, etc.

[0163] The model of the present invention can run on systems with or without built-in inference engines. The model of the present invention and the related technologies for deploying the model are essentially independent of any platform. The model deployment framework also has a built-in ONNX Runtime inference engine (a cross-platform machine learning model accelerator), which can support running on any device with a CPU processor. In essence, by abstracting the implementation of multiple inference engines, it is possible to mount any one or more inference engines and run our model. Therefore, if the platform has a built-in inference engine, the inference engine in the platform can be used first to achieve lower power consumption, higher performance inference and minimize dependence, because the platform's built-in engine can better drive the platform's built-in hardware.

[0164] The present invention supports a variety of AI tasks, has a variety of built-in preprocessors, and supports a variety of AI tasks including audio, NLP natural language processing, cv vision, etc. The present invention can realize dynamic model updates, does not rely on App release, supports dynamic distribution of models and configurations, and realizes automatic update and loading of models in the deployment framework.

[0165] Figure 7 A block diagram of a computing device is shown according to an exemplary embodiment.

[0166] like Figure 7 As shown, computing device 30 includes processor 12 and memory 14. Computing device 30 may also include bus 22, network interface 16, and I / O interface 18. Processor 12, memory 14, network interface 16, and I / O interface 18 may communicate with each other via bus 22.

[0167] The processor 12 may include one or more general-purpose CPUs (Central Processing Units, processors), microprocessors, or application-specific integrated circuits, etc., for executing relevant program instructions. According to some embodiments, the computing device 30 may also include a high-performance graphics card (GPU) 20 for accelerating the processor 12.

[0168] The memory 14 may include a machine system readable medium in the form of a volatile memory, such as a random access memory (RAM), a read-only memory (ROM) and / or a cache memory. The memory 14 is used to store one or more programs including instructions and data. The processor 12 can read the instructions stored in the memory 14 to execute the above-mentioned method according to the embodiment of the present invention.

[0169] The computing device 30 may also communicate with one or more networks via the network interface 16. The network interface 16 may be a wireless network interface.

[0170] The bus 22 may include an address bus, a data bus, a control bus, etc. The bus 22 provides a path for exchanging information between components.

[0171] It should be noted that, in the specific implementation process, the computing device 30 may also include other components necessary for normal operation. In addition, those skilled in the art may understand that the above device may only include components necessary for implementing the embodiments of this specification, and need not include all components shown in the figure.

[0172] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a microdrive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), a network storage device, a cloud storage device, or any type of medium or device suitable for storing instructions and / or data.

[0173] An embodiment of the present invention further provides a computer program product, which includes a computer program. The computer program is operable to enable a computer to execute part or all of the steps of any one of the methods described in the above method embodiments.

[0174] Those skilled in the art can clearly understand that the technical solution of the present invention can be implemented with the help of software and / or hardware. "Unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array, an integrated circuit, etc.

[0175] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0176] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0177] In the several embodiments provided by the present invention, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0178] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0180] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention.

[0181] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0182] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended clauses.

Claims

1. An interactive method for combining terminal and cloud models based on a language interactive interface, characterized in that: include: Respond to user requests and invoke the client-side intelligent assistant; Determine user intent through the client-side model. According to the user intention, different interaction results are generated for the intention recognition result, wherein: If it is determined that the user intention is a first type of intention that the terminal side can directly respond to, the terminal side model directly processes and returns the result to complete the interaction; If it is determined that the user intention is a second type of intention that the terminal side cannot directly respond to, it enters the cloud side large model for processing, and then returns the result to the terminal side to complete the interaction.

2. The interactive method according to claim 1, characterized in that: The responding to the user request to call up the end-side intelligent assistant is completed by voice and / or clicking on the interactive entrance of the end-side intelligent assistant; The user interacts with the terminal-side intelligent assistant in the form of voice or text; The end-side model converts and analyzes the voice or text content so that the end-side intelligent assistant can capture the user's intention.

3. The interactive method according to claim 2, characterized in that: The responding to the user request to wake up the terminal side intelligent assistant is a global voice wake-up, and the terminal side intelligent assistant is woken up by the voice wake-up program on any page; A transparent page is popped up on the current page to prompt the user that the terminal-side intelligent assistant has been invoked.

4. The interactive method according to claim 1, characterized in that: The client-side model directly processes and returns results, including: For the first type of intent, if the end-side model determines that the user's intention is achieved by interacting with the end-side intelligent assistant, it enters the interactive chat page; if it determines that the user's intention is achieved by jumping to the corresponding page, the corresponding page is directly called up through routing navigation.

5. The interactive method according to claim 1, characterized in that: The processing of the data in the cloud-side big model and the result returned to the client side to complete the interaction includes: For the second type of intent, the user voice request conversion content is brought into the chat interface of the terminal-side intelligent assistant as a query word and displayed in the chat interface after obtaining the feedback result through the cloud-side big model.

6. The interactive method according to claim 1, characterized in that: Also includes: The terminal-side intelligent assistant actively generates push messages based on the user traffic data using the terminal-side model and the cloud-side big model, and pushes them to the user through recommendation tags and / or windows.

7. The interactive method according to claim 6, characterized in that: The device-side intelligent assistant actively generates a push message based on the user movement data using the device-side model and the cloud-side big model, including: In response to the user entering a predetermined page, acquiring a behavior sequence through an application interface, the behavior sequence including the user movement line data; The behavior sequence is used as an input of the end-side model to perform end-side model reasoning, and an intent classification corresponding to a standard operating procedure is obtained as a reasoning result; Sending the intent classification and page context information to the cloud-side big model to obtain a personalized push message; The personalized push message is pushed to the user through a pop-up recommendation tag and / or window.

8. The interactive method according to claim 7, characterized in that: Also includes: Collect user behaviors, including user page browsing, click and slide behaviors; Process the collected user behaviors into behavior sequences and provide corresponding application interfaces for calling.

9. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the interaction method according to any one of claims 1 to 8.

10. A computing device, characterized in that: include: processor; as well as A memory storing a computer program, which, when executed by the processor, implements the interaction method according to any one of claims 1 to 8.

Images