AI interaction method and device, wearable equipment and storage medium

Abstract

The invention provides an AI interaction method and device, wearable equipment and a storage medium, and belongs to the technical field of man-machine interaction, and the method comprises the following steps: constructing a user portrait based on multi-source data; the multi-source data is data of a target user, and the target user is a user wearing the wearable device. And in response to the received first voice of the target user, determining a target intention type based on the first voice. And determining multi-modal sensing data of the target user based on the target intention type. And determining associated information corresponding to the first voice based on the multi-modal perception data. And determining recommendation information corresponding to the first voice based on the associated information and the user portrait. And generating a second voice based on the association information and the recommendation information. The second voice is voice played by AI in the wearable device. According to the AI interaction method and device, the wearable device and the storage medium provided by the invention, the user experience can be improved, and diversified requirements of users can be met.

Description

Technical Field

[0001] The present disclosure belongs to the field of human-computer interaction technology, and more specifically, to an AI interaction method and apparatus, a wearable device, and a storage medium. Background Art

[0002] With the development of science and technology, people's expectations for human-computer interaction are increasing. Traditional interaction methods such as keyboards and mice have gradually failed to meet the needs of diversity and convenience. People's demand for more natural and convenient human-computer interaction has promoted the progress of AI interaction methods. In terms of voice interaction, acoustic models and language models continue to evolve, gradually shifting from models based on traditional statistical methods to neural network models based on deep learning, such as deep neural networks, recurrent neural networks and their variants, long short-term memory networks and gated recurrent units. The development of these technologies has led to continuous innovation in speech recognition and synthesis technology.

[0003] However, current AI interaction technology still has problems with poor user experience and difficulty in meeting the diverse needs of users. Summary of the invention

[0004] The purpose of the present disclosure is to provide an AI interaction method and apparatus, a wearable device, and a storage medium to enhance user experience and meet the diverse needs of users.

[0005] According to a first aspect of an embodiment of the present disclosure, an AI interaction method is provided, which is applied to a wearable device, and includes: Constructing a user portrait based on multi-source data; the multi-source data is data of a target user, and the target user is a user wearing the wearable device; In response to receiving a first speech of the target user, determining a target intent type based on the first speech; determining multimodal perception data of the target user based on the target intent type; determining associated information corresponding to the first speech based on the multimodal perception data; and determining recommendation information corresponding to the first speech based on the associated information and the user portrait; A second voice is generated based on the associated information and the recommended information; the second voice is the voice played by the AI ​​in the wearable device.

[0006] A second aspect of the embodiments of the present disclosure provides an AI interaction device, which is applied to a wearable device, including: A first computing module, configured to construct a user portrait based on multi-source data; the multi-source data is data of a target user, and the target user is a user wearing the wearable device; a second calculation module, configured to determine, in response to receiving a first voice of the target user, associated information corresponding to the first voice, and determine recommended information corresponding to the first voice based on the associated information and the user portrait; The third computing module is used to generate a second voice based on the associated information and the recommended information; the second voice is the voice played by the AI ​​in the wearable device.

[0007] According to a third aspect of an embodiment of the present disclosure, a wearable device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the above-mentioned AI interaction method when executing the computer program.

[0008] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned AI interaction method are implemented.

[0009] The AI ​​interaction method and device, wearable device, and storage medium provided by the embodiments of the present disclosure have the following beneficial effects: By constructing a user portrait based on multi-source data, the disclosed embodiment can gain an in-depth understanding of user characteristics, preferences, and habits, provide users with suggestions that suit their personalities, and improve user satisfaction. The disclosed embodiment combines the associated information of the first voice, including hidden information, contextual information, etc., and is not limited to the surface content of the voice. It can more accurately understand the user's true intentions in a specific environment and state, avoid misunderstandings, effectively improve the accuracy of interaction, and enhance trust between users and wearable devices. In summary, the disclosed embodiment enables wearable devices to better meet the diverse needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A flowchart of an AI interaction method provided by an embodiment of the present disclosure; Figure 2 A structural block diagram of an AI interaction device provided in one embodiment of the present disclosure; Figure 3 A schematic block diagram of a wearable device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0012] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present disclosure. However, it should be clear to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present disclosure with unnecessary details.

[0013] In order to make the purpose, technical solutions and advantages of the present disclosure more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0014] Please refer to Figure 1 , Figure 1 This is a flow chart of an AI interaction method provided in one embodiment of the present disclosure. The method is applied to a wearable device and may include S101 to S103.

[0015] S101: Build a user portrait based on multi-source data. The multi-source data is data of a target user, and the target user is a user wearing a wearable device.

[0016] In this embodiment, the multi-source data includes application behavior data, historical interaction data, and user identity information.

[0017] Build user profiles based on multi-source data, including: Build user profiles based on application behavior data, historical interaction data, and user identity information.

[0018] In this embodiment, the wearable device may include smart glasses. Multi-source data refers to a collection of data from multiple different channels or aspects, and the multi-source data may include data surrounding a target user and related to the wearable device. The target user is an individual who is wearing a specific wearable device, and the data collected and analyzed by the device is all targeted at the user's situation.

[0019] Application behavior data may include the type of application used, frequency of use, duration of use, in-application operation path, etc. Historical interaction data may include content involving voice interaction, interaction time, interaction frequency, interaction method, and feedback on interaction results. User identity information may include age, gender, occupation, health status, living area, etc. By collecting multi-source data of different dimensions, mining the rules and associations, analyzing the user's behavior patterns, interests, hobbies, living habits and needs, we can outline the user portrait and provide a basis for personalized services.

[0020] For example, a dedicated data collection module is set up in the wearable device to record application behavior data, historical interaction data in real time, and obtain user identity information when the user sets up the device for the first time. The collected data is classified and stored by category, such as establishing different database tables to store application behavior, historical interaction and user identity information data respectively, to facilitate subsequent processing.

[0021] Use data analysis and machine learning techniques, such as cluster analysis and decision tree algorithms. For application behavior data, analyze the usage frequency, duration distribution and other characteristics of different applications. For historical interaction data, extract common interaction patterns, instruction preferences, etc. For user identity information, combine the other two types of data to see the behavioral differences of users with different identity attributes. By integrating these analysis results, construct user portraits in the form of vectors or structured data, for example, using the user's preference for different applications, common interaction behaviors, etc. as different dimensions of the portrait.

[0022] User portraits can be updated regularly, and the characterization of users can be continuously optimized and improved based on newly collected data to make it more in line with the user's actual situation.

[0023] S102: In response to receiving a first voice of a target user, determine a target intent type based on the first voice. Determine multimodal perception data of the target user based on the target intent type. Determine associated information corresponding to the first voice based on the multimodal perception data. Determine recommendation information corresponding to the first voice based on the associated information and the user portrait.

[0024] In this embodiment, the associated information corresponding to the first voice may include hidden information corresponding to the first voice.

[0025] The first voice refers to the voice commands issued by the wearable device users, which have various contents, such as "I am a little tired and want to relax."

[0026] The associated information is data related to the first voice, including the content directly expressed in the voice and potential hidden information. Hidden information can be user emotions and potential needs inferred through voice intonation, specific vocabulary, etc. It can also include contextual information when the voice is uttered, such as the noise level of the environment in which the device is located, the user's motion status, etc. It can also include information hidden in the translated text corresponding to the first voice. For example: the first voice is: "What delicious food is nearby", the associated information corresponding to the first voice (that is, hidden information) can include content corresponding to "nearby" and "eat", such as determining and generating the location of the target user, generating restaurants within a radius of a kilometers, and determining the restaurant name and location, and the distance from the target user.

[0027] User portrait is a description of the comprehensive characteristics of the user, including information such as the user's interests, hobbies, living habits, and health status.

[0028] Recommended information is a suggestion given based on related information and user portraits to meet the user's current needs, such as recommending a quiet cafe nearby or providing soothing music playback suggestions.

[0029] For example, after receiving the first voice, the voice is first analyzed and processed to obtain its key content and relevant context information, and the corresponding related information is generated. Combined with the constructed user portrait, which includes the user's preferences, habits, etc., the recommended information that meets the user's needs and characteristics is screened from the resource library.

[0030] For example, the first speech is converted into text content using speech recognition technology, and then the intention and key entities of the text are analyzed using natural language processing technology. At the same time, the current environment and user status information are obtained by combining the sensors of the wearable device to determine the associated information. According to the key content in the associated information, a matching search is performed in the target retrieval resource library determined based on the user portrait to generate recommended information.

[0031] In this embodiment, the target intention types include information query and life service. The multimodal perception data includes user physiological data and environmental perception data.

[0032] Determine the multimodal perception data of the target user based on the target intent type, including: If the target intention type is information query, the multimodal perception data of the target user is determined to be environmental perception data.

[0033] If the target intention type is life service type, the multimodal perception data of the target user is determined to be user physiological data and environmental perception data.

[0034] In this embodiment, the target intent type refers to the main purpose category expressed by the user through the first voice, which can be divided into information query category (such as asking for news, weather and other information) and life service category (such as ordering takeout, making appointments, and other needs related to improving life convenience).

[0035] Multimodal perception data refers to the acquired text, audio, image and other data, which is used to gain a more comprehensive understanding of the user status and surrounding environment.

[0036] User physiological data may include parameters reflecting the user's physical condition, such as heart rate, blood pressure, body temperature, body posture, and fatigue level.

[0037] Environmental perception data can include information about the temperature, humidity, light intensity, noise level, geographic location, and surrounding object identification around the wearable device. Wearable devices can be connected to smart bracelets or other auxiliary electronic devices to obtain multimodal perception data.

[0038] Exemplarily, a large language model is used to perform text recognition and semantic analysis on the first speech. By training a neural network-based classifier, the speech is classified into information query or life service categories according to features such as keywords and grammatical structures in the speech content.

[0039] Integrate appropriate sensors such as heart rate sensors and accelerometers into wearable devices. Heart rate sensors continuously monitor heart rate data, and accelerometers can determine body posture and exercise intensity by analyzing movement patterns. Filter and calibrate the collected data before storing it. Alternatively, connect the relevant sensors to wearable devices (smart glasses) for data transmission.

[0040] Equipped with temperature sensors, humidity sensors, microphones, GPS modules, etc. Each sensor collects environmental information in real time and performs corresponding unit conversion, data fusion and other data processing.

[0041] After the intent type is determined, the corresponding multimodal perception data is obtained from the data acquisition module. If it is an information query type, only the environmental perception data is extracted. If it is a life service type, the user's physiological data and environmental perception data are extracted. Then these data are combined with the content of the first voice, and the associated information is determined through a data analysis algorithm (such as association rule mining). For example, in the life service type, based on the user's fast heart rate and high ambient temperature, the user is recommended a heat-relief-related service or recommendation.

[0042] This embodiment can accurately understand user needs by distinguishing target intent types and combining multimodal perception data, taking into account the actual situation of the user and the environment, and provide related information that is more suitable for the user's current situation, thereby improving the accuracy of recommendations. This embodiment combines user physiological and environmental data to fully understand the user. This avoids the one-sidedness of focusing on a single factor, so that the related information can reflect the user's true state, better meet user needs, and enhance user experience. Accurate related information helps to improve the quality of interaction, allowing users to feel the intelligence and intimacy of the device, and enhancing users' trust and dependence on wearable devices.

[0043] S103: Generate a second voice based on the associated information and the recommended information. The second voice is the voice played by the AI ​​in the wearable device.

[0044] In this embodiment, the recommended information is content that is useful to the user based on the associated information and the user portrait, such as a list of restaurants and reviews that meet the user's taste and distance requirements. The second voice is the device's response voice to the user, which feeds back the recommended information and other content to the user in voice form. The second voice can be set with a suitable tone according to the type of recommended information, such as a cheerful tone for recommending entertainment activities and a calm tone for important notifications. The second voice may include specific text content, prompts, etc. of the recommended information, such as "The following restaurants are recommended for you". The voice playback duration of the second voice can be determined based on the amount of content.

[0045] For example, based on the associated information and recommended information, a natural language generation template or model is used to generate smooth and logical text content. A speech synthesis engine is used to select a suitable timbre to convert the generated text into a second voice, and the intonation, speech speed and other parameters of the voice are adjusted to make it more natural, and finally played to the user through the speaker of the wearable device.

[0046] From the above, it can be concluded that this embodiment can provide users with highly personalized recommendation information through detailed user portraits and in-depth analysis of the first voice. This embodiment combines hidden information in the associated information, such as user emotions, potential needs, and environmental and status information, to make the interaction more intelligent. It avoids simply responding to the surface content of the user's voice, and can better understand the user's true intention, accurately give suggestions that meet the current situation, and reduce misunderstandings and invalid interactions.

[0047] Filtering recommended information based on user portraits and related information can effectively utilize the data in the resource library. Avoiding the search and processing of a large amount of irrelevant information improves operational efficiency and provides users with valuable content more quickly. Regularly updating user portraits can ensure that the interaction process always fits the user's latest behavior patterns and demand changes, thereby improving user experience and meeting the diverse needs of users.

[0048] In one embodiment of the present disclosure, building a user profile based on application behavior data, historical interaction data, and user identity information includes: The application behavior data is divided based on the application type to obtain application behavior data corresponding to multiple time periods, where each time period corresponds to an application type.

[0049] A first partial user portrait corresponding to the application behavior data is determined based on the application type and data volume corresponding to each time period.

[0050] A second local user portrait is determined based on the historical interaction data.

[0051] A third partial user portrait is determined based on the user identity information.

[0052] Based on the weight coefficient matrix corresponding to the application behavior data, the historical interaction data and the user identity information, the first local user portrait, the second local user portrait and the third local user portrait are fused to obtain a user portrait.

[0053] In this embodiment, determining the first partial user portrait corresponding to the application behavior data based on the application type and data volume corresponding to each time period includes: An application type-data volume sequence is determined based on the application type and data volume corresponding to each time period, and a first local user portrait is determined based on the application type-data volume sequence.

[0054] In this embodiment, the application type-data volume sequence includes an application type sequence and a data volume sequence corresponding to the application type sequence, and the application type sequence and the data volume sequence are mapped to each other in the dimension of time period.

[0055] In this embodiment, determining the second partial user portrait based on the historical interaction data includes: The historical interaction data is divided based on the interaction type to obtain the historical interaction data corresponding to multiple time periods, where each time period corresponds to an interaction type.

[0056] An interaction type-data amount sequence is determined based on the interaction type and data amount corresponding to each time period, and a second local user portrait is determined based on the interaction type-data amount sequence.

[0057] In this embodiment, the interaction type-data volume sequence includes an interaction type sequence and a data volume sequence corresponding to the interaction type sequence, and the application type sequence and the data volume sequence are mapped to each other in the dimension of time period.

[0058] In this embodiment, determining the third partial user portrait based on the user identity information includes: determining the third partial user portrait from a user tag library based on the user identity information.

[0059] In this embodiment, application types may include social applications, entertainment applications, health applications, office applications, etc. The identification of application types facilitates the division of application behavior data, and each application behavior data subset after division corresponds to a unique time interval and data volume. The data volume refers to the number of data records generated by a certain application type or interaction type within a certain time period.

[0060] The first partial user portrait refers to the part of the portrait that preliminarily depicts the user's characteristics and habits from the perspective of the target user's application usage behavior. It mainly reflects the user's usage preferences and usage activity for different types of applications at different times, and is part of the content that constitutes a complete user portrait. The second partial user portrait represents the partial content of the portrait in terms of topics that the target user pays attention to, demand directions, and frequency of interaction during the interaction with the system at different times. The third partial user portrait is based on the user's identity information, and matches the relevant tags that match the user from a pre-set set of typical descriptive tags that can represent different identity characteristics (i.e., the user tag library), so as to outline the partial portrait that reflects the user's characteristics, preferences, etc. from the level of basic user attributes, and supplement the feature description related to the identity background for the complete portrait.

[0061] The weight coefficient matrix is ​​a matrix representation that measures the relative importance of application behavior data, historical interaction data, and user identity information. The elements in the weight coefficient matrix correspond to the weight values ​​of each data type when constructing a user portrait. By setting these weights reasonably, the contribution of different data to the final user portrait can be comprehensively coordinated according to business needs and actual application scenarios, thereby achieving a more practical and targeted portrait fusion.

[0062] For example, the weight coefficient matrix of application behavior data, historical interaction data and user identity information can be calculated based on the principal component analysis weighting method: This embodiment divides application behavior data based on application type, which can present the user's usage preferences and activity of various applications at different times in detail. Combined with historical interaction data, it further clarifies the user's needs and interactions in different scenarios, making the portrait more targeted. This embodiment uses user identity information to match relevant tags from the tag library, supplements the user's identity background characteristics, and comprehensively enriches the user portrait content. The weight coefficient matrix is ​​determined by the principal component analysis weighting method, and the contribution of each data type to the portrait is reasonably coordinated to achieve a more realistic and accurate portrait fusion, providing users with personalized services.

[0063] In one embodiment of the present disclosure, determining recommendation information corresponding to the first voice based on the association information and the user portrait includes: Determine the target retrieval resource library based on user portrait.

[0064] Based on the associated information, a reply text corresponding to the first speech is determined from the target retrieval resource library.

[0065] Determine recommended information corresponding to the first voice from a target retrieval resource library based on the reply text.

[0066] In this embodiment, determining a target search resource library based on a user profile includes: Determine user tags and time series features based on user portraits.

[0067] A first search resource library is matched from search resources based on the user tag.

[0068] A target retrieval resource library is determined from the first retrieval resource library based on the temporal characteristics.

[0069] In this embodiment, the target retrieval resource library is selected from numerous retrieval resources, highly matched with the needs and features of the current user, and is used to find a specific database of reply text and recommended information corresponding to the first voice.

[0070] User tags are identifiers extracted from user portraits to describe user characteristics, preferences, behavioral habits, etc., such as "sports enthusiast", "food expert", "night owl", etc., which facilitate quick classification of users and location of relevant resources.

[0071] Temporal features include user characteristics related to the time sequence, which may include parameters such as the user's activity level in different time periods, behavioral preferences in specific time periods, and changes in the frequency of device use over time.

[0072] In this embodiment, user tags and time series features are analyzed based on the constructed user portrait. The user tags can be used to preliminarily screen out the first search resource library that matches the user's general interests and habits. This step is based on the correlation between tags and resources. Then, the time series features are used to further accurately locate in the first search resource library, and find the target search resource library that is more in line with the user's current time-related behavior habits and other situations. Finally, based on the associated information, the corresponding reply text is found in this precise target search resource library, and then the recommended information is determined, so that the recommended information not only meets the overall characteristics of the user but also meets the current specific situation needs.

[0073] For example, the constructed user portraits are analyzed and user tags are identified through feature extraction algorithms, such as determining tags based on the types of applications frequently used by users and the activities they frequently participate in. At the same time, time series analysis algorithms are used to analyze user behavior data in different time periods to extract time series features, such as calculating the mean and variance of the frequency of use in a specific time period. A huge search resource library is established to store resource content in various aspects such as knowledge questions and answers, life services, and hobbies.

[0074] According to the user tag, the text matching algorithm is used to screen out the first search resource library from the search resources, and the resources with high relevance to the user tag are grouped together. Based on the temporal characteristics, the target search resource library is further determined in the first search resource library. For example, if the user is active at night and likes entertainment, the target search resource library with richer entertainment-related resources at night is selected.

[0075] Based on the key content in the associated information, such as user needs, current environment, etc., the reply text corresponding to the first voice is found in the target search resource library through text retrieval technology. Then, the recommended information corresponding to the first voice is further determined based on the guiding information and related resource links in the reply text. For example, if the reply text mentions that a certain restaurant is good, the recommended information can be the detailed address of the restaurant, discount information, etc.

[0076] This embodiment uses the tags and time series features in the user portrait to filter out the target retrieval resource library, which can ensure that the recommended information fits the user's interests and current situation. By accurately locating the target retrieval resource library, it avoids searching in a large number of irrelevant resources, reduces computing resource consumption and time costs, and provides users with answers and recommendations quickly and accurately. Recommended information that meets the user's overall characteristics and current needs allows users to feel that the device understands them, enhances the interaction satisfaction between users and devices, and improves user loyalty.

[0077] In one embodiment of the present disclosure, generating a second voice based on the associated information and the recommended information includes: A first reply text is generated based on the associated information, and a second reply text is generated based on the recommended information.

[0078] The first reply text and the second reply text are concatenated to generate a target text, and a second speech is synthesized based on the target text.

[0079] In this embodiment, the first reply text: the response content generated for the associated information, focuses on feedback on the user's current situation or needs, such as "I understand that you are looking for nearby restaurants. 15 nearby restaurants have been searched for you."

[0080] Second reply text: focuses on the explanation and description of the recommended information, such as "I recommend restaurant A for you. It is 500 meters away from you, has very good reviews, and specializes in your favorite Sichuan cuisine" or "There are hot pot, barbecue, and BBQ nearby. Which kind of food would you like to eat?"

[0081] Target text: The complete text content formed by concatenating the first reply text and the second reply text, which serves as the basis for synthesizing the second speech.

[0082] Second voice: The voice played by AI in wearable devices to users, used to convey related information and recommended information.

[0083] For example, related information: the user asks "I feel a little tired and uncomfortable", and the wearable device recognizes "feeling" "tired" and "uncomfortable", and determines it as a life service. Wake up the relevant devices connected to the wearable device, obtain the user's physiological characteristics and environmental characteristics, and obtain data information about the user's fast heart rate, slightly high blood pressure, and high current ambient temperature and humidity. It is judged that the user may be in a state of fatigue and slight discomfort, and the location information is displayed at home.

[0084] Recommended information: It is recommended that users take a rest first, turn on the air conditioner to adjust the indoor temperature to a comfortable range, and remind users to replenish appropriate amount of water.

[0085] First reply text: "You may feel tired now due to a variety of factors." Second reply text: "You can take a rest first, turn on the air conditioner to a comfortable temperature, and drink some water. Do you need me to call you?" Second voice: "You may feel tired due to many factors. You can take a rest first, turn on the air conditioner to a comfortable temperature, and drink some water. Do you need me to call you?" Exemplary, associated information: the user says "I want to go to a nearby shopping mall", the wearable device connects to the user's mobile phone, identifies "nearby" and "shopping mall", wakes up the map APP in the mobile phone, and obtains that the current user is in a busy traffic area, the surrounding noise is loud, and the user walks slowly.

[0086] This embodiment can fully cover user needs by generating a first reply text and a second reply text respectively and splicing them together, and provide users with complete information from need confirmation to specific suggestions, thereby avoiding information loss.

[0087] The second voice combines the understanding of user context and targeted recommendations, making the interaction more in line with human communication habits, making users feel like they are communicating with an understanding assistant, and improving the user experience. The voice form clearly and accurately conveys related information and recommended information, especially when the user is busy or inconvenient to view the screen, which can efficiently serve the user.

[0088] Corresponding to the AI ​​interaction method in the above embodiment, Figure 2 This is a structural block diagram of an AI interaction device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 2 The AI ​​interaction device 20 is applied to a wearable device, and includes: a first computing module 21, a second computing module 22 and a third computing module 23.

[0089] The first calculation module 21 is used to construct a user portrait based on multi-source data. The multi-source data is data of a target user, and the target user is a user wearing a wearable device.

[0090] The second calculation module 22 is used to determine the target intention type based on the first voice in response to receiving the first voice of the target user. Determine the multimodal perception data of the target user based on the target intention type. Determine the associated information corresponding to the first voice based on the multimodal perception data. Determine the recommended information corresponding to the first voice based on the associated information and the user portrait.

[0091] The third calculation module 23 is used to generate a second voice based on the associated information and the recommended information. The second voice is the voice played by the AI ​​in the wearable device.

[0092] In an embodiment of the present disclosure, the multi-source data includes application behavior data, historical interaction data and user identity information. The first calculation module 21 is specifically used to construct a user portrait based on the application behavior data, historical interaction data and user identity information.

[0093] In an embodiment of the present disclosure, the first calculation module 21 is further configured to divide the application behavior data based on the application type to obtain application behavior data corresponding to a plurality of time periods, where each time period corresponds to an application type.

[0094] A first partial user portrait corresponding to the application behavior data is determined based on the application type and data volume corresponding to each time period.

[0095] A second local user portrait is determined based on the historical interaction data.

[0096] A third partial user portrait is determined based on the user identity information.

[0097] Based on the weight coefficient matrix corresponding to the application behavior data, the historical interaction data and the user identity information, the first local user portrait, the second local user portrait and the third local user portrait are fused to obtain a user portrait.

[0098] In one embodiment of the present disclosure, the target intent type includes information query and life service. The multimodal perception data includes user physiological data and environmental perception data.

[0099] The second calculation module 22 is further specifically configured to determine that the multimodal perception data of the target user is environmental perception data if the target intention type is information query type.

[0100] If the target intention type is life service type, the multimodal perception data of the target user is determined to be user physiological data and environmental perception data.

[0101] In one embodiment of the present disclosure, the second calculation module 22 is further configured to determine a target retrieval resource library based on the user portrait.

[0102] Based on the associated information, a reply text corresponding to the first speech is determined from the target retrieval resource library.

[0103] Determine recommended information corresponding to the first voice from a target retrieval resource library based on the reply text.

[0104] In an embodiment of the present disclosure, the second calculation module 22 is further configured to determine a user tag and a time series feature based on the user portrait.

[0105] A first search resource library is matched from search resources based on the user tag.

[0106] A target retrieval resource library is determined from the first retrieval resource library based on the temporal characteristics.

[0107] In an embodiment of the present disclosure, the third calculation module 23 is specifically configured to generate a first reply text based on the associated information, and generate a second reply text based on the recommended information.

[0108] The first reply text and the second reply text are concatenated to generate a target text, and a second speech is synthesized based on the target text.

[0109] See also Figure 3 , Figure 3 A schematic block diagram of a wearable device provided by an embodiment of the present disclosure. Figure 3 The wearable device 300 in the embodiment shown may include: one or more processors 301, one or more input devices 302, one or more output devices 303 and one or more memories 304. The processors 301, input devices 302, output devices 303 and memories 304 communicate with each other via a communication bus 305. The memory 304 is used to store computer programs, which include program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. The processor 301 is configured to call the program instructions to execute the functions of each module in the above-mentioned device embodiments, such as Figure 2 The functions of modules 21 to 23 are shown.

[0110] It should be understood that in the embodiment of the present disclosure, the processor 301 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0111] The input device 302 may include a touch panel, a fingerprint collection sensor (for collecting the user's fingerprint information and fingerprint direction information), a microphone, etc., and the output device 303 may include a display (LCD, etc.), a speaker, etc.

[0112] The memory 304 may include a read-only memory and a random access memory, and provide instructions and data to the processor 301. A portion of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store information about the device type.

[0113] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiments of the present disclosure can execute the implementation methods described in the first and second embodiments of the AI ​​interaction method provided in the embodiments of the present disclosure, and can also execute the implementation methods of the wearable device 300 described in the embodiments of the present disclosure, which will not be repeated here.

[0114] In another embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by the processor, all or part of the processes in the above-mentioned embodiment method are implemented, and the computer program can also be completed by instructing the relevant hardware through the computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0115] The computer-readable storage medium may be an internal storage unit of the wearable device of any of the foregoing embodiments, such as a hard disk or memory of the wearable device. The computer-readable storage medium may also be an external storage device of the wearable device, such as a plug-in hard disk equipped on the wearable device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the wearable device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the wearable device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0116] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0117] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the wearable device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0118] In the several embodiments provided in the present application, it should be understood that the disclosed wearable devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces or units, or it can be an electrical, mechanical or other form of connection.

[0119] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure.

[0120] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0121] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present disclosure, and these modifications or replacements should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. An AI interaction method, characterized in that: Applied to wearable devices, including: Constructing a user portrait based on multi-source data; the multi-source data is data of a target user, and the target user is a user wearing the wearable device; In response to receiving a first speech of the target user, determining a target intent type based on the first speech; determining multimodal perception data of the target user based on the target intent type; determining associated information corresponding to the first speech based on the multimodal perception data; and determining recommendation information corresponding to the first speech based on the associated information and the user portrait; A second voice is generated based on the associated information and the recommended information; the second voice is the voice played by the AI ​​in the wearable device.

2. The AI ​​interaction method according to claim 1, characterized in that: The multi-source data includes application behavior data, historical interaction data and user identity information; The method of constructing a user portrait based on multi-source data includes: The user portrait is constructed based on the application behavior data, the historical interaction data and the user identity information.

3. The AI ​​interaction method according to claim 2, characterized in that: The constructing the user portrait based on the application behavior data, the historical interaction data and the user identity information includes: Dividing the application behavior data based on application type to obtain application behavior data corresponding to multiple time periods, each time period corresponding to an application type; Determine a first partial user portrait corresponding to the application behavior data based on the application type and data volume corresponding to each time period; Determine a second partial user portrait based on the historical interaction data; Determine a third partial user portrait based on the user identity information; The first local user portrait, the second local user portrait and the third local user portrait are fused based on the application behavior data, the historical interaction data and a weight coefficient matrix corresponding to the user identity information to obtain the user portrait.

4. The AI ​​interaction method according to claim 1, characterized in that: The target intention types include information query and life service; the multimodal perception data includes user physiological data and environmental perception data; The determining the multimodal perception data of the target user based on the target intention type includes: If the target intention type is the information query type, determining that the multimodal perception data of the target user is the environmental perception data; If the target intention type is the life service type, it is determined that the multimodal perception data of the target user is the user physiological data and the environmental perception data.

5. The AI ​​interaction method according to claim 1, characterized in that: The determining, based on the association information and the user portrait, recommendation information corresponding to the first voice includes: Determine a target search resource library based on the user portrait; Determine a reply text corresponding to the first speech from the target retrieval resource library based on the association information; Determine recommendation information corresponding to the first voice from the target retrieval resource library based on the reply text.

6. The AI ​​interaction method according to claim 5, characterized in that: The determining of a target search resource library based on the user portrait includes: Determine a user tag and a time series feature based on the user portrait; Matching a first search resource library from search resources based on the user tag; The target retrieval resource library is determined from the first retrieval resource library based on the temporal characteristics.

7. The AI ​​interaction method according to claim 1, characterized in that: The generating a second voice based on the associated information and the recommended information includes: Generate a first reply text based on the associated information, and generate a second reply text based on the recommended information; The first reply text and the second reply text are concatenated to generate a target text, and the second speech is synthesized based on the target text.

8. An AI interactive device, characterized in that: Applied to wearable devices, including: A first computing module, configured to construct a user portrait based on multi-source data; the multi-source data is data of a target user, and the target user is a user wearing the wearable device; a second calculation module, configured to determine, in response to receiving a first voice of the target user, associated information corresponding to the first voice, and determine recommended information corresponding to the first voice based on the associated information and the user portrait; The third computing module is used to generate a second voice based on the associated information and the recommended information; the second voice is the voice played by the AI ​​in the wearable device.

9. A wearable device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images