Digital human application method and system under wearable device and electronic device

Abstract

The invention relates to a digital human application method and system under wearable equipment and electronic equipment. The method comprises the following steps: a digital human server obtains real-time state data of a user sent by a digital human terminal application through a wearable device, and generates a corresponding digital human artificial intelligence (AI) behavior instruction and digital human performance data according to the real-time state data of the user; and sending the digital human AI behavior instruction and the digital human performance data to the digital human terminal application, wherein the digital human AI behavior instruction is used for indicating the digital human terminal application to execute the digital human performance data. According to the invention, the digital human visual intelligent terminal application under the wearable device is realized, and the digital human terminal effects of real-time multi-mode interaction, scene monitoring, intelligent early warning, intelligent voice and the like of the combination of the digital human and the wearable device are achieved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a method, system and electronic device for digital human applications in wearable devices. Background Technology

[0002] Digital humans refer to virtual characters that exist in digital space in digital form and have human-like or real-life appearances, behaviors, and characteristics. They are also known as virtual avatars, digital virtual humans, virtual digital humans, etc.

[0003] With the gradual maturation of artificial intelligence and interactive technologies, the digital human industry is developing at a high penetration rate, and more and more market players are beginning to lay out their industry strategies. Among related technologies, digital humans have a wide range of applications, such as multi-person spatial interaction, digital human live streaming, all-in-one machines, VR (Virtual Reality) devices, and in-vehicle systems, among other scenarios. Summary of the Invention

[0004] This disclosure provides a method, system, and electronic device for digital human applications in wearable devices.

[0005] In a first aspect, embodiments of this disclosure provide a method for digital human applications in wearable devices, the method being applied to a digital human server, the method comprising: Acquire real-time status data of users sent by digital human terminal applications through wearable devices; Based on the user's real-time status data, corresponding digital human AI behavior instructions and digital human performance data are generated; The digital human AI behavior instructions and digital human performance data are sent to the digital human terminal application, and the digital human AI behavior instructions are used to instruct the digital human terminal application to execute the digital human performance data.

[0006] Secondly, embodiments of this disclosure provide a method for digital human applications in a wearable device, the method being applied to the wearable device, the wearable device having a digital human terminal application installed, the method comprising: Obtain real-time status data of users; The user's real-time status data is sent to the digital human server through the wearable device. The user's real-time status data is used by the digital human server to generate corresponding digital human artificial intelligence (AI) behavior instructions and digital human performance data. Obtain the AI ​​behavior instructions and performance data of the digital human sent by the digital human server; Based on the digital human AI behavior instructions, the digital human performance data is executed.

[0007] Thirdly, embodiments of this disclosure provide a wearable device for digital human applications, the device being configured on a digital human server, the device comprising: The acquisition module is used to acquire real-time status data of the user sent by the digital human terminal application through the wearable device; The data processing module is used to generate corresponding AI behavior instructions and digital human performance data based on the user's real-time status data. The sending module is used to send the digital human AI behavior instructions and digital human performance data to the digital human terminal application, wherein the digital human AI behavior instructions are used to instruct the digital human terminal application to execute the digital human performance data.

[0008] Fourthly, embodiments of this disclosure provide a digital human application device for a wearable device, the device being configured on the wearable device, the wearable device having a digital human terminal application installed, the device comprising: The acquisition module is used to acquire real-time status data of users; The sending module is used to send the user's real-time status data to the digital human server through the wearable device. The user's real-time status data is used by the digital human server to generate corresponding digital human artificial intelligence (AI) behavior instructions and digital human performance data. The receiving module is used to receive the AI ​​behavior instructions and performance data of the digital human sent by the digital human server; The execution module is used to execute the digital human performance data based on the digital human AI behavior instructions.

[0009] Fifthly, embodiments of this disclosure provide a digital human application system for wearable devices, including: The digital human server is configured to perform the method described in the first aspect above; A wearable device, the wearable device having a digital human terminal application installed, the wearable device being configured to perform the method described in the second aspect above.

[0010] Sixthly, embodiments of this disclosure provide an electronic device, including: One or more processors; The processor is used to invoke instructions to cause the electronic device to perform the method described in the first or second aspect above.

[0011] In a seventh aspect, embodiments of this disclosure provide a storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first or second aspect.

[0012] Eighthly, embodiments of this disclosure provide a program product including at least one of a program and instructions, wherein when the program or instructions are executed by an electronic device, they implement the steps of the methods described in the first or second aspect.

[0013] According to the technical solution disclosed herein, a new digital human visual and voice interaction method is proposed based on the digital human terminal application on wearable devices and combined with the device capabilities. This provides an end-to-end basic solution for the implementation of digital humans on wearable devices, which can improve the overall user experience of wearable devices and make up for the shortcomings of digital human multimodal interaction on wearable devices. By receiving and processing device information, the wearable device can realize the automatic adaptation and processing capabilities of digital human intelligent voice, intelligent rendering, intelligent interaction and device commands in different real environments, and push streaming data to the wearable device in real time. On the general server data flow, a three-stream collaborative framework design of device command stream, digital human AI behavior stream and digital human voice and visual stream is added, which can improve the application collaborative control performance, resource utilization and accuracy.

[0014] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0016] Figure 1 This is a flowchart illustrating a method for applying digital human technology on a wearable device according to an exemplary embodiment.

[0017] Figure 2 This is a schematic diagram of the architecture of a digital human application system under a wearable device, according to an exemplary embodiment.

[0018] Figure 3 This is a flowchart illustrating a method for applying digital human technology on a wearable device according to an exemplary embodiment.

[0019] Figure 4 This is a flowchart illustrating a method for applying digital human technology on a wearable device according to an exemplary embodiment.

[0020] Figure 5 This is a flowchart illustrating a method for applying digital human technology on a wearable device according to an exemplary embodiment.

[0021] Figure 6 This is a flowchart illustrating a method for applying digital human technology on a wearable device according to an exemplary embodiment.

[0022] Figure 7This is a block diagram illustrating a digital human application device under a wearable device according to an exemplary embodiment.

[0023] Figure 8 This is a block diagram illustrating a digital human application device under a wearable device according to an exemplary embodiment.

[0024] Figure 9 This is a block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0025] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0026] This disclosure is not exhaustive, but merely illustrative of some embodiments, and is not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment can be arbitrarily interchanged. Furthermore, the optional implementation methods in a particular embodiment can be arbitrarily combined; moreover, the embodiments can be arbitrarily combined, for example, some or all steps of different embodiments can be arbitrarily combined, and a particular embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0027] In each of the disclosed embodiments, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of the embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0028] It should be noted that the acquisition, transmission, storage, use, and processing of data in this disclosed technical solution comply with the relevant provisions of national laws and regulations and do not violate public order and good morals.

[0029] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0030] It is worth noting that in the embodiments disclosed herein, certain software, components, models, and other existing solutions in the industry may be mentioned. These should be considered as exemplary, and their purpose is only to illustrate the feasibility of implementing the technical solutions disclosed herein. However, this does not mean that the applicant has used or necessarily used such solutions.

[0031] The following description, with reference to the accompanying drawings, outlines a method, system, and electronic device for digital human applications in wearable devices, according to embodiments of the present disclosure.

[0032] Figure 1 This is a flowchart illustrating a method for digital human application in a wearable device according to an exemplary embodiment. It should be noted that the executing entity of the method for digital human application in a wearable device in this disclosure embodiment can be a digital human application device for a wearable device. This device can be implemented by software and / or hardware, and can be configured in an electronic device, which may include a digital human server. For example... Figure 1 As shown, the digital human application method under this wearable device may include, but is not limited to, the following steps.

[0033] In step 101, the real-time status data of the user sent by the digital human terminal application through the wearable device is obtained.

[0034] In some embodiments, the digital human terminal application (also called the digital human front-end application) can be an application installed on a wearable device, that is, a wearable device-based digital human smart terminal application, which can be used for digital human image presentation and wearable device data collection and transmission to the digital human server. The digital human server can receive device data sent by the digital human terminal application, process the data, generate device commands and digital human voice and image representations according to different real-world environments, and send them to the device and the digital human terminal application.

[0035] In some embodiments, the wearable device described above can be a wearable device worn on the user's body, such as a fitness tracker, but not limited to this, such as a smartwatch.

[0036] In some embodiments, wearable devices may include, but are not limited to, various sensors, such as motion and environmental sensors (e.g., accelerometers, gyroscopes, barometers, ambient light sensors, microphones, geomagnetic sensors, cameras, etc.), physiological sensors (e.g., optical sensors, electrical sensors, temperature sensors, etc.), and position sensors (e.g., GPS (Global Positioning System)).

[0037] In the embodiments of this disclosure, a digital human terminal application can invoke sensors on a wearable device to collect data, analyze the collected sensor data to obtain real-time status data of the user wearing the wearable device, and send the user's real-time status data to a digital human server so that the digital human server can obtain the user's real-time status data sent by the digital human terminal application through the wearable device. Exemplarily, the real-time status data may include physiological status data, but is not limited to this; for example, it may also include environmental data (such as image data and / or audio data of the user's surrounding environment).

[0038] In some embodiments, the triggering method for the above-mentioned digital human terminal application to call the sensors on the wearable device for data collection can be user-initiated request, or it can be automatically triggered based on context (the core logic can be environmental and state perception, intelligent prediction, such as automatic recording when starting to run, or guiding breathing when under great pressure, etc.), or it can be triggered based on prediction and health goals (such as reminding to complete exercise goals, or strengthening monitoring based on risks, etc.), or it can be triggered based on external events and coordination (such as synchronizing with a weight scale, or receiving remote instructions from a doctor, etc.), or it can be triggered based on emergency and abnormal events (such as the accelerometer and gyroscope detecting a severe impact and body posture consistent with a fall, or the electrocardiogram detecting suspected atrial fibrillation, or the blood oxygen detection being severely low, etc.).

[0039] In step 102, based on the user's real-time status data, corresponding digital human AI (Artificial Intelligence) behavioral instructions and digital human performance data are generated.

[0040] In some embodiments, real-time user status data can be analyzed to determine the user's current status category, and preset digital human AI behavior instructions corresponding to that status category can be generated based on the determined status category. For example, taking real-time physiological status data as an example, the user's real-time physiological status data is analyzed to determine the user's current status category, and preset digital human AI behavior instructions corresponding to that status category are generated based on the determined status category. The physiological status data may include, but is not limited to, data on posture, falls, heart rate, blood oxygen saturation, and acceleration due to motion. The status category may include, but is not limited to, normal state, in motion state, dangerous state, fatigued state, and comatose state.

[0041] Optionally, in some embodiments, the user's current status category can be determined based on user identity data and real-time status data. For example, the user's current status category can be determined based on user identity data such as gender, height, and weight, combined with the user's real-time status data. Considering user identity data when determining the user's current status category can further improve the accuracy of the determination.

[0042] In the embodiments disclosed herein, the digital human performance data may include, but is not limited to, the digital human's voice and visual performance. This digital human performance data may be related to the content of the digital human AI behavior commands. For example, taking a digital human AI behavior command as a phone call command, the digital human performance data may include the digital human's audio-visual performance. For instance, the digital human's voice content might be "I am about to make a phone call to XXX," and the visual performance could be an image of the digital human wearing headphones (such as Bluetooth headphones or a headset).

[0043] In some embodiments, when the user's state is determined to be dangerous or unconscious, a digital human AI distress signal command corresponding to that state is generated. In some embodiments, after the digital human AI distress signal command is sent to the wearable device, the digital human terminal application on the wearable device can send the collected image data and / or audio data as real-time environmental data to the digital human server in response to the digital human AI distress signal command via image data collected by the terminal device's camera and / or audio data collected by the terminal device's microphone. The terminal device may include at least the wearable device, such as the user's mobile phone. When the digital human terminal application receives the digital human AI distress signal command from the digital human server, it can call upon the camera and / or microphone on its linked terminal device (including but not limited to the wearable device, such as a mobile phone) to collect environmental data. When the digital human server receives real-time environmental data from the user's terminal device, it can analyze the real-time environmental data, determine the current environmental state, and generate a digital human behavior corresponding to the determined current environmental state. This digital human behavior may include self-rescue AI behavior or distress call AI behavior.

[0044] In some embodiments, different digital human performance data can be generated based on the self-rescue AI behavior or the distress call AI behavior. In some embodiments, optional implementations of generating corresponding digital human performance data based on the self-rescue AI behavior may include: when the user is unconscious, generating a distress call voice by selecting the voice of a relative of the user, automatically dialing a distress call based on the world environment characteristics surrounding the user, and taking over the dialogue through a digital human after a successful call to complete the self-rescue description; or, when the user is not unconscious, generating different digital human self-rescue behavior performance data based on different world environment characteristics surrounding the user.

[0045] In some embodiments, an optional implementation of generating corresponding digital human performance data based on the AI's distress call behavior may include: when the user is unconscious, generating a user's voice by selecting the user's own timbre, and generating different digital human distress call behavior performance data using the user's voice in a corresponding distress call manner based on the world environment characteristics and the types of people in the user's surrounding environment; or, when the user is not unconscious, generating auxiliary distress call performance data of the digital human for use in assisting distress calls through the digital human.

[0046] In step 103, the digital human AI behavior instructions and digital human performance data are sent to the digital human terminal application. The digital human AI behavior instructions are used to instruct the digital human terminal application to execute the digital human performance data.

[0047] In the embodiments of this disclosure, the generated digital human AI behavior instructions and digital human performance data can be sent to a digital human terminal application, so that the digital human terminal application can execute the digital human performance data based on the digital human AI behavior instructions. In the embodiments of this disclosure, the digital human performance data may include, but is not limited to, digital human voice and visual performance. For example, the digital human terminal application can output digital human visuals and voice through a wearable device.

[0048] In some embodiments, the aforementioned digital human AI behavior instructions may include, but are not limited to, one or more of the following: making a phone call, mood switching, encouragement, updating physical status, exercise growth, intelligent communication, intelligent guidance, and calling for help. The aforementioned digital human performance data may be related to the content of the digital human AI behavior instructions. For example, taking the digital human AI behavior instruction of making a phone call as an example, the digital human performance data may include the digital human's audio-visual performance. For instance, the digital human's voice content might be "I am about to make a phone call to XXX," and the visual presentation could be the digital human wearing headphones (such as Bluetooth headphones or a headset).

[0049] In the above embodiments, relying on the digital human terminal application on wearable devices, and combining the device capabilities, a new digital human visual and voice interaction method is proposed. This provides an end-to-end basic solution for the implementation of digital humans on wearable devices, which can improve the overall user experience of wearable devices and make up for the shortcomings of digital human multimodal interaction on wearable devices. By receiving and processing device information, the wearable device can realize the automatic adaptation and processing capabilities of digital human intelligent voice, intelligent rendering, intelligent interaction and device commands in different real environments, and push streaming data to the wearable device in real time. On the general server data flow, a three-stream collaborative framework design of device command stream, digital human AI behavior stream and digital human voice and visual stream is added, which can improve the application collaborative control performance, resource utilization and accuracy.

[0050] Figure 2 This is a schematic diagram illustrating the architecture of a digital human application system under a wearable device, according to an exemplary embodiment. Figure 2 As shown, the wearable device-based digital human application system may include a wearable device 201 and a digital human server 202. The digital human server 202 can be configured to execute the steps in the digital human server-side method embodiments disclosed herein. The wearable device 201 is equipped with a digital human terminal application 203, and the wearable device 201 can be configured to execute the steps in the wearable device-side method embodiments disclosed herein. The digital human terminal application 203 can be used for digital human image presentation and wearable device data collection and transmission to the large-screen digital human server. The digital human server 202 can be used to: receive and process data from the digital human front-end application device, generate device commands and digital human voice and image representations according to different real-world environments, and send them to the device and the digital human front-end application.

[0051] against Figure 2 Under the framework shown, an example of the application design process for using physiological state data in an emergency environment can be given here. Figure 3 This is a flowchart illustrating a method for digital human application in a wearable device according to an exemplary embodiment. It should be noted that the executing entity of the method for digital human application in a wearable device in this embodiment can be a digital human application device for a wearable device. This device can be implemented by software and / or hardware, and can be configured in an electronic device, which may include a digital human server. The method flow will be described below using real-time status data as physiological status data as an example. Figure 3 As shown, the digital human application method under this wearable device may include, but is not limited to, the following steps.

[0052] In step 301, the user's physiological status data sent by the digital human terminal application through the wearable device is obtained.

[0053] In embodiments of this disclosure, physiological state data is used as an example, including gender, height, weight, posture, fall, heart rate, blood oxygen saturation, and acceleration. The digital human terminal application can acquire user physiological state data collected by wearable devices and send this data to a digital human server via the wearable device, enabling the digital human server to obtain the user's physiological state data from the wearable device.

[0054] In step 302, the user's physiological state data is analyzed to determine the user's current state category.

[0055] In some embodiments, the user state category may include, but is not limited to, normal state, active state, dangerous state, fatigued state, and comatose state. The specific technical implementation process for using physiological state data collected by wearable devices to logically analyze and determine the user state category is as follows: (1) Normal state: The heart rate and blood oxygen levels are dynamically adjusted according to the user's gender and height, for example (heart rate 60-100 bpm, blood oxygen 95%-100%, acceleration 0.1-0.3g). For example, when the heart rate and blood oxygen levels in the user's physiological state data collected by the wearable device are within the normal range of heart rate and blood oxygen levels that match the user's gender and height, the user's state category can be determined to be normal.

[0056] (2) In motion state: The physiological state data is within the normal range and the motion trajectory data has a certain periodic amplitude and periodic fluctuation. For example, when the physiological state data of the user collected by the wearable device is within the normal range and the motion trajectory data has a certain periodic amplitude and periodic fluctuation, the user's state category can be determined as an in motion state.

[0057] (3) Fatigue state: Based on the decreasing trend of heart rate recovery rate, blood oxygen fluctuation, and vertical displacement height of movement (such as smaller arm or movement swing, weak swing, etc.). For example, when the heart rate recovery rate, blood oxygen fluctuation, and vertical displacement height of movement in the user's physiological state data collected by the wearable device show a decreasing trend, the user's state category can be determined as fatigue state.

[0058] (4) Dangerous state: Heart rate continuously >180 bpm or <40 bpm for more than 10 seconds, blood oxygen <85% for 30 seconds, movement data conforming to "fall mode" (such as judging "fall mode" based on sudden deceleration + horizontal displacement + equipment drop height), and equipment movement in a free-fall state. Real-time detection is performed through the above points, and if one or more of them are met, it can be judged as a dangerous state.

[0059] (5) Coma: Continuous low displacement and approaching zero, stable low heart rate, blood oxygen <85% and fluctuation <1%. If these conditions are met, it can be judged as a coma.

[0060] In step 303, based on the determined user status category, the corresponding digital human AI behavior is triggered, and the digital human audio-visual performance is executed based on the digital human AI behavior.

[0061] In some embodiments, the digital human AI behavior may include, but is not limited to: making phone calls, mood switching, encouragement, updating physical status, physical progress, intelligent communication, intelligent guidance, and calling for help. This disclosure specifically designs a "call for help" mode, which triggers the digital human AI's call for help behavior when the user's status category is dangerous or unconscious. The execution process of the call for help mode is described below: (1) When the user is in a dangerous or unconscious state, the digital human AI will trigger a distress signal. The digital human server will send a device instruction stream and call the device functions of the wearable device (including the user's mobile phone camera, wearable device camera, microphone, etc.) through the digital human terminal application to obtain image and audio data in real time. The digital human terminal application will send the obtained image and audio data to the digital human server.

[0062] (2) The digital human server receives data in real time and determines whether to perform self-rescue or call for help based on image and audio data. For example... Figure 4 As shown, the digital human server can extract image features from real-time image data transmitted from wearable devices and speech features from real-time audio data transmitted from wearable devices. For example, it can collect image data and perform feature detection, including crowd (density), environmental structural features (cubicles, elevators, walls), and natural elements (trees, windows, etc.). It can also collect audio data and perform voiceprint analysis to identify human voices, dialogues, and environmental noise (water sounds, elevator sounds, etc.). Using deep learning and large model technology, the environmental state decision engine determines the current environmental state, classifying it as either a closed or open environment. If the environment is closed, it executes a digital human AI self-rescue action (or activates a digital human self-rescue mode); if the environment is open and a crowd is detected, it triggers a digital human distress call AI action (or activates a digital human distress call mode); if the environment is open and no one is detected, it executes a digital human self-rescue AI action (or activates a digital human self-rescue mode).

[0063] (3) Based on the self-rescue AI behavior or the call for help AI behavior, perform different digital human performances, such as including but not limited to digital human image rendering and voice generation, thereby sending data (such as digital human image rendering data, generated voice data, etc.) to wearable devices, and wearable devices output images and voice (such as high decibels).

[0064] For example, regarding self-rescue AI behavior, different actions can be performed on the digital human based on the user's coma state and the characteristics of the world environment. For instance... Figure 5As shown, in the digital human self-rescue mode, when a user is unconscious, a distress call can be generated by selecting the voice of a family member. Based on environmental characteristics, it automatically dials emergency numbers: at home, it automatically dials the property management number; in an elevator, it automatically dials the elevator maintenance number; outdoors, it automatically dials a family member's number. These phone numbers can be pre-configured in the wearable device or digital human terminal application. After a successful call, the digital human takes over the dialogue and completes the self-rescue description. The digital human server performs real-time rendering of the digital human based on these multimodal self-rescue behaviors to obtain a voice stream and a digital human rendering stream. This voice stream and digital human rendering stream are then sent to the digital human terminal application, which can then output the voice and video streams via the wearable device.

[0065] In digital human self-rescue mode, when the user is not unconscious, different digital human self-rescue behaviors can be performed depending on the current environment, including gesture escape guidance, emotional stabilization dialogue, and humorous or friendly digital human visual rendering. The digital human visuals and voice are generated in real-time through a digital human server cloud generation service and streamed to ensure real-time reception on the device. For example, if the user's surrounding environment is determined to be in a fire based on global environmental characteristics, and an escape space is identified based on real-time global environmental characteristics, the user can be guided to escape using digital human gesture escape guidance and voice. Alternatively, if the user is determined to be in an elevator or water based on global environmental characteristics and physiological characteristics such as heart rate, indicating an emergency situation, the user can be stabilized emotionally through digital human-assisted dialogue.

[0066] For example, regarding AI-powered distress calls, when a user is unconscious, the system can generate a voice using the user's own voice tone, and then use different methods to call for help depending on the current world environment and the type of people involved. The specific process is as follows: (1) Select local dialect or Mandarin based on the age of the target audience; (2) Adjust the language, digital human image, and shouting content in real time according to the crowd's response to the call for help; (3) When calling for help, the data includes two types of data: digital human face and voice driven image generated in the cloud, and the voice of the call; (4) When the user's location information is detected to have moved, someone makes contact or a crowd responds, the digital human will actively start a dialogue mode; (5) The content of the digital human dialogue includes important information such as user information, current physiological status data, recommended rescue methods, and contact information of relatives.

[0067] For example, regarding AI-powered emergency calls, when the user is not unconscious, a digital human can assist in calling for help through methods such as guidance, voice prompts, and intelligent communication, or allow the user to actively request an automatic emergency call.

[0068] In the above embodiments, by utilizing wearable device data and combining camera and voice hardware capabilities, innovative algorithms and application processes such as self-rescue and emergency call functions are added to enhance emergency response capabilities. This enables wearable devices to automatically adapt and process digital human intelligent voice, intelligent rendering, intelligent interaction, and device commands in different real-world environments, compensating for the shortcomings of multimodal digital human interaction on wearable devices. Furthermore, it can seize crucial time in special circumstances to protect users' lives and health. In terms of general server data flow, a new three-stream collaborative framework design—device command stream, digital human AI behavior stream, and digital human voice / video stream—can improve application collaborative control performance, resource utilization, and accuracy.

[0069] Figure 6 This is a flowchart illustrating a method for digital human application on a wearable device according to an exemplary embodiment. It should be noted that the executing entity of the method for digital human application on a wearable device in this disclosure embodiment can be a digital human application device on a wearable device. This device can be implemented by software and / or hardware, and can be configured in an electronic device, which may include a wearable device, and the wearable device has a digital human terminal application installed. For example... Figure 6 As shown, the digital human application method under this wearable device may include, but is not limited to, the following steps.

[0070] In step 601, the user's real-time status data is obtained.

[0071] In some embodiments, the digital human terminal application (also called the digital human front-end application) can be an application installed on a wearable device, that is, a wearable device-based digital human smart terminal application, which can be used for digital human image presentation and wearable device data collection and transmission to the digital human server. The digital human server can receive device data sent by the digital human terminal application, process the data, generate device commands and digital human voice and image representations according to different real-world environments, and send them to the device and the digital human terminal application.

[0072] In some embodiments, the wearable device described above can be a wearable device worn on the user's body, such as a fitness tracker, but not limited to this, such as a smartwatch.

[0073] In some embodiments, wearable devices may include, but are not limited to, various sensors, such as motion and environmental sensors (e.g., accelerometers, gyroscopes, barometers, ambient light sensors, microphones, geomagnetic sensors, cameras, etc.), physiological sensors (e.g., optical sensors, electrical sensors, temperature sensors, etc.), and position sensors (e.g., GPS (Global Positioning System)).

[0074] In embodiments of this disclosure, a digital human terminal application can invoke sensors on a wearable device to collect data and analyze the collected sensor data to obtain real-time status data of the user wearing the wearable device. Exemplarily, this real-time status data may include physiological status data, but is not limited to this; for example, it may also include environmental data (such as image data and / or audio data of the user's surrounding environment).

[0075] In some embodiments, the triggering method for the above-mentioned digital human terminal application to call the sensors on the wearable device for data collection can be user-initiated request, or it can be automatically triggered based on context (the core logic can be environmental and state perception, intelligent prediction, such as automatic recording when starting to run, or guiding breathing when under great pressure, etc.), or it can be triggered based on prediction and health goals (such as reminding to complete exercise goals, or strengthening monitoring based on risks, etc.), or it can be triggered based on external events and coordination (such as synchronizing with a weight scale, or receiving remote instructions from a doctor, etc.), or it can be triggered based on emergency and abnormal events (such as the accelerometer and gyroscope detecting a severe impact and body posture consistent with a fall, or the electrocardiogram detecting suspected atrial fibrillation, or the blood oxygen detection being severely low, etc.).

[0076] In step 602, the user's real-time status data is sent to the digital human server via the wearable device. The user's real-time status data is used by the digital human server to generate corresponding digital human artificial intelligence (AI) behavior instructions and digital human performance data.

[0077] In the embodiments of this disclosure, the digital human terminal application can send the user's real-time status data to the digital human server, enabling the digital human server to obtain the user's real-time status data sent by the digital human terminal application through the wearable device. The digital human server can generate corresponding digital human AI behavior instructions and digital human performance data based on the user's real-time status data; optional implementation methods are described above. Figure 1 The optional implementation methods of step 102 and related steps are not described here.

[0078] In step 603, the system receives digital human AI behavior instructions and digital human performance data sent by the digital human server.

[0079] In step 604, the digital human performance data is executed based on the digital human AI behavior instructions.

[0080] In the embodiments disclosed herein, the digital human performance data may include, but is not limited to, digital human voice and visual performance. For example, a digital human terminal application may output digital human visuals and voice through a wearable device.

[0081] In some embodiments, the aforementioned digital human AI behavior instructions may include, but are not limited to, one or more of the following: making a phone call, mood switching, encouragement, updating physical status, exercise growth, intelligent communication, intelligent guidance, and calling for help. The aforementioned digital human performance data may be related to the content of the digital human AI behavior instructions. For example, taking the digital human AI behavior instruction of making a phone call as an example, the digital human performance data may include the digital human's audio-visual performance. For instance, the digital human's voice content might be "I am about to make a phone call to XXX," and the visual presentation could be the digital human wearing headphones (such as Bluetooth headphones or a headset).

[0082] In the above embodiments, relying on the digital human terminal application on wearable devices, and combining the device capabilities, a new digital human visual and voice interaction method is proposed. This provides an end-to-end basic solution for the implementation of digital humans on wearable devices, which can improve the overall user experience of wearable devices and make up for the shortcomings of digital human multimodal interaction on wearable devices. By receiving and processing device information, the wearable device can realize the automatic adaptation and processing capabilities of digital human intelligent voice, intelligent rendering, intelligent interaction and device commands in different real environments, and push streaming data to the wearable device in real time. On the general server data flow, a three-stream collaborative framework design of device command stream, digital human AI behavior stream and digital human voice and visual stream is added, which can improve the application collaborative control performance, resource utilization and accuracy.

[0083] Figure 7 This is a block diagram illustrating a digital human application device under a wearable device according to an exemplary embodiment. The digital human application device under the wearable device can be configured on a digital human server, such as... Figure 7 As shown, the wearable device for digital human applications may include: an acquisition module 701, a data processing module 702, and a transmission module 703.

[0084] The acquisition module 701 is used to acquire real-time status data of the user sent by the digital human terminal application through the wearable device.

[0085] The data processing module 702 is used to generate corresponding AI behavior instructions and digital human performance data based on the user's real-time status data.

[0086] The sending module 703 is used to send digital human AI behavior instructions and digital human performance data to the digital human terminal application. The digital human AI behavior instructions are used to instruct the digital human terminal application to execute the digital human performance data.

[0087] In some embodiments, the data processing module 702 is used to: analyze the user's real-time status data to determine the user's current status category; and generate preset digital human AI behavior instructions corresponding to the determined status category.

[0088] In some embodiments, the data processing module 702 is used to: when it is determined that the user's state category is dangerous or unconscious, generate a digital human AI distress signal instruction corresponding to the state category.

[0089] In some embodiments, the data processing module 702 is further configured to: receive real-time environmental data from a user's terminal device, wherein the real-time environmental data is image data captured by the terminal device's camera and / or audio data captured by the terminal device's microphone in response to a digital human AI distress call command, and the terminal device includes at least a wearable device; analyze the real-time environmental data to determine the current environmental state; and generate digital human behavior corresponding to the determined current environmental state, wherein the digital human behavior includes self-rescue AI behavior or distress call AI behavior. The data processing module 702 is also configured to: generate different digital human performance data based on the self-rescue AI behavior or distress call AI behavior.

[0090] In some embodiments, the data processing module 702 is further configured to: generate a distress call voice by selecting the voice of a relative of the user when the user is in a coma, automatically dial a distress call based on the world environment characteristics of the user's surrounding environment, and take over the dialogue through a digital human after the call is successfully completed to complete the self-rescue description; or, when the user is not in a coma, generate different performance data of the digital human's self-rescue behavior based on the different world environment characteristics of the user's surrounding environment.

[0091] In some embodiments, the data processing module 702 is further configured to: generate a user's voice by selecting the user's own timbre when the user is in a coma, and generate different digital human distress call performance data using the user's voice in a corresponding distress call manner based on the world environment characteristics and the type of people in the user's surrounding environment; or, generate digital human auxiliary distress call performance data when the user is not in a coma, for use in assisting distress calls through digital human assistance.

[0092] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0093] Figure 8 This is a block diagram illustrating a digital human application device under a wearable device according to an exemplary embodiment. The digital human application device under the wearable device can be configured in a wearable device, such as... Figure 8 As shown, the wearable device for digital human applications may include: an acquisition module 801, a sending module 802, a receiving module 803, and an execution module 804.

[0094] The acquisition module 801 is used to acquire the user's real-time status data.

[0095] The sending module 802 is used to send the user's real-time status data to the digital human server through the wearable device. The user's real-time status data is used by the digital human server to generate corresponding digital human artificial intelligence (AI) behavior instructions and digital human performance data.

[0096] The receiving module 803 is used to receive digital human AI behavior instructions and digital human performance data sent by the digital human server.

[0097] Execution module 804 is used to execute digital human performance data based on digital human AI behavior instructions.

[0098] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0099] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0100] like Figure 9 The diagram shown is a block diagram of an electronic device according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as wearable devices (e.g., smartwatches, fitness trackers, etc.) and servers (e.g., digital human servers). The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0101] like Figure 9As shown, the electronic device includes one or more processors 901, a memory 902, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 9 Take the 901 processor as an example.

[0102] The memory 902 is the non-transitory computer-readable storage medium provided in this disclosure. The memory stores instructions executable by at least one processor to cause the at least one processor to perform the digital human application method for wearable devices provided in this disclosure. The non-transitory computer-readable storage medium of this disclosure stores computer instructions for causing a computer to perform the digital human application method for wearable devices provided in this disclosure.

[0103] Memory 902, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the digital human application method under the wearable device in the embodiments of this disclosure (e.g., attached). Figure 7 The acquisition module 701, data processing module 702, and sending module 703 shown are attached. Figure 8 The acquisition module 801, sending module 802, receiving module 803, and execution module 804 are shown. The processor 901 executes various functional applications and data processing of the server by running non-transient software programs, instructions, and modules stored in the memory 902, thereby realizing the digital human application method under the wearable device in the above method embodiment.

[0104] The memory 902 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 902 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 902 may optionally include memory remotely located relative to the processor 901, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0105] The electronic device may also include an input device 903 and an output device 904. The processor 901, memory 902, input device 903, and output device 904 can be connected via a bus or other means. Figure 9 Taking the example of a connection between China and Israel via a bus.

[0106] Input device 903 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as touch screens, keypads, mice, trackpads, touchpads, joysticks, one or more mouse buttons, trackballs, joysticks, etc. Output device 904 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The display device may include, but is not limited to, liquid crystal displays (LCDs), light-emitting diode (LED) displays, and plasma displays. In some embodiments, the display device may be a touch screen.

[0107] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0108] These computational programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0109] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0110] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0111] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0112] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0113] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for applying digital humans in wearable devices, characterized in that, The method is applied to a digital human server, and the method includes: Acquire real-time status data of users sent by digital human terminal applications through wearable devices; Based on the user's real-time status data, corresponding digital human AI behavior instructions and digital human performance data are generated; The digital human AI behavior instructions and digital human performance data are sent to the digital human terminal application, and the digital human AI behavior instructions are used to instruct the digital human terminal application to execute the digital human performance data.

2. The method according to claim 1, characterized in that, Based on the user's real-time status data, corresponding digital human artificial intelligence (AI) behavior instructions are generated, including: Analyze the user's real-time status data to determine the user's current status category; Based on the determined state category, a preset digital human AI behavior instruction corresponding to the state category is generated.

3. The method according to claim 2, characterized in that, The step of generating preset digital human AI behavior instructions corresponding to the determined state category includes: When the user's status is determined to be dangerous or unconscious, a digital human AI distress signal command corresponding to the status category is generated.

4. The method according to claim 3, characterized in that, The method further includes: The system receives real-time environmental data from the user's terminal device. The real-time environmental data is generated by the terminal device in response to the digital human AI's distress call command, and includes image data captured by the terminal device's camera and / or audio data captured by the terminal device's microphone. The terminal device includes at least the wearable device. Analyze the real-time environmental data to determine the current environmental status; Based on the determined current environmental state, generate digital human behavior corresponding to the current environmental state, including self-rescue AI behavior or distress call AI behavior; The generation of the digital human performance data includes: Based on the self-rescue AI behavior or the distress call AI behavior, different digital human performance data are generated.

5. The method according to claim 4, characterized in that, Based on the self-rescue AI behavior, corresponding digital human performance data is generated, including: If the user is unconscious, a distress call can be generated by selecting the voice of a relative of the user, and an emergency call can be automatically dialed based on the characteristics of the user's surrounding environment. After the call is successfully connected, a digital human will take over the dialogue to complete the self-rescue description; or, When the user is not unconscious, different performance data of the digital human's self-rescue behavior are generated based on the different characteristics of the world environment surrounding the user.

6. The method according to claim 4, characterized in that, Based on the AI's distress call behavior, corresponding digital human performance data is generated, including: When the user is unconscious, a user voice is generated by selecting the user's own voice timbre, and based on the characteristics of the surrounding environment and the types of people, different digital human distress call behaviors are generated using the user voice in a corresponding distress call manner; or, While the user is not unconscious, data on the digital human's assistive distress call performance is generated for use in assisting with distress calls through the digital human.

7. A method for applying digital humans in a wearable device, characterized in that, The method is applied to the wearable device, which has a digital human terminal application installed, and the method includes: Obtain real-time status data of users; The user's real-time status data is sent to the digital human server through the wearable device. The user's real-time status data is used by the digital human server to generate corresponding digital human artificial intelligence (AI) behavior instructions and digital human performance data. Receive the AI ​​behavior instructions and performance data of the digital human sent by the digital human server; Based on the digital human AI behavior instructions, the digital human performance data is executed.

8. A digital human application system for wearable devices, characterized in that, include: A digital human server is configured to perform the method according to any one of claims 1-6; A wearable device, wherein the wearable device is equipped with a digital human terminal application, and the wearable device is configured to perform the method of claim 7.

9. An electronic device, characterized in that, include: One or more processors; The processor is used to invoke instructions to cause the electronic device to perform the method of any one of claims 1-6 and 7.

10. A storage medium storing instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method of any one of claims 1-6 and 7.

Images