39370results about "Mechanical pattern convertion" patented technology

The invention provides an intelligent real-time interactive question-answering system based on a virtual digital human, and belongs to the technical field of voice signal processing and voice recognition, and the system comprises a data acquisition module which receives a voice or text interaction request input by a user, collects the expression dynamic parameter sequence and limb movement sequence data of the user in real time, and transmits the data to a user interaction module; obtaining a standardized voice feature vector and structured text data; the cross-modal fusion module is used for constructing an interactive feature matrix; the behavior decision module outputs a decision instruction set; the knowledge retrieval module is used for generating an answer text with emotional adaptability and voice features; and the voice generation module is used for generating a mouth shape animation key frame, a micro expression parameter sequence and a limb action track of the virtual digital human, generating a voice response in combination with the answer text and the voice characteristics, and pushing the voice response to the user terminal. According to the method, the interaction experience and adaptability of the virtual digital human are remarkably improved.

In some embodiments, a computer system changes a visual prominence of a respective virtual object in response to detecting a threshold amount of overlap between a first virtual object and a second virtual object. In some embodiments, a computer system changes a visual prominence of a respective virtual object based on a change in spatial location of a first virtual object with respect to a second virtual object. In some embodiments, a computer system applies visual effects to representations of physical objects, virtual environments, and / or physical environments. In some embodiments, a computer system changes a visual prominence of a virtual object relative to a three-dimensional environment based on display of overlapping objects of different types in the three-dimensional environment. In some embodiments, a computer system changes a level of opacity of a first virtual object overlapping a second virtual object in response to movement of the first virtual object.

The present disclosure generally relates to techniques and user interfaces for controlling and displaying representations of user in environments, such as during a live communication session and / or a live collaboration session.

A knowledge fusion multi-modal interaction method and apparatus based on an improved alignment method. The method comprises: acquiring multi-modal data (S101); constructing a query transformation model (S102); on the basis of a multi-stream feature extraction method, performing improved processing on a vision transformer to obtain a multi-scale vision transformer (S103); on the basis of a joint text method, training the multi-scale vision transformer to obtain a target vision transformer (S104); freezing the target vision transformer, and on the basis of the target vision transformer, performing vision-text alignment training processing on the query transformation model to obtain a target query transformation model (S105); connecting the target query transformation model to a language model to obtain a knowledge fusion multi-modal interaction model (S106); and inputting the multi-modal data into the knowledge fusion multi-modal interaction model to obtain an interaction result (S107). In the embodiments of the present invention, the knowledge fusion multi-modal interaction model aligns visual representation with text representation, so that the mutual information between the visual representation and the text representation can be maximized, and the present invention can be widely applied to the technical field of artificial intelligence.

Some examples of the disclosure are directed to systems and methods for displaying an editing a virtual scene. In some examples, an electronic device can display an immersive virtual scene. In some examples, the immersive virtual scene is shared in a communication session with other electronic devices. In some examples, the electronic device can detect user input requesting insertion and display of an annotation into the virtual scene. In some examples, the electronic device can determine user context to determine placement and content associated with the inserted annotation, and display a representation of the annotation. In some examples, the electronic device can capture a virtual screenshot of the virtual scene. In some examples, the electronic device can display a user interface providing an overview of the annotations associated with the virtual scene.

Some examples of the disclosure are directed to systems and methods for displaying and interacting with a representation of a user interface of an electronic device in a three-dimensional environment. While presenting a first user interface, a first electronic device presents a representation of a second user interface of a second electronic device based on display data representing the second user interface. In response to detecting a respective event corresponding to user input, if a gaze of a user of the first electronic device is directed to the representation of the second user interface, the first electronic device causes the second electronic device to perform a first operation directed to the second user interface based on the respective event. Alternatively, if the gaze is directed to the first user interface, the first electronic device performs a second operation directed to the first user interface based on the respective event.

In some embodiments, an electronic device enhances interactions with virtual objects in a three-dimensional environment. In some embodiments, an electronic device enhances interactions with selectable user interface elements. In some embodiments, an electronic device enhances interactions with slider user interface elements. In some embodiments, an electronic device moves virtual objects in a three-dimensional environment and facilitates accessing actions associated with virtual objects.

Some examples of the disclosure are directed to methods for application-based spatial refinement in a multi-user communication session including a first electronic device and a second electronic device. While the first electronic device is presenting a three-dimensional environment, the first electronic device receives an input corresponding to a request to move a shared object in the three-dimensional environment. In accordance with a determination that the shared object is an object of a first type, the first electronic device moves the shared object and an avatar of a user in the three-dimensional environment in accordance with the input. In accordance with a determination that the shared object is an object of a second type, different from the first type, and the input is a first type of input, the first electronic device moves the shared object in the three-dimensional environment in accordance with the input, without moving the avatar.

Systems, devices, and methods for presenting audio associated with events associated with spatialized audio effects or non-spatialized audio effects in three-dimensional environments are disclosed. The spatialized and / or non-spatialized audio effects correspond with displaying, via a display generation component, a three-dimensional environment from a viewpoint of a user. While displaying the three-dimensional environment from the viewpoint of the user, the computer system detects an event. When the computer system detects an event which corresponds to a spatialized sound effect, the computer system presents the sound effect as emanating from a location in the three-dimensional environment associated with the event. When the computer system detects an event which corresponds to a non-spatialized sound effect, the computer system presents the sound effect as emanating from a predetermined location for audio in the three-dimensional environment that has a predetermined spatial relationship relative to the viewpoint of the user

Some examples of the disclosure are directed to systems and methods for changing a level of zoom of displayed content. In some examples, an electronic device displays visual content. In some examples, in response to detecting a change in position and / or orientation of the user of the electronic device, in accordance with a determination that the position and / or orientation of the user satisfies the one or more criteria, the electronic device increases the size of the content and / or zooms in on the content.

A computer system displays representations of second computer systems included in a communication session and / or content in an environment in some embodiments. For example, the computer system displays the representations of the second computer systems in a first region while content is not shared in the communication session and displays the representations of the second computer systems in a second region different from the first region concurrently with content that is shared in the communication session. In some embodiments, the computer system displays the representations of the second computer system (e.g., a representation of a shared content item) such that it partially occludes a representation of the one or more participants of the video communication session when the representation of the one or more participants is a non-spatial representation.

An extended reality environment includes one or more virtual objects. A virtual object can be selected for manipulation by orienting a pointing device towards the virtual object and performing a selection input. In some embodiments, if the selection input is a first type of selection gesture received at the pointing device, the virtual object is selected for movement operations. In some embodiments, if the selection input is a second type of selection gesture received at the pointing device, the virtual object is selected for rotation operations. While the virtual object is selected for movement or rotation operations, the virtual object moves or rotates in accordance with the movement or rotation of the user's hand, respectively. In some embodiments, the virtual object has a first visual characteristic when the pointing device is pointing towards the virtual object and has a second visual characteristic when the virtual object is selected.

Some examples of the disclosure are directed to systems and methods for scrolling a user interface element. A user interface element including a plurality of selectable options scrollable along a first axis can be presented in an environment by a device. In some examples, the device detects an input corresponding to a request to scroll the user interface element. In accordance with a determination that the input satisfies one or more first criteria, including a criterion that is satisfied when the input includes movement of the device that corresponds to movement of a head of a user that exceeds a threshold amount of movement over a period of time, the device scrolls the user interface element in a first mode of operation. In accordance with a determination that the input satisfies one or more second criteria, the device scrolls the user interface element in a second mode of operation.

In some embodiments, an electronic device automatically updates the orientation of a virtual object in a three-dimensional environment based on a viewpoint of a user in the three-dimensional environment. In some embodiments, an electronic device automatically updates the orientation of a virtual object in a three-dimensional environment based on viewpoints of a plurality of users in the three-dimensional environment. In some embodiments, the electronic device modifies an appearance of a real object that is between a virtual object and the viewpoint of a user in a three-dimensional environment. In some embodiments, the electronic device automatically selects a location for a user in a three-dimensional environment that includes one or more virtual objects and / or other users.

A system and method for creating complex, immersive, and interactive digital content is disclosed. The system integrates advanced artificial intelligence, multi-modal input processing, cloud-based shared environments, and immersive hardware to generate, optimize, and deliver rich interactive experiences. The platform supports content mashups, custom scenario generation, and adaptive AI behaviors, enabling the creation of unique and engaging digital environments across various media formats.

Some examples of the disclosure are directed to systems and methods for communicating image sensor data to a computer system using one or more media links that allow for image sensor data to be displayed within a three-dimensional environment. In some examples, while presenting a three-dimensional environment that includes an electronic device that is communicatively coupled to an image sensor (e.g., a camera, or some other imaging device) within the three-dimensional environment, the electronic device utilizes a visual code displayed on the electronic device to establish a communication link with the image sensor receiving an indication to display image sensor data from the image sensor that is communicatively coupled to the electronic device. In some examples, while presenting a three-dimensional environment that includes a first pre-defined location within the three-dimensional environment, the electronic device facilitates the user to choose an image sensor to display at the pre-defined location.

In some embodiments, an electronic device facilitates cursor interactions in different regions in a three-dimensional environment. In some embodiments, an electronic device facilitates cursor interactions in content. In some embodiments, an electronic device facilitates cursor movement. In some embodiments, an electronic device facilitates interaction with multiple input devices. In some embodiments, a computer system facilitates cursor movement based on movement of a hand of a user of the computer system and a location of a gaze of the user in the three-dimensional environment. In some embodiments, a computer system facilitates cursor selection and scrolling of content in the three-dimensional environment.

In some embodiments, a computer system changes a visual prominence of a respective virtual object in response to detecting a threshold amount of overlap between a first virtual object and a second virtual object. In some embodiments, a computer system changes a visual prominence of a respective virtual object based on a change in spatial location of a first virtual object with respect to a second virtual object. In some embodiments, a computer system applies visual effects to representations of physical objects, virtual environments, and / or physical environments. In some embodiments, a computer system changes a visual prominence of a virtual object relative to a three-dimensional environment based on display of overlapping objects of different types in the three-dimensional environment. In some embodiments, a computer system changes a level of opacity of a first virtual object overlapping a second virtual object in response to movement of the first virtual object.

A vehicle dash cam may be configured to execute one or more neural networks (and / or other artificial intelligence), such as based on input from one or more of the cameras and / or other sensors associated with the dash cam, to intelligently detect safety events in real-time. Detection of a safety event may trigger an in-cab alert to make the driver aware of the safety risk. The dash cam may include logic for determining which asset data to transmit to a backend server in response to detection of a safety event, as well as which asset data to transmit to the backend server in response to analysis of sensor data that did not trigger a safety event. The asset data transmitted to the backend server may be further analyzed to determine if further alerts should be provided to the driver and / or to a safety manager.

The method can include: recording a neural state, determining a latent array, and determining an output based on the latent array. In variants, the method can function to enable a subject to use neural signals to navigate within a high-dimensional concept space to control an output (e.g., image generation, motor control, communication, etc.).

In some embodiments, an electronic device displays a user interface of a content application in a three-dimensional environment with or without virtual lighting effects applied to the three-dimensional environment depending on whether the user interface is in a mode that does not include virtual lighting or a mode that includes virtual lighting. In some embodiments, an electronic device updates display of interactive user interface elements in response to detecting a respective pose of a respective portion of a user of the electronic device.

The disclosed system for interacting with objects in an extended reality (XR) environment generated by an XR system may include (1) neuromuscular sensors configured to sense neuromuscular signals from a wrist of a user and (2) at least one computer processor programmed to (a) determine, based at least in part on the sensed neuromuscular signals, information relating to an interaction of the user with an object in the XR environment and (b) instruct the XR system to, based on the determined information relating to the interaction of the user with the object, augment the interaction of the user with the object in the XR environment. Other embodiments of this aspect include corresponding apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

The invention discloses a global dynamic optimization system for an ultra-high-definition glass-based display screen. The operation process of the system specifically comprises the following steps: acquiring a real-time state data set of the display screen; generating an initial optimization parameter set based on a preset optimization strategy and the user behavior model; capturing a user interaction signal and an environment change signal in real time; generating a dynamic correction coefficient set according to the user interaction signal and the environment change signal; performing global parameter fusion based on the initial optimization parameter set and the dynamic correction coefficient set to generate a target optimization instruction set; and feeding back an optimization effect evaluation table based on the optimization effect. The method has the following advantages and effects: collaborative optimization and thermodynamic balance of global display parameters are realized, the system can balance visual experience improvement and thermal diffusion suppression requirements in real time through multi-dimensional data fusion and global parameter collaborative calculation, and the system can improve the visual experience while enhancing local contrast and optimizing color rendition. And actively inhibiting the temperature rise of the screen through thermodynamic equilibrium constraint.

A multi-modal articulation system translates human gestures into control commands for digital and automation devices. The system integrates a camera module, which captures facial and hand movements, with an image processing unit that extracts facial features, such as eyebrow movement, lip curvature, eyelid motion, and eye trajectory. These extracted features are analyzed in real time by an articulation recognition module that compares them against a database to recognize specific gestures. The system tracks and processes air-drawn hand gestures using a gesture trajectory tracking unit, which converts the motions into digital representations and classifies them using deep learning algorithms. The recognized gestures are converted into machine-readable control signals and executed on various connected devices via a relay control unit. The system incorporates adaptive algorithms to improve accuracy, account for environmental conditions, and stabilize hand tremors, providing a gesture-based control interface for applications ranging from smart home systems to multimedia devices.

A three-dimensional preview of content can be generated and presented at an electronic device in a three-dimensional environment. The three-dimensional preview of content can be presented concurrently with a two-dimensional representation of the content in a content generation environment presented in the three-dimensional environment. While the three-dimensional preview of content is presented in the three-dimensional environment, one or more affordances can be provided for interacting with the one or more computer-generated virtual objects of the three-dimensional preview. The one or more affordances may be displayed with the three-dimensional preview of content in the three-dimensional environment. The three-dimensional preview of content may be presented on a three-dimensional tray and the one or more affordances may be presented in a control bar or other grouping of controls outside the perimeter of the tray and / or along the perimeter of the tray.

In some embodiments, a computer system performs virtual object manipulation operations using respective portions of the user's body and / or input device(s). In some embodiments, a computer system manipulates a virtual object based on input from a hand of a user and / or a handheld device. In some embodiments, a computer system manipulates a virtual object directly or indirectly.

The invention discloses a digital human real-time interaction system and method. The interaction system comprises terminal equipment and a cloud server. The terminal device receives voice input by a user and transmits the voice to the cloud server. The cloud server comprises a video acquisition and processing module, an intelligent question and answer and voice generation module, an audio processing module and an image rendering module. A video capture and processing module captures original video data indicative of an object of a digital figure and generates successive image frames. The intelligent question and answer and voice generation module converts the voice into a text; performing semantic analysis on the text, understanding the intention of the user, and generating a corresponding reply text; the reply text is also converted into a reply voice. The audio processing module processes the reply voice and the continuous image frames to generate first video data. And the image rendering module fuses the first video data with the virtual background, and performs real-time processing on the fused video to generate a response video. And the terminal equipment outputs the response video.

A system may include a robotic control circuit configured to control one or more robotic functions of a robot. A system may include a plurality of sensors configured to collect data. A system may include a governance analysis circuit configured to analyze the data and select one or more governance frameworks based on the analyzed data. A system may include a governance model circuit configured to generate a model that applies the one or more governance frameworks to determine one or more governance actions, wherein the robotic control circuit is configured to control the one or more robotic functions in accordance with the one or more governance actions, wherein the robotic control circuit, the governance analysis circuit, and the governance model circuit are integrated on a single substrate.

A system comprises a processor and a memory having computer readable instructions stored thereon, which, when executed by the processor, cause the system to display an image of a surgical environment. The image includes an information icon corresponding to an instrument in the surgical environment. The instructions further cause the system to display an image of a body part of a user. The image of the body part is used to interact with the information icon. The instructions further cause the system to receive a gesture of the body part, via a gesture-based input device registering movement of the body part, to cause the image of the body part to interact with the information icon. The instructions further cause the system to display an information menu in the image of the surgical environment based on the interaction between the image of the body part and the information icon.