Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

46710 results about "Speech sound" patented technology

Speech sound. noun. 1 : any one of the smallest recurrent recognizably same constituents of spoken language produced by movement or movement and configuration of a varying number of the organs of speech in an act of ear-directed communication.

Multimodal data-based method and system for recognizing cognitive engagement in classroom

PendingUS20250022314A1Speech recognitionEnergy efficient computingCognitive engagementData set
A method and system are introduced for recognizing cognitive engagement in classrooms by utilizing multimodal data. Associated modalities of cognitive engagement are learned from visual and audio data. This approach involves constructing a multidimensional representation model for cognitive engagement that includes behaviors, emotions, and speech. To identify engagement, three distinct deep learning models are employed: You Only Look Once version 8 (Yolov8) for analyzing body posture, Efficient Network (EfficientNet) for facial expressions, and Text Convolution Neural Network (TextCNN) for speech text. These models are trained and refined with the aid of student engagement surveys, leading to a decision-making process that integrates the results from the different modalities. Additionally, a dataset and a data annotation system are developed for engagement recognition. This innovative method aims to achieve detailed engagement recognition, addressing various perception needs in real-world applications.
Owner:HUAZHONG NORMAL UNIV

Domain speech recognition method and system based on RAG

The invention discloses a domain speech recognition method and system based on RAG. The method comprises the steps that an original domain document set is acquired, a hierarchical document structure is constructed through subject classification, paragraph segmentation and sentence segmentation, vectorization processing is executed to generate a multi-granularity vector matrix set, and a hierarchical knowledge vector index structure is established; performing multi-scale time-frequency analysis on the real-time audio stream, extracting and enhancing acoustic features, and generating a multi-scale feature tensor; performing adaptive blocking and progressive attention processing on the feature tensor, constructing a probability estimation model, and generating a candidate text sequence; executing multi-layer retrieval in the knowledge index based on the multi-dimensional query vector to generate related knowledge fragments; and carrying out space mapping and dynamic attention processing on the features, and executing probability optimization decoding to obtain a final recognition text. According to the invention, through knowledge enhancement and multi-level feature processing, the accuracy and robustness of voice recognition in the professional field are improved.
Owner:NANJING ARTIFICIAL INTELLIGENCE CHIPS RES INST OF AUTOMATION CHINESE ACAD OF SCI

Intelligent real-time interactive question-answering system based on virtual digital human

The invention provides an intelligent real-time interactive question-answering system based on a virtual digital human, and belongs to the technical field of voice signal processing and voice recognition, and the system comprises a data acquisition module which receives a voice or text interaction request input by a user, collects the expression dynamic parameter sequence and limb movement sequence data of the user in real time, and transmits the data to a user interaction module; obtaining a standardized voice feature vector and structured text data; the cross-modal fusion module is used for constructing an interactive feature matrix; the behavior decision module outputs a decision instruction set; the knowledge retrieval module is used for generating an answer text with emotional adaptability and voice features; and the voice generation module is used for generating a mouth shape animation key frame, a micro expression parameter sequence and a limb action track of the virtual digital human, generating a voice response in combination with the answer text and the voice characteristics, and pushing the voice response to the user terminal. According to the method, the interaction experience and adaptability of the virtual digital human are remarkably improved.
Owner:XIAMEN DUOXIANG ANIMATION CO LTD

Transform-based cross-modal fusion multi-modal emotion recognition method

The invention discloses a Transform-based cross-modal fusion multi-modal emotion recognition method and device, which are used for solving the problems of modal isomerism, difficulty in time alignment and insufficient dynamic emotion modeling in a multi-modal emotion recognition task, and the method takes the accuracy and robustness of emotion recognition as performance evaluation indexes. Firstly, feature information of three modes of vision, voice and text is obtained, feature extraction is performed on each mode through a deep learning model, then features of different modes are fused by using a cross-mode Transform module, and a complex dependency relationship between the modes is dynamically modeled through a multi-head self-attention mechanism, so that more accurate emotion recognition is realized, and the emotion recognition efficiency is improved. And finally, performing emotion prediction on the fused features based on time sequence modeling and an emotion classification module. According to the method, the problems of modal isomerism, difficulty in time alignment and insufficient dynamic emotion modeling in multi-modal emotion recognition can be effectively solved.
Owner:SOUTHEAST UNIV

Dialogue system and a dialogue method

A dialogue system, comprising: an input configured to receive input data relating to speech or text provided by a user; an output configured to provide output data relating to speech or text to a user; and one or more processors, configured to: receive, by way of the input, input data relating to speech or text provided by a user; provide the input data to a subject safety module, the subject safety module configured to receive the input data and evaluate the input data before a system response is output, evaluating the input data comprising performing a first determination on the input data using a matching module and a second determination on the input data using a trained model, wherein the matching module performs the first determination to determine whether the input data matches one or more items from a pre-determined set of one or more items; generate a system response using a first process or using a second process, wherein the first process uses at least one trained language model to generate a dynamically determined system response and wherein the second process retrieves a pre-determined system response, wherein a selection between the first process and the second process is made based on the evaluation of the input data; output, by way of the output, the system response.
Owner:LIMBIC LTD

Customer data processing and insight system based on large language model

The invention belongs to the technical field of artificial intelligence and big data, and discloses a customer data processing and insight system based on a big language model. The system is composed of a multi-source data access module, a data preprocessing and label fusion module, a large language model semantic understanding module, a knowledge enhancement and semantic linkage module, an insight generation and visualization module, an intelligent strategy output module and a feedback learning and self-optimization module. According to the method, multi-source heterogeneous data such as texts, voices and structured behaviors are integrated, and the deep semantic analysis capability of a large language model is combined, so that global modeling of customer behaviors and intentions is realized; a multi-modal synchronous acquisition and standardization mechanism eliminates data format barriers, and a dynamic label mechanism adapts to context changes, so that the system can capture deep semantic association in customer expression, and compared with a traditional keyword matching method, the semantic understanding accuracy is improved by more than 40%, and a more complete data base is provided for insight generation.
Owner:SICHUAN JUFUREN TECHNOLOGY CO LTD

Content recommendation method and system based on semantic recognition

ActiveCN119089398AAccurate semantic representation of dataEfficient learning processSemantic analysisBiological modelsPersonalizationProcessing
The invention discloses a content recommendation method and system based on semantic recognition, and the method comprises the steps: analyzing the text, voice or video input of a user through a natural language processing technology, and generating user semantic representation data; the method comprises the following steps: dynamically constructing a user interest graph by combining user historical behavior data, and performing representation learning of nodes and edges through a multilayer graph convolutional network to generate time sequence user interest graph representation data; a self-attention mechanism and a multi-head attention mechanism of a Transform model are utilized, the matching degree of user interest representation and recommended content is optimized in combination with a contrast learning module, and personalized recommended content data are generated; collecting multi-modal information of the user, fusing the multi-modal information with the personalized recommendation content data, and generating optimized multi-modal recommendation content data through a multi-modal fusion algorithm; and collecting user feedback feature data, and dynamically adjusting and optimizing a generation strategy of recommended content through an adaptive learning algorithm. According to the invention, efficient, accurate and personalized recommendation of the recommendation system is realized, and the user experience is improved.
Owner:HANGZHOU SCENE INNOVATION CENTER CO LTD

Interactive cyber-security user-interface for cybersecurity components that cooperates with a set of llms

An interactive cyber-security user-interface for cybersecurity components can receive a voice input from a user as well as ii) a text input as a user input. The interactive cyber-security user-interface works with a set of differently trained LLMs to carry out tasks on behalf of the user input. The interactive cyber-security user-interface cooperates with the set of differently trained LLMs, which are grouped together to operate as an orchestrated system to provide different tasks. The tasks can include a collection of supplementary information, a summarization of cyber security information, translating a query in the natural human speech format into the required search syntax, how to integrate with an API, acting as a first line of support to user inquiries, a suggested response to a cyber security issue, etc. The interactive cyber-security user-interface for the cybersecurity components acts as the user interface for one or more of the cybersecurity components.
Owner:DARKTRACE HLDG LTD

System that conducts dialogs using artificial intelligence

An artificial intelligence system that conducts dialogs with one or more participants to achieve an educational outcome, such as teaching material with a Socratic question-and-answer method, reviewing material to reinforce ideas or to prepare a participant for future tests or events, or evaluating a participant's knowledge or communication skills. The system may use an artificial intelligence engine that can understand and generate natural language, such as a large-language model like ChatGPT. Text-to-speech and speech-to-text converters may be coupled to the AI engine to enable the system to communicate directly with participants using spoken language. The system may provide an interface for entry of contextual information for a dialog, such as background information on topics for discussion. It may provide an interface for moderation of a dialog, for example by a teacher, lawyer, thesis advisor, etc. System may evaluate the performance of the participants at the end of a dialog.
Owner:SOCRATIC METRIC LLC

Online teaching interaction method based on multi-modal knowledge graph, medium and equipment

The invention discloses an online teaching interaction method based on a multi-modal knowledge graph, a medium and equipment, and the method comprises the steps: firstly collecting original teaching information, and carrying out the cross-modal semantic alignment processing, and obtaining structured teaching information; then constructing a multi-modal knowledge graph containing text concept entities, video key frame feature vectors and voice text transcription contents, and forming a concept-visual feature association edge, a concept-voice segment association edge and a cross-modal similarity association edge; converting the user interaction behavior data into a knowledge graph query vector, extracting a three-dimensional teaching situation sub-graph from the multi-modal knowledge graph, and generating progressive or comparative teaching content according to a user operation type; and finally, dynamically adjusting a teaching strategy by constructing a cognitive state tracking matrix, and generating a personalized learning navigation map. The problems of knowledge fragmentation and insufficient interaction intellectualization in virtual simulation teaching are solved, and the teaching effect of ideological and political education is improved.
Owner:UNION COLLEGE OF FUJIAN NORMAL UNIV

Vision generation method and device based on semantic association modeling, equipment and medium

The invention relates to the technical field of voice semantics, can be applied to business scenes of financial science and technology, medical health, poster design and the like, and discloses a visual sense generation method and device based on semantic association modeling, equipment and a medium. Generating a demand text containing theme and style parameters; semantic features in the demand text are extracted, semantic association weights are constructed, and element layout coordinates are optimized in combination with spatial distribution constraints; and encoding the layout information into a control matrix, fusing the control matrix with the initial noise, adjusting a noise reduction process through an encoding and decoding network, and generating target visual content highly matched with the semantic meaning of the user instruction. According to the method, the layout optimization function is constructed, the diffusion model is guided to focus the semantic salient region in space, language model output and the visual generation process are closely combined, structured response and space mapping of user semantic requirements are achieved, and the expression consistency and personalized adaptation capacity of visual content generation are improved.
Owner:SHENZHEN PINGAN COMM TECH CO LTD

Intelligent multi-mode virtual digital human interaction system based on AI language large model, interaction method and application

The invention discloses an intelligent multi-modal virtual digital human interaction system based on an AI language large model. The system comprises a high-authenticity face generation module; the high-authenticity face generation module uses an AdaAN network, based on adaptive feature fusion and voice driving and time sequence modeling of voice features, feature information related to voice is extracted, the extracted voice features are processed through a deep neural network, it is ensured that the voice and facial expressions are highly aligned in time and space, and the face recognition accuracy is improved. Collecting a bio-electricity signal, mapping the signal to facial muscle movement, generating a final facial expression, and interacting with a user; the system further comprises an intelligent interaction module, a training optimization and efficient generation module, an efficient integration module, a multi-modal data acquisition module, an AI large model core processing module, a digital human image generation and driving module, an interaction scene adaptation module and a feedback optimization module. The invention further discloses a multi-mode digital human interaction method which has wide application value.
Owner:EAST CHINA NORMAL UNIV

Conference summary processing method and system using AI

The invention relates to the technical field of intelligent conference processing, and relates to a conference summary processing method and system using AI, and the method comprises the steps: carrying out the real-time noise suppression of a collected conference audio stream and associated text data through a noise suppression algorithm, and carrying out the cross-modal alignment of the denoised data through a cross-modal alignment algorithm; a domain-specific attention head is inserted into an attention layer of the pre-trained Transform model, a domain-enhanced speech recognition model is constructed, and audio is converted into a text sequence with a speaker tag; adopting a heterogeneous graph neural network to construct a structured topic evolution graph; key decision nodes in the structured topic evolution graph are extracted based on a reinforcement learning strategy, and a final conference summary document is generated. In the decoding stage, the fusion proportion of the acoustic model and the language model is dynamically adjusted based on the real-time acoustic confidence coefficient, the recognition rate of the vocabularies in the professional field is increased, and the problems of frequent term transcription errors and poor semantic coherence in the professional conference are effectively solved.
Owner:GUANGZHOU DAZZLE VIEW INTELLIGENT TECH CO LTD

E-commerce platform search recommendation and intelligent secretary integrated system

The invention belongs to the technical field of e-commerce, and particularly relates to an e-commerce platform search recommendation and intelligent secretary integrated system, which comprises the following steps of: establishing an accurate user portrait by analyzing behavior data of a user and personalized preference of the user; through a basic search function, a commodity search function, a keyword matching function and an advanced function, a user is helped to quickly find a target commodity; personalized commodity recommendation is provided for the user based on collaborative filtering, and the user experience is improved; a virtual secretary function is provided, and a user is supported to perform natural language interaction; the behavior of the user on the platform is monitored and analyzed in real time, the demand and trend of the user are extracted, and searching and recommendation strategies are optimized; the system supports intelligent interaction of various input modes of voice, texts and pictures, searches commodities through voice or uploads pictures to search pictures through pictures, brings convenient, fast and personalized service experience for vast consumers, and also provides intelligent and efficient operation management for merchants.
Owner:SHENZHEN WOLE TRADING CO LTD

Intelligent real-time emotion evaluation method for social media and online text data based on multi-modal knowledge graph

The invention discloses an intelligent real-time emotion evaluation method for social media and online text data based on a multi-modal knowledge graph, and the method comprises the steps: firstly obtaining and preprocessing multi-modal original data; secondly, designing single-mode feature extraction of dialogue texts, voices and facial expressions based on BERT-BiLSTM, AST and ResNet-50 pre-training models; furthermore, sharing among multiple modes and private emotion semantic information are considered, and complementation of cross-mode emotion information is promoted by enhancing repeated emotion information and utilizing uniqueness of private information. Meanwhile, a multi-mode knowledge graph method is provided from bottom to top to carry out emotion real-time evaluation. Analyzing user behaviors by using DeepFM and generating user features, and capturing external knowledge layer features in combination with K-BERT; finally, on the basis of a TransformerEncoder structure, cross-modal emotion information interaction and feature fusion are achieved, real-time recognition and classification of multi-modal emotion perception are completed through a Softmax layer, and therefore the purpose of intelligent real-time emotion evaluation is achieved.
Owner:XI AN JIAOTONG UNIV

Multi-modal analysis method, system and equipment for industrial inspection scene and medium

ActiveCN119128810ADetailed video image featuresRealize intelligent analysisInference methodsEngineeringVideo image
The invention discloses a multi-modal analysis method, system and device for an industrial inspection scene and a medium. The method comprises the following steps: acquiring multi-modal data of an industrial inspection area in real time, wherein the multi-modal data comprises a video image and voice data; inputting the currently collected multi-modal data into a scene type identification model, and identifying a scene type; matching a corresponding multi-modal fusion mode based on the determined scene type; according to the matched multi-modal fusion mode, performing feature extraction on the video image by using a deep learning algorithm, performing feature extraction on the text converted from the voice data by using a natural language processing technology, and performing multi-modal fusion on the extracted features; performing joint reasoning on the multi-modal fusion features by using a deep learning algorithm, and identifying potential risks or equipment faults in the industrial inspection area; and generating an inspection report and decision based on the identified potential risk or equipment fault. According to the invention, intelligent analysis of the industrial inspection scene can be realized, and the inspection efficiency and accuracy are improved.
Owner:NANJING MAGIC GRP INFORMATION TECH CO LTD

Multi-microphone array beamforming signal enhancement method and device

The invention relates to a multi-microphone array beam forming signal enhancement method and device, and the method comprises the steps: obtaining a multi-channel time domain signal collected by a multi-microphone array, and carrying out the time-frequency conversion, and obtaining a plurality of time-frequency domain signals; acquiring a spatial distribution vector of the multi-microphone array, and performing deformation compensation processing on the complex number of time-frequency domain signals to obtain sound field consistency signals; performing sound source direction feature extraction on the sound field consistency signal, and performing complex time-space attention beam forming network processing to obtain a beam forming signal; performing beam direction and gain parameter optimization on the beam forming signal through a preset adversarial environment simulator to obtain a dynamic frequency band enhanced signal; and performing inverse time-frequency conversion on the dynamic frequency band enhanced signal, and performing low-delay optimization to obtain a target quality voice output signal. According to the invention, signal deformation compensation can be carried out more accurately.
Owner:DONGGUAN HUAZE ELECTRONIC TECH CO LTD

Knowledge management system and method for constructing large voice model

The embodiment of the invention provides a method for constructing a large language model knowledge management system. The method comprises a knowledge construction step, a knowledge retrieval step and an answer production step. According to the scheme, the artificial intelligence large language model technology can be utilized. External knowledge is obtained from multiple data sources, a search technology and a cue word function of a large language model are combined, RAG proposes a problem to the large model, knowledge obtained by searching the problem in the multiple data sources is used as a background context, and information of the problem and the background context is integrated into cue words of the large language model. And therefore, the large language model can make a final accurate answer.
Owner:BEIJING XJ ELECTRIC

Digital human real-time interaction system and digital human real-time interaction method

The invention discloses a digital human real-time interaction system and method. The interaction system comprises terminal equipment and a cloud server. The terminal device receives voice input by a user and transmits the voice to the cloud server. The cloud server comprises a video acquisition and processing module, an intelligent question and answer and voice generation module, an audio processing module and an image rendering module. A video capture and processing module captures original video data indicative of an object of a digital figure and generates successive image frames. The intelligent question and answer and voice generation module converts the voice into a text; performing semantic analysis on the text, understanding the intention of the user, and generating a corresponding reply text; the reply text is also converted into a reply voice. The audio processing module processes the reply voice and the continuous image frames to generate first video data. And the image rendering module fuses the first video data with the virtual background, and performs real-time processing on the fused video to generate a response video. And the terminal equipment outputs the response video.
Owner:BEIJING HONGLIAN 95 INFORMATION IND CO LTD +1

Intelligent sound box voice processing method and system based on artificial intelligence

The invention provides an intelligent sound box voice processing method and system based on artificial intelligence, and the method comprises the steps: obtaining audio data and mouth shape video data, carrying out the processing of the audio data and the mouth shape video data, and carrying out the multi-modal feature fusion, and obtaining a fusion feature; performing bimodal voice activity detection on the fusion features to obtain effective voice data; performing context sensing recognition of audio and video fusion on the effective voice data to obtain a first text; constructing a user feature model, and performing semantic understanding on the text based on the model to obtain an understanding result; performing intention recognition and slot filling based on the understanding result to obtain user intention and key information; generating a response strategy in combination with the user intention, the key information and the environment perception data; generating response voice according to the response strategy; and monitoring feedback information of the user to the response voice in real time, and updating the user feature model and the response strategy evaluation model based on feedback. According to the scheme, the voice can be recognized more accurately, and the safety and robustness of the system are enhanced.
Owner:SHENZHEN ZHANDIAN SMART TECH CO LTD

Voice data interaction feedback control processing method based on large language model

The invention relates to the technical field of large language models, and discloses a voice data interaction feedback control processing method based on a large language model. The method comprises the following steps: acquiring an industrial voice instruction through an AMR main controller, and obtaining a standardized vector through industrial lexicon matching and intention classification; inputting an industrial large language model for reasoning processing, and generating an AMR execution scheme; carrying out distributed coordination and task allocation on the AMR cluster to form a control instruction sequence; and the motion controller executes monitoring, processes exceptions and outputs a feedback strategy. Accurate classification and semantic understanding of complex industrial instructions are achieved, the processing capacity of professional knowledge in the industrial field is improved, and meanwhile high-precision real-time state monitoring is achieved.
Owner:TIANJIN HONGHUANG TECH CO LTD

Systems, methods, and graphical user interfaces for training a code generation model for low-resource languages

A system, method, and computer-program product includes identifying a plurality of code synthesis items for a target programming language, generating a code synthesis prompt based on a first sampling of the plurality of code synthesis items, synthesizing, via a large language model, a plurality of raw code segments using the code synthesis prompt, executing the plurality of raw code segments with a code interpreter associated with the target programming language, determining one or more valid code segments of the plurality of raw code segments that the code interpreter successfully executed, aggregating, via a second sampling, the one or more valid code segments into one or more validated code synthesis training samples, and training a code generation model using the one or more validated code synthesis training samples. User interfaces may be provided to allow target coding tasks to be specified via text or speech.
Owner:SAS INSTITUTE INC

Real-time virtual reality scene system based on natural language description using multimodal artificial intelligence

A real-time system for the multimodal generation of virtual reality scenes based on artificial intelligence for the creation of immersive three-dimensional environments from natural language narratives, consisting of: a speech capture module configured to continuously record a user's spoken narrative via one or more directional microphones, preprocesses the captured signal by noise reduction and temporal alignment, and outputs a digital speech stream; A speech-to-text processing unit that is operationally coupled to the speech capture module and configured for real-time speech recognition using a continuous neural transformer model. The unit is trained to transcribe natural language utterances into structured text data while maintaining contextual continuity throughout the evolving narrative. a semantic interpretation processing unit that is communicatively linked to the speech recognition unit and configured to perform natural language understanding techniques to extract contextual entities, spatial references, temporal relationships, and object attributes from the transcribed narrative; the engine includes a large language model that is fine-tuned for spatial reasoning tasks; a scene graph generation module configured to transform the interpreted semantic data into a structured, hierarchical representation that defines nodes for identified entities and edges for corresponding relationships, with each node associated with metadata describing geometry, position, orientation, texture, and linking attributes between objects; a multimodal image-language model processor coupled with the scene graph generation module, wherein the processor is configured to retrieve, adapt, or synthesize appropriate three-dimensional elements from a pre-trained visual-lexical embedding space and align these elements with their semantic and spatial definitions derived from the scene graph; a scene assembly and rendering controller configured to create a cohesive virtual scene from the aligned assets, perform real-time rendering using a GPU-accelerated ray tracing pipeline, and produce a stereoscopic visual output that corresponds to the evolving narrative; A head-mounted virtual reality visualization device connected to the rendering engine and configured to display the generated immersive environment to the user in real time. The device features motion sensors and inside-out tracking cameras to detect head and body movements, dynamically updating viewing angles and perspective within the rendered scene; and a bidirectional feedback module integrated into the head-mounted device and connected to the semantic interpretation processing unit; the module is configured to interpret corrective commands, gestures, or supplementary comments from the user to refine or modify specific scene elements without interrupting the real-time visualization; The system continuously updates the virtual scene as the narrative develops, ensuring temporal synchronization between speech input and rendered output below a defined latency threshold, thus enabling a natural, dialogic construction of complex three-dimensional virtual environments.
Owner:GOUNDER MOHAN SELLAPPA DR BENGALURU +3

Automatic labeling and acquisition standardization method and system based on video content

The invention discloses an automatic labeling and acquisition standardization method and system based on video content, and the method comprises the steps: decoding a video file into a processable video frame sequence, carrying out the standardization processing, unifying the format and the coding standard, and adjusting the resolution and the frame rate; performing content analysis on the standardized video frame sequence, extracting key frames, identifying objects, scenes and actions, and generating a video analysis result and a preliminary label; according to a video content analysis result, separating an audio stream from the video to carry out noise reduction processing and voice-to-text analysis, and combining the processed text with the video content analysis result to generate multi-modal comprehensive annotation information; based on the comprehensive labeling information, taking feedback of the user in video retrieval, label clicking and watching behaviors as an adjustment basis of label weights, and updating weight distribution of labels by analyzing user behavior data; labeling information and weight distribution are integrated into an index database, and indexes are updated in real time according to user behaviors and content changes.
Owner:NANJING SUYI IND +1

Intelligent mechanical arm control system scheme based on multi-model cooperation

The invention provides an intelligent mechanical arm control system scheme based on multi-model cooperation, and belongs to the field of automation and intelligent control. The system is divided into four parts of man-machine interaction, detection and perception, task decoupling planning and program generation, and robot execution operation. Man-machine interaction is carried out, a user inputs an instruction in a text or voice format, and instruction processing is carried out through voice recognition and a natural language translation model; detecting and sensing, and identifying and positioning a target captured by the camera by using a multi-modal large model; task decoupling planning and program generation: analyzing the user instruction and the perception information into an executable program by applying a task planning model and a program translation model; and checking and correcting the program by using the program checking model, and driving the mechanical arm to complete a task. According to the system scheme, variable environments can be dealt with, complex language instructions can be understood, intelligent control over the mechanical arm is achieved, and the multi-task execution capacity and the intelligent level of the mechanical arm are remarkably improved.
Owner:NORTHEASTERN UNIV CHINA

Systems and methods for providing voice assistance in a vehicle

Methods, computing systems, and technology are presented for providing voice assistance in a vehicle. For example, a computing system may be configured to access a first voice prompt indicative of a speech-based voice query provided by a user of the vehicle. The computing system may be configured to determine, based on the first voice prompt, a first prompt message corresponding to a text-based transcription of the first voice prompt. The computing system may be configured to process the first prompt message with a machine-learned large language model trained to generate a digital message response based on the first prompt message. The computing system may be configured to convert the digital message response to a speech-based voice response. The computing system may be configured to provide the speech-based voice response as audio output to the user of the vehicle.
Owner:MERCEDES BENZ GROUP AG

Multi-modal digital employee reception system and method based on edge calculation

The invention relates to a multi-modal digital employee reception system and method based on edge calculation, and relates to the technical field of artificial intelligence, and the system comprises a multi-modal data collection module, an edge calculation module, a data fusion and decision module, a trigger activation module, an emotion recognition module, an interaction response module and a behavior prediction module. The method comprises the following steps: acquiring voice, image and action information of a user through a high-resolution camera, a microphone array and an infrared sensor; locally processing data by using an edge calculation module, and extracting voice, vision and thermal imaging features; a Transformer model is dynamically optimized through Test Time Training (TTT), multi-modal data are dynamically fused in combination with a multi-head attention mechanism, a comprehensive decision vector is generated, the system can accurately judge the emotional state of a user, user behaviors are predicted based on a fused Token sequence and historical records, personalized services are actively provided, and by dynamically adjusting a trigger threshold value and a weight distribution mechanism, the emotion state of the user can be accurately judged. The false triggering rate and the missed triggering rate are reduced, and the stability and the robustness of the system are enhanced.
Owner:BEIJING HUAYUAN TECHNOLOGY CO LTD

Marketing video auditing method based on AI

The invention provides an AI-based marketing video auditing method, and relates to the technical field of AI marketing video auditing, and the method comprises the steps: obtaining a multi-modal data original structure set, and extracting image semantic features, voice expression features, text semantic features and scene label information, and obtaining an image semantic feature set, a visual rhythm feature set, a voice expression feature set, a voice and picture synchronous association vector structure, a text semantic feature set and a subtitle semantic and image main body linkage relation graph. By constructing an image semantic feature set, a voice expression feature set, a text semantic feature set and a visual rhythm feature set and fusing the image semantic feature set, the voice expression feature set, the text semantic feature set and the visual rhythm feature set into a multi-modal content fusion feature tensor, unified modeling of an AI marketing video at visual, auditory and semantic levels can be realized; and subsequent microscopic consistency detection, compliance knowledge graph and emotion semantic conflict identification are effectively performed, so that full-link risk perception and accurate auditing of video contents are realized.
Owner:SHANGHAI WANGMAI INFORMATION TECH GRP CO LTD

Devices for non-audible speech detection

Provided herein is a device for non-audible speech detection. The device comprises at least one transmitting antenna configured to transmit a plurality of radio frequency (RF) signals. The device further comprises at least one receiving antenna configured to collect RF signal data associated with movement of one or more speech articulators of a user, wherein the device is configured to be coupled to a head of the user, wherein the RF signal data is processed to determine a non-audible speech of the user, and wherein the non-audible speech of the user comprises continuous speech by the user.
Owner:REFLEX TECHNOLOGIES INC

Semi-supervised dialect emotion speech synthesis system based on hybrid experts

The invention relates to dialect speech synthesis, in particular to a semi-supervised dialect emotion speech synthesis system based on hybrid experts, which comprises a text analysis module for preprocessing an input dialect text and generating a text representation vector through feature extraction and feature fusion; the mixed expert module is used for acquiring dialect acoustic features, rhythm features, emotional features and general acoustic features; the dynamic routing module is used for realizing intelligent cooperation among experts through a task-aware soft routing algorithm; the semi-supervised learning module is used for training supervised learning by using the marked dialect emotion voice data and training self-supervised learning by using the unmarked dialect emotion voice data; the acoustic parameter generation module is used for integrating the output of each expert to generate a complete acoustic parameter set; the neural vocoder is used for converting the acoustic parameter set into final dialect emotion voice; the method can effectively overcome the defect that the dialect emotional voice is difficult to accurately synthesize under the condition of lack of sample resources.
Owner:ANHUI XINGHONGYE INTELLIGENT TECH CO LTD