270 results about "Multimodal interaction" patented technology

The invention relates to the field of multi-modal interaction analysis, in particular to a multi-modal interaction method and system of a digital human agent. The method comprises the following steps: acquiring a real-time face image and a voice signal input stream of an interactive user based on an intelligent agent; performing real-time micro-expression recognition and deep emotion analysis based on the real-time facial image to obtain real-time emotion features of the user; performing time sequence evolution analysis on the real-time emotion characteristics of the user, performing holographic user emotion deep mining, and constructing a user emotion holographic characteristic spectrum; carrying out adaptive acoustic gain processing on the voice signal input stream, and carrying out voice-emotion association analysis based on the user emotion holographic characteristic spectrum to generate a voice-emotion linkage mapping spectrum; and carrying out eyeball fixation point migration tracking based on the user emotion holographic feature map and the real-time face image, and generating a user interaction depth intention signal. Through the real-time deep semantic understanding and emotion perception ability, the intelligent agent interaction intelligence and response accuracy are improved.

The invention discloses a multi-mode interaction method for an accompanying robot, and relates to the technical field of robot interaction.The multi-mode interaction method comprises the steps that voice, visual and tactile signals are converted into quantum states through multi-mode quantum state coding, emotion weights are dynamically fused, and unified quantum representation is constructed; the dynamic quantum decision engine analyzes environmental noise and user emotion intensity in real time based on a quantum measurement theory, generates an adaptive strategy through dynamic modal weight distribution, and solves a multi-modal instruction conflict; holographic reinforcement learning optimization is combined with quantum acceleration calculation and classical reinforcement learning, the reward function weight is dynamically adjusted, cross-modal knowledge migration is achieved through the quantum tunneling effect, and the system strategy is continuously optimized. The problems that a traditional multi-mode interaction system is rigid in mode switching, high in emotion recognition error rate, lack of self-adaptive optimization of a feedback strategy and the like are solved, and the interaction efficiency and the user experience are improved.

The invention belongs to the technical field of artificial intelligence, and discloses a government information consultation system based on a large language model, which comprises a user interaction module, a large language model core engine, a knowledge base integration module, a multi-level authority management module, a feedback optimization mechanism and a risk control module. The comprehensive intelligent government affair service system is constructed through the six core modules, remarkable advantages are shown in government affair service digital transformation, the system innovatively adopts multi-mode interactive design, multiple input and output modes of voice, text and images are supported, an intelligent authority management mechanism is matched, and the intelligent authority management mechanism is matched with the intelligent authority management mechanism. According to the technical scheme, the accessibility and convenience of government affair services are greatly improved, precise services for different user groups are achieved, it is guaranteed that sensitive data are safe and controllable while wide spreading of government affair information is guaranteed, and a large language model of a system core is subjected to professional government affair scene optimization training and is combined with a dynamically-updated knowledge graph technology.

The invention relates to the technical field of XR extension, and discloses an XR augmented reality-based educational culture multi-mode interactive learning system, which comprises a culture scene intelligent construction module used for constructing an interactive historical scene through LiDAR scanning, ancient books and literature and AI generation technologies, dynamically associating cultural relics, buildings, historical events and scientific principles based on a knowledge graph technology, and establishing a multi-mode interactive learning model based on the knowledge graph technology; the multi-modal interactive learning system comprises an AR (Augmented Reality) subsystem, a VR (Virtual Reality) deep learning workshop, an AI (Artificial Interaction) real-time error correction function and an interdisciplinary fusion education platform, wherein the AR subsystem supports scanning textbooks to trigger a 3D (Three-Dimensional) scene and explores an internal structure of a cultural relic through gesture operation, and the VR deep learning workshop provides force feedback gloves to simulate non-abandoned tool operation and AI (Artificial Interaction) real-time error correction function. According to the educational culture multi-mode interactive learning system based on the XR augmented reality, visual interaction of historic building mechanical analysis and process mathematical modeling is achieved, a cross-region cooperation space is achieved, the educational culture multi-mode interactive learning system based on the XR augmented reality forms a dynamic optimization closed loop, and the teaching efficiency and the learning effect are remarkably improved.

The invention discloses an intelligent digital human training method and system based on multi-modal interaction, and belongs to the technical field of semantic indexing.The method specifically comprises the steps that voice, vision and text data are analyzed and converted into high-dimensional feature vectors through a modal exclusive encoder, the high-dimensional feature vectors are projected to a unified semantic space through a cross-modal semantic mapping model, and the high-dimensional feature vectors are obtained; generating a semantic primitive containing a modal identifier, a core semantic tag and a feature weight; semantic primitives are used as nodes, directed edges and edge weight table association strength are established based on semantic similarity, typical scene node connection weights are strengthened, and a mesh map containing intra-modal hierarchy and inter-modal cross association is formed; constructing a double-layer index on the basis of the mesh map; semantic primitives are extracted from newly added data, the position of a new node in an association graph is determined through a graph matching algorithm, an association edge with an existing node is automatically established, and a lower-layer modal exclusive index is synchronously updated.

The invention discloses an AI-based digital media interface design optimization method, and relates to the technical field of interface design, and the method comprises the following steps: collecting multi-modal interaction input data of a user in real time, and constructing a user behavior sequence tensor with multiple time steps and multi-modal interaction dimensions; carrying out joint modeling on the user focus state vector, the user behavior sequence tensor and the user real-time feedback vector; generating a recommended new layout scheme based on an output result of the interactive intention weight model; when the probability is higher than a preset threshold value, triggering an interface pushing mechanism; after the interface is rearranged, data are fed back according to subsequent interaction behaviors and feedback data of the user; the technical problem that the layout or position of the interface element cannot be dynamically adjusted according to the behavior of the user in the traditional interface design is solved.

The invention relates to the technical field of display terminal interaction, in particular to an intelligent display terminal multi-mode interaction method and system. Comprising a multi-module acquisition unit, and the multi-module acquisition unit is used for acquiring visual, voice and text data; the modal feature extraction unit is used for extracting key semantic features from vision, voice and texts, the differential fusion unit dynamically allocates fusion coefficients based on weight coefficients of all modals, and an optimal fusion logic is selected for different input modal combinations by adopting a differential fusion strategy; and the multi-module response unit is used for outputting the multi-mode content and outputting and displaying the multi-mode content through the terminal equipment. The current available modal combination is identified through the differential fusion strategy, and the optimal fusion sub-module is scheduled, so that different fusion logics are dynamically called during operation, different fusion sub-modules are dynamically selected according to modal availability and switched during operation, and the problems of resource waste and precision reduction caused by general fusion are avoided.

The invention relates to a customer service interaction method and system fusing AI digital employees and multi-agent decision, and the method comprises the steps: receiving a multi-mode interaction request from a user, converting the multi-mode interaction request into interaction text data in a unified format, and forwarding the interaction text data to a multi-agent decision unit. And performing intention recognition and sentiment analysis on the interactive text data through the multi-agent decision-making unit to obtain a recognition analysis result. And based on the identification analysis result, guiding the interaction process of the user in combination with the historical interaction content so as to determine the interaction task type, and feeding back the interaction task of the corresponding type to the AI digital employee. And in response to the interaction task, calling the AI digital employee to execute the interaction task according to the rules and knowledge in the knowledge base, and generating a task execution result. According to the task execution result and the recognition analysis result, reply content based on the multi-modal interaction request is generated, the reply content is fed back to the user side, and the flexibility and accuracy of the interaction process are improved.

The embodiment of the invention relates to the technical field of digital twinning, in particular to a digital twinning multi-modal practical training teaching interaction method and system combined with an industrial scene, and the method comprises the steps: obtaining a real-time operation data set of target industrial equipment and a multi-modal data set of a practical training teaching environment, constructing a virtual twinborn model of the target industrial equipment according to the real-time operation data set, performing dynamic feature fusion processing on the multi-modal data set, generating a multi-modal interaction feature set synchronized with the virtual twinborn model, and training the target industrial equipment on the basis of a preset training teaching strategy network. And carrying out collaborative analysis on the multi-modal interaction feature set and the real-time operation state of the virtual twin model, generating an interaction feedback instruction of the practical teaching environment, and adjusting parameters of the practical teaching strategy network according to the interaction feedback instruction to optimize a teaching path.

The invention discloses a multi-modal interactive fusion virtual reality emotion computing system, which comprises a multi-modal interactive fusion virtual reality emotion computing device, a multi-modal interactive fusion virtual reality emotion computing device and a multi-modal interactive fusion virtual reality emotion computing device, the virtual reality emotion computing device based on multi-modal interaction fusion further comprises a distributed multi-modal sensing device, a user adaptation interaction module, an interaction feedback module, an emotion extraction module, a multi-modal fusion module and a self-adaptation emotion computing model. The multi-modal fusion module is used for carrying out full-dimensional monitoring and intelligent decision making on a complex scene, the user adaptive interaction module can realize accurate identification and response to user requirements, and the multi-modal fusion module is used for making up for information limitation of a single modal, so that effective interaction and accurate identification can be realized on the whole.

The invention provides a structured target detection method, device and equipment based on a multi-modal language model, and relates to the technical field of target detection. The method comprises the following steps: inputting acquired image data and cue words into a multi-modal language model; the multi-modal language model comprises a visual encoder, a cross attention module and a decoder; performing feature extraction on the image data through a visual encoder; performing multi-modal interaction in a cross attention module based on the cue word and the data after feature extraction; wherein an Adapter module is inserted into the cross attention module so as to realize fusion of image and language information; performing low-rank fine tuning updating on the weights of the query vector and the value vector of the cross attention module, and keeping the weights of the other models frozen; and reasoning and outputting a plurality of target token group sequences at least comprising a target type and bounding box coordinates thereof through a decoder. According to the method, an additional target detection module is not needed, and complete structure information of multiple targets can be generated at a time through the improved multi-mode language model.

The invention relates to the technical field of man-machine interaction, in particular to a dialogue Agent interaction method based on multi-modal intention understanding, which comprises the following steps: S1, collecting multi-modal data in a user interaction process in real time, and calculating a time synchronization deviation value of each modal data source; s2, constructing a space-time fusion feature vector; s3, analyzing a dominant action instruction and a recessive behavior clue in the space-time fusion feature vector; s4, generating a multi-level intention analysis tree; s5, when the corrected confidence coefficient of any node in the intention analysis tree is lower than a set threshold value, activating a targeted sensor to complementarily collect data; and S6, analyzing a tree drive response decision according to the finally confirmed intention. According to the method, high-precision identification and response control of the dialogue Agent on the user intention in a complex scene are realized by constructing a multi-modal interaction method with space-time consistency fusion capability, an explicit and implicit intention analysis mechanism and an adaptive modal clarification strategy.

The invention provides an interaction method and device based on artificial intelligence, equipment, a medium, a program product and an intelligent agent, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models and the like, and can be applied to scenes such as AIGC content generation based on artificial intelligence. The method comprises the steps that a multi-modal problem is acquired, and the multi-modal problem comprises a text and an image; information matched with the text and the image is retrieved, and multi-source retrieval information is obtained; performing multi-task processing on the multi-source retrieval information based on a multi-modal problem by utilizing a multi-modal knowledge extraction large model to obtain retrieval enhancement information; and processing the retrieval enhancement information based on the multi-modal problem by using the multi-modal interaction large model to obtain reply content aiming at the multi-modal problem.

A computer-implemented method for controlling a robot collaborating with a human in an environment of the robot comprises: obtaining, by at least one sensor, multimodal information on the environment of the robot including information on a human acting in the environment; converting, by a first converter, the obtained multimodal information into text information; estimating, by an intent estimator, an intent of the human based on the text information; determining, by a state estimator, a current state of the environment including the human based on the text information; planning, by a behavior planner, based on the current state of the environment and the estimated intent of the human, a behavior of the robot including at least one multimodal interaction output for execution by the robot, and generating control information including text information on the at least one multimodal interaction output; converting, by a second translator, the generated text information into multimodal actuator control information; and controlling at least one actuator of the robot based on the multimodal actuator control information.

The invention relates to the technical field of intelligent equipment interaction, in particular to an intelligent equipment intention rapid insight and intelligent response method, which comprises the following steps: acquiring multi-modal interaction data input into intelligent equipment by a user, and generating a standardized intention input vector; performing hierarchical feature extraction on the standardized intention input vector, and outputting a fusion intention feature matrix; inputting the fused intention feature matrix into an intention classifier to obtain initial intention probability distribution data of the user; dynamically correcting the initial intention probability distribution data based on the historical intention record to generate a final intention judgment label; the optimal response action set is matched from the intention response strategy library on the basis of the final intention judgment label, the characteristic representation capacity is improved through multi-modal fusion, intention continuity understanding is enhanced through dynamic context perception, response self-adaptive optimization is achieved through reinforcement learning, and therefore the intention recognition accuracy can be improved in the noise environment, and the intention recognition accuracy is improved. And response delay is reduced.

The invention provides an AI role interaction method, interaction device and interaction equipment in a game platform, and relates to the technical field of game interaction.The method comprises the steps that a current game scene, game task information and game character information are obtained, and a first behavior parameter set of AI roles is generated; collecting current operation behavior data and voice interaction data of a user; performing intention recognition processing on the operation behavior data and the voice interaction data to generate a user behavior feature vector, and determining a relation link between the user behavior feature vector and at least one of game scene information, task information and character information to generate a dynamic interaction strategy; a second behavior parameter set is generated according to the dynamic interaction strategy, and multi-modal interaction content matched with the second behavior parameter set is rendered and generated; and displaying the multi-modal interaction content to the user terminal. According to the method, the interaction of the AI role can be more real and natural, and the game immersion and satisfaction of the user can be remarkably improved.

The invention belongs to the technical field of information management, and discloses an intelligent management system and method for large model technology government affairs, and the system comprises four layers of architectures of data collection and access, processing and storage, intelligent analysis and decision, and user interaction and application. Through federated learning and large model fusion governance data, a business process is optimized based on a knowledge graph and a large model, and intelligent service interaction is realized by applying multi-modal interaction and the large model. Experiments prove that the system and the method can significantly improve the business handling efficiency, enhance the decision accuracy, improve the user satisfaction, provide effective technical support for intelligent transformation of government affair management, and assist in improving the government service efficiency and decision scientificity.

The invention belongs to the field of intelligent information system management, and particularly discloses an intelligent voice telephone robot system and method, voice and image multi-mode data are collected through a microphone and a camera, and after preprocessing, voice, emotion and semantic features are fused through an improved Transform architecture to achieve accurate recognition of user intentions; a double-layer decision network based on reinforcement learning is combined with a dynamic reward function to generate an optimal response strategy; and realizing rapid task migration and parameter optimization of the model by adopting a meta-learning mechanism. The system also has the functions of adaptive noise robustness, multi-language interaction, user portrait dynamic updating, man-machine collaboration and the like. Compared with a traditional scheme, the intention recognition accuracy, the task completion rate and the scene adaptability are remarkably improved, the interaction experience is effectively improved, and the method can be widely applied to the fields of customer service, intelligent marketing and the like.

The invention discloses an elderly cognitive function adaptive training method and system based on multi-modal interaction feedback, and relates to the technical field of smart medical treatment. The method comprises the steps that basic information of a user is collected for initial cognitive ability evaluation, a user cognitive portrait is constructed according to an evaluation result, and an initial training task with the corresponding difficulty is allocated; collecting multi-modal interaction data in real time according to the initial training task; carrying out fusion analysis on the multi-modal interaction data by utilizing a machine learning model to obtain a quantized real-time state index; based on the real-time state index and the performance data of the current task, dynamically adjusting a subsequent training task through an adaptive decision rule engine; all-dimensional data of each training task is recorded, a visual cognitive competence development trend report is generated through longitudinal comparative analysis, and a machine learning model and a self-adaptive decision rule engine are continuously optimized and trained by utilizing accumulated user data to form an optimized training closed loop. The cognitive function training effect of the old people can be improved.

The invention discloses an AGI-based multi-mode holographic light field interaction system, method and device. The method comprises the following steps: S1, collecting and preprocessing multi-mode interaction data; s2, constructing a FusionNet network model, extracting visual, audio and tactile features, and fusing to generate multi-modal feature representation; s3, adopting a Laplace operator to carry out adaptive smoothing processing on the features, retaining significant feature information and removing high-frequency noise; s4, aligning the multi-modal features through a cross-modal mapping module, and generating three-dimensional light field data by using the light field reconstruction network; s5, decomposing the three-dimensional light field data, converting the three-dimensional light field data into a dynamic holographic image through light field display equipment, and performing layered rendering; and S6, adjusting network parameters according to real-time feedback, and optimizing holographic image details and rendering effects. According to the invention, multi-modal data fusion and real-time holographic image generation can be realized, and the intellectualization and immersion of interaction experience are significantly improved.

The invention relates to the technical field of intelligent wearable devices, and discloses a fusion multi-modal interaction method and device and a storage medium. The method comprises the following steps: acquiring an input signal, and performing gesture recognition processing on touch data to obtain a gesture track; comparing the gesture track with a preset gesture model to obtain a gesture recognition result; performing voice recognition processing on the voice data to obtain a voice recognition result; according to a preset motion feature model, performing identification operation processing on the motion data to obtain a motion identification result; and according to a preset cloud large model, carrying out fusion analysis on the gesture recognition result, the voice recognition result and the motion recognition result to obtain operation recognition data of multi-modal interaction. In the embodiment of the invention, the operation identification data is obtained through the method, and the equipment can perform multi-modal interaction, so that the overall interaction is more natural to establish a stable user connection feeling.

The invention relates to the technical field of man-machine interaction, and discloses a multi-modal task interaction assisting method and system and computer equipment, and the method comprises the steps: collecting multi-modal interaction data of a user executing a multi-dimensional interaction task; determining a user behavior feature vector based on the multi-modal interaction data, and constructing a task completion quantitative model; determining the weight of each operation step by adopting a dynamic weight distribution strategy based on the multi-dimensional interaction task completion degree quantification model, and dynamically adjusting a preset adaptive coefficient by adopting a preset adjustment mechanism based on the multi-modal interaction data and the weight of each operation step; generating a dynamic task guiding strategy by adopting a dynamic path planning algorithm based on the weight of each operation step and a dynamically adjusted preset adaptive coefficient; and executing the dynamic task guiding strategy to obtain a multi-modal execution result. According to the method, a complete closed-loop process is formed from data acquisition, analysis, decision making to execution, intelligent services can be provided for middle-aged and elderly users, and the user experience and the task execution efficiency are improved.

The embodiment of the invention provides a conference management system and method fusing a large language model and multi-agent collaboration. The system comprises a multi-modal central language large model and a multi-agent platform, and the multi-modal central language large model comprises a multi-modal interaction module, a full-duplex interaction interface, a task reasoning module, a task planning module and an agent scheduling module; the multi-agent platform comprises a data searching and issuing agent, a agenda management agent, a voice transcription agent and a knowledge base updating agent. According to the embodiment of the invention, the conference process is divided into a plurality of independent intelligent agents, and then the intelligent agents are scheduled and dynamically arranged in a unified manner by the central big model. Strategies can be flexibly adjusted according to real-time situations, and intelligence and high adaptability of the whole process are guaranteed. The conference process and the emotional state are monitored. And the central big model sends an intervention instruction. The active conference management can be realized, a targeted intervention means is provided, and the conference efficiency and the communication effect can be improved.

The invention discloses a multi-mode interactive virtual teaching method and system. The method comprises the following steps: acquiring multi-source data; extracting voice data acoustic features, and inputting the voice data acoustic features into a learning model to obtain a text triple; key terms are extracted from the text data, a traceable operation chain is generated, semantic analysis is carried out, and an operation scheme is output in combination with a knowledge graph; performing abnormal state recognition on the image data through a target detection model, positioning abnormal equipment in combination with a character recognition model, performing action mapping through gesture recognition and a spatial constraint rule, and outputting a corresponding instruction; fusing the three types of outputs to generate a scheduling event chain; and performing semantic analysis, evaluation and optimization on the scheduling event chain, interacting with personnel, updating operation suggestions and providing an operation analysis result. According to the method, manual rechecking requirements are reduced through multi-modal data collaboration, a new man-machine interaction database and a new man-machine interaction standard in the power industry can be formed through multi-modal interaction rules and event chain construction, and data intelligent driving is achieved while the training efficiency is improved.

The embodiment of the invention provides a multi-mode fusion man-machine interaction control method, system and device and a storage medium, and relates to the technical field of intelligent driving, and the method comprises the steps: obtaining vehicle driving data and driver state data; based on the vehicle driving data and the driver state data, calculating fusion weights of a visual mode, an auditory mode and a tactile mode to obtain a multi-mode fusion weight matrix; generating a multi-modal interaction signal according to the multi-modal fusion weight matrix, wherein the multi-modal interaction signal comprises a visual signal, an auditory signal and a tactile signal; and triggering corresponding visual warning, auditory warning and tactile warning according to the multi-mode interaction signal. In this way, multi-mode fusion is conducted on vision, hearing and touch, the output intensity of visual warning, hearing warning and touch warning is adjusted according to the weight of each mode of vision, hearing and touch, a driver is helped to make the most urgent operation at present according to the warning of each mode, and the driving safety of man-machine interaction in the driving process is improved.

The invention provides a response method, device and equipment for a customer service robot, and relates to the technical field of customer service robots, and the method comprises the steps: obtaining multi-mode request data inputted into a current session with the customer service robot by a user, and a conversation state of the current session; according to the request data of different modals and the modal features of the request data of the corresponding modals, constructing a multi-modal heterogeneous graph; fusing the different modal features based on the request data of the different modalities, the corresponding modal features, the dialogue states and the multi-modal heterogeneous graph to obtain multi-modal fusion features; inputting the multi-modal fusion feature into a pre-constructed user intention response model based on a causal relationship to obtain a response strategy of the customer service robot to the multi-modal request data; wherein the user intention response model is used for representing the causal relationship between the multi-modal interaction data input by the user and the response strategy of the customer service robot. The response accuracy of the customer service robot can be improved, and the user interaction experience is improved.

The invention discloses a digital cultural tourism management system based on multi-source data analysis, and the system comprises a multi-source heterogeneous data collection module which is used for collecting tourist behavior data, environment data, cultural resource data and third-party platform data in real time; the user demand intelligent analysis module is used for constructing a dynamic user portrait through the multi-modal interaction data and identifying dominant and implicit demands; the culture knowledge graph construction module is used for generating a reasonable multi-dimensional knowledge network based on culture resource attributes and historical data; the travel route dynamic generation module is used for generating a personalized touring route in combination with the user portrait, the real-time environment and the resource state; according to the digital cultural tourism management system based on multi-source data analysis disclosed by the invention, the utilization rate of cultural resources is greatly improved, and the decision response speed is increased; the cultural cognition depth of tourists is improved, and the residence time of the tourists is prolonged; the damage rate of high-sensitivity cultural relics is reduced; and the special group culture acquisition efficiency is improved in a breakthrough manner.

The invention discloses a multi-mode interactive smart home central control screen system and method based on a large model. The system comprises an instruction receiving module, an instruction processing module, a strategy generation and optimization module, an instruction sending and executing module, a state feedback and monitoring module and a learning and adapting module. According to the invention, multiple interaction modes such as touch control, voice, gesture and visual identification are integrated together to form a comprehensive multi-mode interaction system, the multi-mode interaction system is applied to the intelligent home central control screen system, a large model technology is adopted, intelligent services of the intelligent home central control screen system are realized, all intelligent devices at home can be managed in a unified manner, and the intelligent home central control screen system is more intelligent. The remote controller or the mobile phone APP of each device does not need to be operated respectively, more intelligent and personalized services are provided for the intelligent home central control screen system, and efficient, intelligent and personalized control and management of the intelligent home devices are achieved.

The application discloses a multimodal interaction method, device, controller, system, automobile and storage medium. The method comprises the following steps: determining a current interaction dialogue according to a current interaction voice at an interaction time; determining a current scene image and current scene data corresponding to a current interaction interface corresponding to the interaction time; adopting a multimodal recognition model to perform multimodal recognition on the current interaction dialogue, the current scene image and the current scene data, and determining a target control instruction; and executing the target control instruction to complete a human-computer interaction operation. The method can guarantee the output efficiency and accuracy of the target control instruction, reduce the complexity of customizing templates, rules and associations and other complex control logics in the development process, and improve the adaptability and generalization ability of voice interaction.

The invention discloses a robot multi-mode interaction control method and a related device, and the method comprises the steps that a robot controller obtains user interaction information, and the user interaction information comprises user voice information and / or user action information; analyzing the user interaction information, and determining a shopping guide service type and a target vehicle part; acquiring first component information of the target vehicle component from a preset vehicle knowledge base according to the shopping guide service type; according to the shopping guide service type and the first component information, a target action sequence is determined from a preset action library, the target action sequence comprises joint actions and voice actions, the voice actions are used for outputting voice shopping guide information, and the voice shopping guide information is determined according to the first component information; and sending a first control instruction to the target robot, wherein the first control instruction is used for indicating the target robot to execute the target action sequence. According to the invention, the functionality and intelligence of the humanoid robot as an intelligent shopping guide robot can be improved.