4623 results about "Human–robot interaction" patented technology

The invention belongs to the technical field of industrial equipment monitoring, and discloses an industrial production equipment monitoring and early warning system based on the Internet of Things and edge intelligence, and the system collects multi-modal data in real time through a distributed sensor network, and carries out the real-time processing and feature extraction through the edge intelligence technology. Data fusion and optimization are carried out through a cloud deep learning model, a self-adaptive early warning and decision module dynamically adjusts a threshold value and optimizes an operation strategy, an environmental adaptability optimization module ensures the stability of equipment under extreme conditions, and an equipment health management and collaborative maintenance module realizes cross-equipment collaborative maintenance. The remote monitoring and interaction module provides man-machine interaction type control, and the cross-device collaborative learning module improves the model performance through federal learning. According to the system, the accuracy of equipment monitoring, the timeliness of early warning and the intelligent level of maintenance are remarkably improved.

The invention discloses a mechanical equipment state monitoring method and system based on multiple sensors, and the method comprises the five core steps: multi-modal data collection and preprocessing, dynamic feature fusion, adaptive threshold diagnosis, digital twin fault tracing and predictive maintenance decision. All-domain coverage of equipment is realized through a three-layer sensor network architecture, the problems of data synchronization and interference resistance are solved by utilizing a temperature and vibration integrated sensor, deep fusion and anomaly detection of multi-source data are realized in combination with an attention mechanism, a Gaussian mixture model, a three-dimensional convolutional neural network and the like, and finally a precise maintenance strategy is generated through digital twinning and reinforcement learning. The multi-sensor-based mechanical equipment state monitoring system comprises a sensor network layer, an edge computing layer, a cloud platform layer and a man-machine interaction layer, supports federated learning to protect data privacy, improves real-time diagnosis capability through edge-cloud collaboration, and enhances a reality interface to realize intelligent operation and maintenance interaction.

The invention provides an intelligent mechanical arm control system scheme based on multi-model cooperation, and belongs to the field of automation and intelligent control. The system is divided into four parts of man-machine interaction, detection and perception, task decoupling planning and program generation, and robot execution operation. Man-machine interaction is carried out, a user inputs an instruction in a text or voice format, and instruction processing is carried out through voice recognition and a natural language translation model; detecting and sensing, and identifying and positioning a target captured by the camera by using a multi-modal large model; task decoupling planning and program generation: analyzing the user instruction and the perception information into an executable program by applying a task planning model and a program translation model; and checking and correcting the program by using the program checking model, and driving the mechanical arm to complete a task. According to the system scheme, variable environments can be dealt with, complex language instructions can be understood, intelligent control over the mechanical arm is achieved, and the multi-task execution capacity and the intelligent level of the mechanical arm are remarkably improved.

The invention relates to the technical field of man-machine interaction, in particular to a robot behavior mode dynamic adjustment method based on multi-mode perception, which comprises the following steps: S1, collecting a visual image, a voice signal and an environment parameter of a target scene in real time; s2, extracting each modal feature; s3, dynamically allocating weights, and generating a fusion feature vector; s4, identifying a current scene type and a user attribute; s5, matching a corresponding interaction mode from a preset strategy library based on the scene type and the user attribute identified in the step S4; and S6, executing the matched interaction mode in the S5, and updating the weight distribution rule according to the feedback data. According to the method, vision, voice and environment characteristics are fused through a multi-mode perception technology, the behavior mode of the robot is dynamically adjusted based on a self-adaptive weighted fusion algorithm, and the interaction strategy is optimized in combination with user feedback, so that the interaction accuracy and the intelligent level of the robot under different scenes and user attributes are improved.

The invention discloses a robot anthropomorphic interaction method based on multi-modal emotion recognition and customized portrait generation. The method comprises the following steps: S1, dynamically fusing multi-modal emotions; the method comprises the following steps: S1, synchronously acquiring voice, visual and text signals through a multi-source heterogeneous sensor, capturing a user voice stream by a high-fidelity microphone array, and extracting acoustic characteristics such as intonation and speed, S2, performing cross-modal reasoning; s3, synchronously generating contents; step S4: style migration; step S5, anthropomorphic voice and expression generation; according to the method, man-machine interaction emotion is analyzed and generated by utilizing a large language model and multi-modal information fusion, the singleness of interaction emotion and the deficiency of emotional sharing ability are avoided, a strong emotion interaction characteristic is achieved, the image of the robot is obtained through a generative technology and can be migrated to any image, the limitation that a specific image is independently made is broken through, and the interaction effect of the robot is improved. The advantage that one robot can be suitable for different scenes is achieved.

The invention discloses a multi-screen voice interaction system and method applied in an automobile cabin, and relates to the technical field of vehicle-mounted intelligent interaction, and the system comprises a voice collection unit which is used for carrying out sound source positioning and collection, carrying out the noise reduction processing of a collected voice signal, extracting features from the processed voice signal, and generating a voice feature vector; the multi-modal sensing unit is used for collecting behavior information and physiological state data of a driver and passengers through multi-sensor fusion, so as to extract multi-modal information; the man-machine interaction unit is used for carrying out space-time modeling on the multi-modal information to generate a scene state vector; and the cooperative scheduling unit is used for realizing multi-screen intelligent distribution and cooperative control. According to the invention, voice instructions of a driver and passengers can be accurately identified, the safety and experience of the driver are improved, changes in different driving environments are dynamically adapted, the operation efficiency and user experience of a vehicle-mounted system are improved, and the intelligence and adaptability of the system are improved.

The invention relates to the technical field of airport intelligent management, in particular to an airport intelligent operating system, which comprises a multi-modal data acquisition layer for acquiring multi-source data such as flights, passengers and luggage in real time through a heterogeneous sensor network; the intelligent analysis layer processes data by using deep learning and a unique algorithm, and generates information such as flight delay prediction and the like; the collaborative scheduling layer realizes cross-domain dynamic optimization of flight-boarding gate-luggage system-ground service based on digital twinning and multi-agent reinforcement learning; the man-machine interaction layer is equipped with AR navigation and voice interaction equipment and provides personalized services; in addition, a block chain evidence storage module is arranged to ensure that key operation records are safe and traceable, and a federal learning framework realizes multi-airport knowledge sharing and privacy protection; through multi-module cooperation, the airport operation efficiency is improved, the safety is enhanced, the passenger service experience is optimized, many problems faced by traditional airport operation are effectively solved, and the system has remarkable innovativeness and practicability.

The invention relates to the technical field of artificial intelligence rescue, and discloses an unmanned aerial vehicle and robot dog rescue emergency system and method, and the system comprises an unmanned aerial vehicle cluster module which is used for air data collection, communication guarantee and emergency material distribution; the robot dog cluster module is used for ground environment detection, sample collection and special task execution; the communication cooperation module is used for ensuring information intercommunication and task cooperation between air-ground equipment; the central control module provides task instructions for the unmanned aerial vehicle cluster and the robot dog cluster; the human-computer interaction interface module is used for realizing effective communication between rescue workers and the system; the modules work cooperatively, multi-source data acquisition, three-dimensional scene reconstruction, semantic segmentation, risk assessment, fire spreading prediction and decision support information generation of a disaster site are realized, and a reliable basis is provided for rescue command decision.

The invention relates to the technical field of tunnel engineering construction, and discloses an intelligent material transportation and safety monitoring system and method for shield construction, and the system comprises a visual perception unit, a sensor network module, an AI analysis center module, a safety decision module and a human-computer interaction interface. According to the invention, data acquisition is carried out through the visual perception unit and the sensor network module, multi-target detection operation is carried out on image frames through the AI analysis center module after target identification and track prediction, target types, space coordinates, contour boundaries and confidence coefficients are identified and extracted, and safety judgment and early warning output are carried out. According to the invention, by integrating the multi-view camera equipment and the UWB, GNSS and other sensors and adopting a deep learning target detection algorithm, high-precision identification and continuous tracking can be carried out on construction site personnel, equipment, segments and other key objects, and accurate input is provided for subsequent risk analysis.

The invention relates to the technical field of unmanned aerial vehicle cluster control, and particularly discloses an intelligent cooperative flight path planning method and system for an unmanned aerial vehicle cluster system, and the method comprises the steps: firstly constructing a dynamic environment perception model, collecting data through all unmanned aerial vehicle sensors, and carrying out the preprocessing, attention feature extraction and federal learning fusion, and generating a global environment situation map; planning and screening a candidate track set by adopting a particle swarm-genetic hybrid optimization algorithm for adaptive weight adjustment on the basis of the image; a global optimal track consensus is achieved through an improved consensus algorithm and conflict resolution through a distributed collaborative negotiation mechanism and no human-computer interaction evaluation indexes; and finally, monitoring the environment in real time during execution, triggering dynamic re-planning when the environment is abnormal, and ensuring track adaptation through multi-level threshold and incremental planning. According to the method, the unmanned aerial vehicle cluster can quickly respond to the environment change and adjust the flight path, so that the task execution efficiency and success rate of the unmanned aerial vehicle cluster in the complex dynamic environment are improved.

The invention relates to the technical field of medical rehabilitation, and discloses a lower limb weight-bearing gait rehabilitation training system which comprises a data acquisition module, a data processing and analysis module, a patient individualized modeling module, an intelligent decision and control module, a rehabilitation execution module and a man-machine interaction and medical information interface module which are in communication connection through a network. The data acquisition module is used for acquiring multi-modal data of a patient in real time, and the multi-modal data comprises static sign data, dynamic physiological parameters, kinematics and dynamics parameters and non-motion physiological and psychological state data; and the data processing and analysis module is used for carrying out preprocessing, feature extraction and deep analysis on the original data, and outputting a structured patient individualized feature vector and an evaluation result. According to the invention, a patient three-dimensional skeletal muscle digital twinborn model is constructed through the patient individualized modeling module, and in combination with a continuous learning intelligent model library, body sign differences of different patients can be accurately adapted.

The invention relates to the field of man-machine interaction training, virtual reality and intelligent evaluation, and particularly discloses a psychological exercise ability training method and system based on mixed reality and behavior analysis. The method comprises the steps that a multi-scene psychological training environment is constructed and presented in mixed reality equipment, action behavior data, eye movement path data and operation response time data of a pilot in the training process are collected, multi-modal original behavior characteristics are constructed, and then a psychological exercise ability label set is constructed; and realizing capability-task modeling through matching of a capability label and a task structure, generating a personalized training path and a capability difference atlas based on difference analysis with a preset capability standard, adjusting task parameter configuration data, dynamically updating a training scene, generating a capability evolution trajectory, and outputting a training analysis result. In this way, accurate ability modeling, task dynamic adaptation and visual tracking of the training effect can be achieved, and the individuation and intelligence level of psychological exercise ability training is remarkably improved.

The invention provides an intelligent alarm system and method for monitoring power plant equipment, and the method comprises the steps: collecting the temperature, pressure and vibration data of the equipment through a sensor, and generating a holographic image through the scanning of a laser radar, so as to complete the data collection and environment modeling. A space-time atlas is constructed and analyzed by means of a space-time network, faults are analyzed and positioned by means of topological data, and data fusion and deep analysis are achieved. And constructing a Bayesian network by adopting intelligent learning and adaptive optimization of evolutionary strategy optimization and memory enhanced neural network prediction, and carrying out fault diagnosis and causal inference in combination with anti-factual reasoning. And the group intelligent collaborative diagnosis and emotion calculation aided decision is used for intelligent decision and collaborative operation, and finally, an operator is assisted to work based on a man-machine interaction function. The system and the method can comprehensively and accurately monitor equipment, efficiently diagnose faults and assist in decision making, and improve operation safety and reliability of a power plant.

The invention provides an interaction method, system and device and a medium, and relates to the technical field of man-machine interaction, and the method comprises the steps: obtaining an interaction process of a main control agent and a target user in a current round of interaction process according to the intention information and state information of the target user in the current round of interaction process; according to the interaction process, controlling an execution agent to perform response operation on current input information to obtain a response operation result of the current input information; the current input information is information input by the target user in a current round of interaction process; and outputting response information corresponding to the current input information according to the response operation result. According to the method, the interaction process and the dialogue content are flexibly adjusted according to the real-time intention and the real-time state of the user, and the fluency, naturalness and adaptability of interaction, the efficiency and real-time performance of interaction processing and the user experience feeling are remarkably improved.

The invention provides an intelligent interactive question and answer method and system based on multi-modal large model intention recognition, and relates to the technical field of intelligent question and answer, and the method comprises the steps of feature extraction, cross-modal semantic mapping, intention matching, key information extraction and response generation, achieves the unified processing of multi-modal information, can accurately recognize the intention of a user, and improves the user experience. And the association network is dynamically constructed to extract key information, so that the accuracy and semantic coherence of intelligent questions and answers are improved, and the man-machine interaction experience is enhanced.

The invention discloses a robot path planning perception method and system based on a large language model. The method comprises the following steps: acquiring an initial data set; selecting an SLAM algorithm to process the initial data set; performing rasterization processing on the first map; performing semantic annotation processing on the second map; performing path planning according to the large language model to generate a path planning scheme; performing conversion according to the path planning scheme to obtain execution information of the robot; target sensing information and environment information of the robot in the moving process are obtained, and a motion planning adjustment scheme of the robot is generated through a large language model according to the target sensing information and the environment information; and a man-machine interaction module is constructed to realize information interaction between the user and the robot. According to the embodiment of the invention, a more reliable and smoother path can be generated, the navigation and perception capabilities of the robot in a complex environment are improved, and the method can be widely applied to the technical field of computers.

The invention discloses a robot control method based on a multi-modal large model, solves the problems of generalization and adaptability of an existing robot control method based on a large model, and belongs to the field of robot control. The method comprises the steps that a multi-modal planning large model deconstructs a subtask sequence according to the requirements of a user and historical behavior records, and an instruction generation model generates a corresponding control instruction according to the deconstructed subtask sequence and stores the control instruction into a pointer drawing queue; the planning pointer manages the control instructions in the planning pointer queue, and the normally executed control instructions are moved into a historical behavior record; during training, planning control is judged through man-machine interaction, planning content with a good execution effect is stored in an excellent data set, and the excellent data set is utilized to perform fine tuning optimization on the multi-modal planning model and the instruction generation model; and controlling the robot by using the two optimized models and the planning pointer.

The invention discloses an industrial robot intelligent control system which comprises a master control terminal, the master control terminal is provided with an industrial-grade multi-core processor and is integrated with a motion control subsystem and an environment sensing subsystem, the motion control subsystem is connected with the master control terminal through a first communication interface, and the environment sensing subsystem is connected with the master control terminal through a second communication interface. According to the motion control subsystem, a distributed axis controller array and reinforcement learning strategy switching algorithm is adopted, so that the trajectory tracking precision and the energy consumption efficiency are improved; and the environment sensing subsystem realizes real-time anomaly detection through multi-mode sensor fusion and an LSTM-GAN fault prediction model. The system supports containerized deployment and edge computing collaboration and integrates AR human-computer interaction, and the adaptive capacity and safety of the industrial robot are remarkably improved.

The invention discloses a lower limb exoskeleton gait track prediction method based on an LSTM-KAN fusion model, and the method comprises the steps: collecting human motion data through a sensor assembly, and carrying out the filtering, missing value processing and normalization of the human motion data; then constructing an overall architecture of a prediction model based on an LSTM-KAN network, optimizing parameters of the prediction model by using a particle swarm optimization (PSO) algorithm, extracting key features in the preprocessed data as a training data set, and inputting the training data set into the prediction model for training; and finally, collecting current human body motion data, pre-processing the current human body motion data, inputting the pre-processed current human body motion data into the trained prediction model, and outputting future human body gaits and tracks by the prediction model. And the controller takes a future gait track generated by the prediction model as a reference track to generate a driving signal and sends the driving signal to the actuator so as to realize accurate control of the actuator. By adopting the gyroscope sensor, the acceleration sensor and the pressure sensor for gait estimation, exoskeleton motion control can be effectively improved, and man-machine interaction experience is effectively improved.

The invention relates to the technical field of intelligent man-machine interaction, and discloses a customized virtual pet interactive simulation method, which comprises the following steps: S1, perceiving the emotional state of a user, S2, analyzing the behavior and emotion of the user, S3, establishing a virtual pet behavior model, and S4, establishing a virtual pet interaction model. The invention provides a customized virtual pet interactive simulation system, which comprises a sensing module, a decision-making module, an execution module and a feedback module. The method comprises the following steps: S1, selecting a virtual pet behavior, S2, selecting an optimal behavior, S5, carrying out deep reinforcement learning optimization, S6, executing the virtual pet behavior, and S7, feeding back and adjusting a behavior strategy. The invention also provides the customized virtual pet interactive simulation system, which comprises a sensing module, a decision-making module, an execution module and a feedback module. Through long-term emotional state tracking, personalized behavior pattern recognition and long-term reward correction of reinforcement learning, the virtual pet can continuously optimize a behavior strategy, dynamically adapt to long-term emotional changes of a user, and improve the personalized matching degree and long-term stability of interaction.

The invention discloses a wireless high-density electroencephalogram and myoelectricity hybrid synchronous acquisition and monitoring system, and belongs to the field of human-computer interaction. The wireless high-density electroencephalogram and myoelectricity mixed synchronous collecting and monitoring system is composed of three parts, namely a high-density electroencephalogram and surface myoelectricity signal collecting system, a base station system and an upper computer system. The high-density electroencephalogram and surface electromyogram signal acquisition system is responsible for acquisition and primary processing of signals, and the base station system performs data interaction with the high-density electroencephalogram and surface electromyogram signal acquisition system in a wireless transmission mode and transmits the processed signals to the upper computer system for further analysis and display. According to the invention, a wireless and distributed architecture and a high-channel-number data acquisition scheme are integrated, the problems of efficient acquisition and synchronous processing of electroencephalogram and myoelectricity mixed signals are emphatically solved, and the application range in the fields of brain-computer interfaces, virtual reality, neural rehabilitation, cognitive training and the like is widened.

The embodiment of the invention provides an excavator handle control system for remote operation, a handle control method based on a Hall sensor is adopted, non-contact rocker data sensing is achieved, the influence of contact and movement abrasion on the service life of a handle is reduced, and the excavator handle control system has the advantages of being high in sensitivity and good in performance; the handle mechanism is more flexible to operate, and the service life of the handle is prolonged; a two-hand cooperative control mode is adopted, and two three-degree-of-freedom handles are used for simulating the driving environment of a real excavator, so that a good remote control effect is achieved, and the feeling of presence of an operator in the remote operation process is improved; the operation habits of a driver are fully considered, the walking and working of the excavator are respectively controlled through combination of different degrees of freedom of the left hand handle and the right hand handle, an operating lever of the real excavator is simulated in the aspects of structure, motion degree of freedom and working range, and the man-machine interaction environment of excavator driving is restored.

The invention relates to the technical field of electric power operation and maintenance management, in particular to a transformer area intelligent operation and maintenance device and method based on a machine learning expert model, and the device comprises a logistic regression machine learning module and a robot process automation module. Real-time perception and data analysis of a business scene are realized by means of artificial intelligence, total data information of a master station measurement domain and an operation and maintenance work task is dynamically collected, incremental data is continuously utilized for data analysis and rule mining, a machine learning expert model is introduced, and the operation and maintenance efficiency is improved. A transformer area intelligent operation and maintenance system integrating human-computer interaction such as operation and maintenance index intelligent monitoring and analysis, intelligent abnormity diagnosis, intelligent operation list generation, intelligent dispatching, intelligent path planning and field operation guidance is constructed, service data is analyzed in real time, meanwhile, a robot process automation technology is integrated, automatic operation of a part of processes is achieved, and operation efficiency is improved. And support is provided for on-site acquisition operation and maintenance work.

The invention relates to the technical field of mine industry management, in particular to an industrial management system with an industrial configuration picture presentation function, which comprises a heterogeneous data acquisition module which is composed of a plurality of sensors and realizes data fusion through a multi-protocol converter; the three-dimensional dynamic configuration engine covers the geological model construction unit, the real-time data mapping unit and the visual rendering unit; the intelligent scheduling decision module is provided with a transportation path optimization sub-module, a resource allocation sub-module and an emergency response sub-module; and a man-machine interaction terminal. An edge computing node of the heterogeneous data acquisition module adopts an improved LSTM missing data interpolation model; the transportation path optimization sub-module uses an improved ant colony algorithm, the resource allocation sub-module uses an improved multi-objective optimization model, and the emergency response sub-module uses an improved rule reasoning algorithm. The problems of data fusion, path optimization, resource allocation, emergency response, visualization and the like in mine industry management are effectively solved, and the efficiency, the safety and the intelligent level of mine management are improved.

The invention relates to the technical field of computer vision, and discloses a high-speed human motion trail motion mode recognition system. Through multi-modal sensor fusion and an adaptive space-time attention network (ASTAN), the problems that a traditional optical system depends on mark points and IMU accumulative errors exist are effectively solved, and the high-speed movement track reconstruction precision and the environmental adaptability are remarkably improved; self-supervised data enhancement and dynamic model compression technologies are combined, so that the dependence on labeled data is greatly reduced, lightweight edge deployment is realized, and the real-time requirement is met; the system can synchronously output multi-mode feedback instructions, supports multi-scene application such as sports action correction, medical rehabilitation evaluation, security and protection anomaly detection and man-machine interaction control, and solves the core pain points of insufficient precision, real-time performance, robustness and generalization ability in the prior art. And an efficient, reliable and low-cost comprehensive solution is provided for high-speed motion analysis.

The invention provides an intelligent spraying composite robot system based on a large language model and a method thereof, and relates to the field of intelligent robots and automatic spraying, the intelligent spraying composite robot system comprises a chassis, a mechanical arm, a spraying gun, a camera, an instruction receiver and a controller; a mechanical arm and an instruction receiver are carried on the chassis; spray guns and cameras are mounted on the mechanical arms; the instruction receiver receives a natural language voice instruction, analyzes semantics through a large language model, and generates an action instruction; the controller comprises a motion control module, a three-dimensional point cloud data acquisition module, a feature point extraction module, a track planning module and a spraying operation module; the method improves the man-machine interaction, reduces the use threshold of the robot, is easy to operate, gives play to the spraying advantages of the composite robot, does not need to program in mainstream offline simulation software, can generate a spraying path with adjustable and controllable parameters only by collecting the point cloud template of the workpiece and selecting the feature points, and can be widely applied to furniture, furniture and the like. And spraying operation in the fields of automobiles, aerospace and the like.

The invention discloses a depression risk screening optimization method based on large and small model linkage. The method comprises the following steps: selecting depression related indexes, obtaining interviewee questionnaire data, and preprocessing the data to obtain a scale source database; a depression risk prediction model is constructed, and PHQ-9 measurement results are compared for model training and verification; training a dialogue strategy module of a semantic analysis enhanced fine-tuning training large language model, constructing answer mapping through dynamic question generation and dialogue flow control, and converting a natural language of a user into standardized data required by a small model; training a man-machine interaction reinforcement learning model, generating a depression risk screening result based on a small model, inviting a user to carry out recognition degree evaluation, and dividing feedback into two types of recognition and question; for different feedbacks, strengthening or correcting the current interaction strategy and prediction logic, and storing the audited data as high-quality data to a training database by the system for subsequent large model fine tuning; and outputting a result and performing result interpretation and suggestion by using the semantic analysis reinforced fine-tuning large language model. Hierarchical early screening of depression risks is carried out based on large and small model linkage.

The invention discloses an interactive intention understanding and rapid learning system based on multi-mode information fusion, which aims at semantic understanding of an operation instruction of a robot for picking and placing an article, constructs a key information extraction framework based on a large language model in a task semantic understanding module, and comprises multiple key position and action text information extraction, and multi-mode interactive fusion is carried out on the extracted information, vision and a touch screen, so that correct understanding of the instruction intention is realized. The key information extraction architecture based on the big language model is based on a BERT big language model, and a language processing model is trained through the BERT big language model; the language processing model comprises a long-sequence multi-task instruction and a human instruction data set of mood, scene, positive-sequence and inverse-sequence request instructions, and realizes input of long-sequence multi-task instruction statements and output of a key action sequence; according to the system, efficient and accurate human intention understanding can be realized through a voice, touch screen and vision multi-mode human-computer interaction technology, so that the robot can quickly learn new articles.

The invention relates to the technical field of medical rehabilitation, and particularly discloses an anorectal postoperative rehabilitation nursing evaluation method and system based on a sensor. According to the scheme, the system comprises a multi-source physiological information sensing module, a data preprocessing and feature extraction module, a multi-modal data fusion analysis module, a dynamic risk assessment and early warning module and a man-machine interaction and decision support module, multi-dimensional physiological signals are continuously collected, a rehabilitation state index is generated through fusion, the complication risk is assessed in real time, and the rehabilitation effect is improved. Early warning and accurate nursing decision support are realized, and comprehensiveness and timeliness of rehabilitation evaluation are improved.

The invention provides a robot control method, computer equipment and a computer readable storage medium, and the method comprises the steps: obtaining multi-modal data of a user, the multi-modal data comprising attitude data, sound data and force control data; performing feature extraction and fusion processing on the multi-modal data to obtain multi-modal fusion features; according to the environment information of the area where the robot is located and historical interaction data of the robot, context information of man-machine interaction is determined; and according to the multi-modal fusion features and the context information, a response strategy is generated, and the robot is controlled to move according to the response strategy, so that the naturalness and accuracy of man-machine interaction are improved.