718 results about "Service robot" patented technology

The invention provides an IMU (inertial measurement unit)-assisted depth SLAM (simultaneous localization and mapping) method and system fusing language-vision multi-mode perception, and the system comprises functional modules such as initial calibration and semantic map initialization, pre-integration prediction and key frame judgment, dense point cloud reconstruction and relative pose estimation, semantic embedding extraction, semantic guidance loopback detection and semantic three-dimensional map incremental updating. IMU motion priori, depth geometric constraint and language model semantic factors are subjected to combined modeling through a graph optimization framework, and high-precision positioning and labeled map construction in a complex dynamic environment are achieved. Compared with the prior art which only depends on geometric or inertial information, the method has the advantages that the loop-back mismatching rate is reduced, the closed-loop convergence efficiency and the long-time relocation robustness are improved, and a semantic interface is provided for upper-layer tasks such as natural language navigation and target retrieval. The method can be widely applied to the fields of service robots, security inspection, intelligent driving, post-disaster search and rescue and the like.

The invention relates to the technical field of intelligent customer service control, and discloses a federated learning driven customer service robot cooperative control method and system. The method comprises the following steps: deploying a local intention recognition model at a plurality of nodes, collecting a user dialogue stream, extracting a semantic behavior track fragment, and generating a behavior feature vector set containing a time sequence and context association; the federal cooperative controller performs periodic aggregation, constructs a cross-node feature alignment mapping table based on trajectory similarity, and generates a global behavior feature distribution map; calculating node feature offset, screening high-contribution-degree nodes in combination with a sparse activation threshold, and allocating aggregation tasks; a knowledge distillation compression model is used at the high-contribution-degree nodes, weight updating parameters are extracted, compensation coefficients are added, and an encrypted updating package is generated; and the federal cooperative controller carries out heterogeneous fusion on the encrypted packet, reconstructs a global intention decision tree and carries out segmentation and distribution, so that efficient cooperation and optimization are realized, and privacy protection and service adaptability are considered.

The invention discloses a semantic fingerprint adaptive training method for a teaching service robot, and relates to the technical field of education neural network real-time training. Comprising six steps of course version semantic fingerprint injection, real-time drift detection and bucket division positioning, small sample correction and high-level weight patching, hierarchical control incremental learning scheduling, shadow reasoning consistency optimization and learning asset registration and cycle verification and tracking. Measuring drifting in real time by using an information entropy self-adaptive window and multi-scale divergence, generating a lightweight weight patch by small sample contrast learning, and performing online loading; shadow channel parallel reasoning is combined with a grading heat exchange superior weight, four-dimensional learning asset tensor is written into a registry through double-clock witness and chain commitment, random sampling verification and singular value performance verification ensure that assets are consistent with robot online examples, and the comprehensive effects of source traceability, risk self-sensing, model self-repairing and compliance full-chain trace reserving are achieved.

The invention discloses a multi-modal self-adaptive child voice interaction method for an educational service robot, relates to the technical field of voice signal processing, and provides a multi-modal self-adaptive child voice interaction method for an educational service robot through multi-array microphone dynamic acquisition and beam forming, self-adaptive noise suppression, multi-speaker separation and speaker identification, and child exclusive model training and online fine tuning. Technical means such as context semantic correction and multi-modal fusion are adopted, so that the target voice focusing and recognition accuracy is greatly improved; multi-person voice tracks are distinguished through an end-to-end network, and rapid distinguishing of a multi-source voice scene is realized in combination with model optimization of child range features; meanwhile, correct vocabularies and pronunciation information are traced back to a speaker library and a child sound model by using real-time semantic correction and interactive feedback, and multi-mode cooperation of a camera and a positioning sensor is used for assisting, so that beams are quickly adjusted and interference is suppressed when a child moves at will and environmental noise suddenly increases, and the sound quality of the child is improved. And natural and smooth voice interaction experience is provided in class and family scenes.

The invention relates to an indoor robot navigation method based on multi-modal feature fusion. The method comprises the following steps: constructing a semantic map based on visual observation environment information; the method comprises the following steps: acquiring an RGB image of an indoor scene object, converting the RGB image into point cloud data, and preprocessing the RGB image and the point cloud data; image multilayer semantic features and point cloud features in the RGB image and the point cloud data are extracted respectively, and initial fusion is carried out; performing weighted fusion on the multi-layer semantic features and the point cloud features of the image by adopting space-channel-cross-modal multi-attention dynamic cooperation; and predicting a long-term target in a map space from top to bottom based on the fused feature map and the semantic map, and performing navigation path planning based on the current position and the long-term target. Under low-cost hardware configuration, the robustness of environment perception, the real-time performance of decision response and the usability of system integration in a dynamic complex environment are comprehensively improved, and the method is particularly suitable for application scenes such as indoor service robots.

The invention discloses a customer service robot system integrating knowledge base question and answer retrieval and work order processing, which comprises a natural language understanding and generating module, a knowledge retrieval module, a business system integration module, a dialogue management and decision module and a cue word management module, and relates to the technical field of artificial intelligence. Through cooperation of a large language model and a dialogue management module, the system realizes multiple rounds of deep understanding of a natural language, and can actively guide a user to clarify a demand, such as gradually inquiring an equipment model and fault details during repair, memorizing a dialogue context and avoiding information omission. Compared with traditional regular customer service, the method has the advantages that the accuracy of understanding complex problems is remarkably improved, open dialogue scenes such as technical consultation and fault diagnosis are supported, user interaction experience is closer to manual service seats, and the problems that multi-round dialogue understanding is limited, progress data cannot be obtained in real time, multiple knowledge bases are fused, and the automation degree of the business process is low are solved.

The invention relates to the technical field of robots, in particular to a user intention alignment robot task planning method based on a large language model, and the method comprises the steps: firstly receiving human task voice, carrying out the textualization processing of the voice, carrying out the human instruction ambiguity elimination through combining with a robot task environment, and generating a feasible instruction candidate set; and then calling the maintained user feature memory module by the large language model, and further screening the candidate instructions in combination with human user preferences. And finally, the large language model decomposes and plans the reserved task instruction, guides the robot to execute the task, collects human feedback information in the task process, and is used for optimizing and updating the user feature memory module and guaranteeing the task planning continuity of the large language model driven by reflection feedback. According to the method, when the service robot based on the large language model carries out task planning, the instruction intention of the human user is analyzed and aligned more accurately, and customized services are better provided for the human user.

The invention provides a machine grabbing learning method fusing shape features, contact modeling and physical constraints, and the machine grabbing learning method is a robot grabbing posture learning method capable of differentiating. According to the method, a three-dimensional point cloud serves as input, a deep neural network based on sparse voxel convolution is adopted to extract multi-scale shape features, and potential grabbing points and attitude parameters thereof are predicted. The differential contact modeling is realized by constructing a local area of direction perception and extracting geometric features reflecting the contact relationship between the mechanical claw and the object. Furthermore, the collision probability of the grabbing posture is estimated based on an implicit neural network, and a differentiable collision detection module is constructed. Meanwhile, physical constraints such as friction closing, surface alignment and geometric symmetry are introduced to serve as regular terms to jointly optimize grabbing scores, collision risks and stability; according to the method, end-to-end training is supported, efficient, robust and deployable grabbing posture estimation is achieved, and the method is suitable for various automatic grabbing tasks such as industrial robots and service robots.

The invention relates to the technical field of multi-modal posture recognition, and relates to a multi-modal posture recognition system based on Transform fusion, which fully excavates the complementary advantages of visual information in spatial detail representation and inertial information in time sequence dynamic capture through space-time alignment of multi-modal data and deep fusion based on an attention mechanism. The accuracy and the stability of attitude estimation under the conditions of visual occlusion, rapid movement and complex illumination are obviously improved; posture optimization is carried out by introducing physical constraints such as bone length constancy and joint movement range limitation, and time domain smoothing and contact state correction are applied, so that the precision of the generated three-dimensional posture sequence is ensured, a solid technical foundation is laid for improving the naturalness, safety and intelligent level of man-machine interaction, and the method is suitable for popularization and application. And meanwhile, reliable application and deep development of the intelligent robot in key fields of service robots, virtual reality, remote cooperation and the like are powerfully promoted.

The invention discloses a robot visual identification decision control method based on deep learning, and relates to the technical field of control systems. Comprising the steps of constructing a thinking decision tree according to an input instruction and a real-time visual image, and obtaining a linear relationship between a middle-layer leaf node as a main body and a top-layer leaf node for realizing association between a current visual feature and a task intention. According to the method, deep fusion and unified representation of multi-source heterogeneous data are realized by constructing a multi-layer thinking decision tree structure and combining multi-modal feature fusion and a deep learning engine, and text instructions, visual images and other different modal information are effectively integrated by constructing a fusion search engine and a semantic alignment mechanism, so that the multi-source heterogeneous data fusion and unified representation are realized. Cooperative processing of accurate analysis of task intentions and environment perception is realized, and real-time response and decision accuracy of the service robot to complex tasks are improved.

The invention relates to the technical field of service robot navigation and motion control, in particular to a service robot dynamic obstacle pre-judgment obstacle avoidance system, which comprises an environment sensing unit for outputting sensing data; the target state estimation unit outputs a target state sequence; the future evolution prediction unit is used for outputting a target future evolution result; the risk quantification unit outputs the current collision risk degree; operating the strategy management unit, and outputting strategy parameters; the evasion decision and path generation unit is used for outputting candidate evasion tracks and speed curves; the execution and control interface unit is used for outputting a chassis control instruction and implementing trajectory tracking; and the device communication and calculation platform is used for outputting a time sequence synchronization and data channel management result according to operation requirements and providing fault monitoring. According to the method, target state estimation, future evolution prediction, collision risk quantification and strategy linkage are completed within real-time constraints, consistent input and constraints are provided for planning and execution on a unified data channel, and safety and traceability are improved.

The invention discloses a subway service robot real-time path planning method based on a graph neural network, and the method comprises the following steps: scanning the environment in a subway station, completing the map construction, and obtaining basic environment data; performing target detection and personnel statistics to form an environment and passenger flow fusion map; taking the fused map as input, setting a starting point and a target position, and outputting a global path search initial state; calling an improved A * algorithm to carry out node expansion, calculating a cost estimation function, and generating candidate paths; performing redundant node deletion and arc smoothing processing on the candidate path, performing step length dynamic adjustment, and outputting an optimized global path; performing local path planning by adopting an improved dynamic window method, and outputting an update path in real time; a passenger evacuation scheme is generated and is output through a display screen and voice broadcast, evacuation is guided, and passing is guaranteed. According to the invention, real-time path planning of the subway service robot is realized.

The invention provides a service robot cross-modal user intention analysis method and system. The method comprises the following steps: firstly, acquiring and analyzing a voice instruction sequence and a visual flow containing a gesture, fusing a task target object and a gesture pointing direction to generate an intention hypothesis, and then projecting the intention hypothesis to a knowledge graph of an environment with a body; the method comprises the following steps: acquiring an intent hypothesis of a user, querying a candidate object set simultaneously matched with spatial constraints and semantic constraints of the intent hypothesis, when ambiguity exists in the candidate object set, generating an inquiry instruction for eliminating the ambiguity according to a relation path in a knowledge graph of the body environment, and finally analyzing and confirming a final user intent according to the response of the user to the inquiry instruction; according to the technical scheme provided by the invention, the ability of the robot to cope with uncertainties such as unknown reference and environment shielding in a real scene is enhanced, the interactive experience closer to the human cognitive level is realized, and the interactive efficiency and satisfaction of the user in a complex environment are also improved.

The invention relates to the technical field of service robots, and discloses an elderly care service robot based on an emotion adjustment system, and the robot comprises a multi-mode emotion recognition module which is used for carrying out the comprehensive perception of the emotional state of the elderly through combining various emotion signals of voice, facial expression and limb movement; the personalized emotion adjusting module is used for generating personalized emotion responses meeting the requirements of the old people; the emotion memory module is used for recording emotion changes, important life events and personal preferences of the elderly; the scene self-adaption module is used for sensing the living environment of the old people in real time and adjusting the emotional expression mode of the robot according to the environment information; the health intervention module is used for generating a health management scheme through long-term monitoring of the emotional state and health of the old people; and a safety monitoring module. According to the invention, the emotion state of the elderly is analyzed through the multi-modal fusion network, the emotion misjudgment probability is reduced, and the man-machine interaction experience and the emotion support effect of the elderly are remarkably improved.

The invention relates to the field of robot perception and intelligent man-machine interaction, an equipment platform and a perception module, a system is provided with an Intel RealSense D435i depth camera which is used for acquiring an RGB-D image sequence containing depth information in real time, and meanwhile, an advanced YOLOv8-Pose deep learning model is embedded, so that the depth information of the RGB-D image sequence is acquired in real time. High-precision detection, natural interaction, a gesture guidance mechanism, direction inference and target candidate aggregation, interactive target screening and judgment, a robustness enhancement strategy, deep deletion and noise robustness and an incomplete target completion mechanism of human upper body skeleton key points in a front scene are realized; the man-machine collaborative target object identification and positioning method based on depth vision, attitude estimation and semantic understanding is mainly realized. The technology can be suitable for service robots, intelligent assistant robots and other practical application scenes with high requirements for autonomous perception and natural interaction, and man-machine co-fusion and intelligent environment construction are further promoted.

The invention relates to the technical field of path planning, in particular to a path planning method and system for a commercial service robot, and the method comprises the following steps: analyzing regional attributes to obtain passage levels, constructing task response coefficients to adjust a task sequence, evaluating path node conflicts, planning an execution path, recognizing obstacles, and correcting the path. And ground parameters are extracted to adjust the attitude control parameters, and robot attitude adjustment configuration is generated. According to the method, the regional function state and the real-time time are dynamically corresponded, the regional traffic attribute is adjusted in real time, the task response coefficient is constructed, the task priority is dynamically optimized, the traffic frequency and the stay duration of the cross nodes are comprehensively quantized, and the path execution priority is finely evaluated; according to the method, the obstacle displacement track and the contour feature are combined, behavior recognition and path dynamic adjustment are achieved, and the refinement and real-time performance of path planning of the robot in a dynamic scene are effectively improved by evaluating the ground structure parameter value and the interference level in real time.

The invention provides a comprehensive scene map path planning method based on an improved A * algorithm, and the method comprises the steps: 1, carrying out environment modeling for a comprehensive scene map: segmenting a two-dimensional plane scene into more than two square grids, and carrying out the assignment of a single grid in a binarization manner to represent the occupation condition of the grids; constructing a protection area formed by a single-layer grid around the obstacle, and filling the protection area into gray; step 2, planning a path by adopting an improved A * algorithm: step 2-1, establishing an improved bidirectional search strategy; step 2-2, establishing a heuristic function with a barrier linear density weight; step 2-3, adopting a variable neighborhood node expansion mode to complete node expansion; and step 2-4, carrying out path secondary optimization. According to the method, the planning efficiency and the path performability in a complex environment are remarkably improved, and an effective solution is provided for navigation systems in the fields of service robots, intelligent warehousing and the like.

The invention discloses a modular function extension and dynamic task scheduling service robot system, which relates to the technical field of robots, and comprises the following specific steps: step 1, task feature extraction, step 2, machine learning prediction, step 3, dynamic priority calculation, step 4, heuristic task allocation, and step 5, load balancing scheduling. And step 6, task execution and feedback, in the method, the execution time and the resource demand of the task are predicted by using a machine learning model, and an optimal task scheduling scheme is calculated by using dynamic priority calculation, heuristic task allocation and load balancing scheduling. In the first step, tasks are fragmented, then pass through calculation and then are combined, the calculation complexity can be reduced, and the algorithm operation efficiency can be improved. In the fourth step, multiple iterations are carried out after randomization operation, and the scheduling scheme is optimized step by step, so that the global optimality of the scheduling scheme is improved, and falling into a local optimal solution is avoided.

The invention provides a customer service robot response method and system based on a knowledge graph, and relates to the field of natural language processing. The method comprises the following steps: obtaining a user input query; identifying a query intention input and queried by a user and slot position information through an intention identification model; generating a knowledge graph query statement based on the query intention and the slot position information, and retrieving related entities in a knowledge graph and associated entity relationships; executing path-based multi-hop semantic reasoning and rule-based logical reasoning on the basis of entities and associated entity relationships thereof, and determining an optimal response result; and generating a natural language response based on the optimal response result. The method is used in the response process of the customer service robot based on the knowledge graph, and the technical problem that an existing customer service robot is inaccurate in response to complex problems is solved.

The invention provides a data set self-construction method based on a multi-modal large language model, and belongs to the technical field of robots. Aiming at the defects of a traditional data set in the field of home service robots in the aspects of multi-modal association, dynamic scene modeling, time sequence constraint and the like, a structured Prompt framework is designed, and a hierarchical task decomposition strategy is established by combining a language-visual feature alignment technology with a hierarchical task decomposition strategy. Reliable mapping from a natural language instruction of a user to robot execution actions is realized, and a data set oriented to a real family complex scene is constructed. Moreover, by establishing the space-time constraint model and designing a double-layer verification mechanism, the execution rationality and task coherence of the action sequence generated by the multi-modal large language model are effectively improved.

The invention provides a service robot operation quality evaluation method based on a knowledge graph and related equipment, and relates to the technical field of robots. According to the invention, evaluation index quantification processing is carried out on the related operation information of the target service robot in the execution process of the target service task, so that the actual index values of the plurality of operation evaluation indexes are obtained; then, a robot operation knowledge graph matched with the target service robot is called to perform multi-level index analysis based on the obtained actual index value, so that an overall comprehensive index score of the target service robot for executing the target service task is determined, and when the overall comprehensive index score is lower than a preset index score threshold value, the target service robot is determined to execute the target service task; and calling a robot operation knowledge graph to perform reasoning to generate respective quality reduction reasons and index optimization suggestions of all the to-be-optimized evaluation indexes, thereby effectively improving the reliability of the service robot operation quality evaluation result by using the knowledge graph, and facilitating the auxiliary improvement of the related performance of the service robot in executing the complex long-term service task.

The invention discloses a modular reconfigurable multifunctional service robot, and relates to the technical field of robots, the modular reconfigurable multifunctional service robot comprises a mobile base, a man-machine interaction module is arranged above the mobile base, the man-machine interaction module and the mobile base are matched to form a welcome service robot, and a modular upper end module is further arranged at the top of the mobile base. The modularized upper end module comprises a distribution module, a health security check module, a security inspection module and a disinfection module, and the distribution module, the health security check module, the security inspection module and the disinfection module are respectively matched with the welcome service robot. According to the modular reconfigurable multifunctional service robot, various modular upper end modules are additionally mounted or replaced on the movable base, so that the multi-environment, multi-scene and multi-task adaptability is high; and due to the design of the universal movable base and the modular upper end module, the purchase cost of the multi-task function service robot is remarkably reduced, meanwhile, management and control are convenient and fast, and the overall use complexity and difficulty of the robot are reduced.

Provided in the embodiments of the present application are a generative question-answering system and method, and a device, a storage medium and a program product. In the embodiments of the present application, a question-answering server end in the generative question-answering system cooperates with a question-answering model and a knowledge base to reply to question information raised by a user, and correct, with the assistance of a question-answering assistance end in the generative question-answering system, answer information for replying to the question information, such that a personalized reply is given to the user in a more intelligent, efficient and accurate manner, thereby meeting personalized requirements of the user. Especially when serving as a customer service robot, the question-answering server end cooperates with the question-answering model and the knowledge base to provide a more intelligent, more efficient and personalized automatic answering function during a reception process of the customer service robot, a humanlike consultation experience is provided for consumers by means of automatic replies, and by means of agent assistance and knowledge base linkage, the reception efficiency is improved for a seller.

The invention provides a response method, device and equipment for a customer service robot, and relates to the technical field of customer service robots, and the method comprises the steps: obtaining multi-mode request data inputted into a current session with the customer service robot by a user, and a conversation state of the current session; according to the request data of different modals and the modal features of the request data of the corresponding modals, constructing a multi-modal heterogeneous graph; fusing the different modal features based on the request data of the different modalities, the corresponding modal features, the dialogue states and the multi-modal heterogeneous graph to obtain multi-modal fusion features; inputting the multi-modal fusion feature into a pre-constructed user intention response model based on a causal relationship to obtain a response strategy of the customer service robot to the multi-modal request data; wherein the user intention response model is used for representing the causal relationship between the multi-modal interaction data input by the user and the response strategy of the customer service robot. The response accuracy of the customer service robot can be improved, and the user interaction experience is improved.

The invention provides a message interaction method based on an AI customer service robot, and the method comprises the steps: receiving first user session data synchronized by a user terminal according to a preset session period, the first user session data comprising second user session data stored in a local cache or a session history library and synchronized in an abnormal interruption state, and the second user session data comprising an uncompleted session; and generating a comparison conclusion by comparing the context marks of the first user session data and the second user session data with the current dialogue context mark of the server, and taking the session data indicated by the conclusion as the updated multi-round dialogue context. Wherein the context mark comprises a timestamp and a session sequence number, when the mark is later than the current mark of the server, the data are merged, and the mark is updated, otherwise, the mark is ignored. According to the method, automatic recovery of abnormal interruption of the AI customer service dialogue and seamless synchronization of multi-terminal data can be realized, and the continuity of multiple rounds of dialogues and the user experience are improved.

The invention belongs to the technical field of robot navigation and path planning, and particularly discloses and provides a dynamic navigation planning method and system for a service robot, which overcomes the limitation of single-dimensional analysis in a complex dynamic scene by constructing a dynamic social thermodynamic diagram and deducing group dynamic intention data. The perception precision of the environment structure and the people stream situation is improved; a linkage mechanism among a dynamic social thermodynamic diagram, individual behavior characteristics and path planning is established, and a social cost function is generated to optimize a navigation path according to the traffic efficiency and social etiquette of group future dynamic vectorization evaluation; the actual influence of the navigation behavior of the robot on the environment is monitored in real time, and a planning strategy is adjusted in combination with feedback data, so that the adaptability of the navigation decision and the continuous improvement of the environment acceptance degree are realized.

The invention relates to a robot motion reasoning method and system based on a Gaussian motion field, and the method comprises the steps: inputting a sparse and uncalibrated multi-view RGB image, extracting mixed scene features through a visual Transform backbone network, constructing a dynamic Gaussian motion field, endowing each Gaussian unit with a learnable motion attribute, and carrying out the reasoning of the motion of a robot. And realizing synchronous modeling of scene geometry and motion evolution. The system further comprises a multi-modal query module, a point cloud registration module, a diffusion model optimization module and a closed-loop control module which are respectively used for current and future scene reconstruction, mechanical arm tail end clamping jaw action estimation, action sequence optimization and real-time feedback and model updating in the execution process. According to the method, through a unified space-time modeling and closed-loop control mechanism, the robot operation problem in dynamic, shielding and uncalibrated environments is effectively solved, and the method is suitable for various application scenes such as industrial automation and service robots.

Disclosed is a square-shaped service serving robot with an excellent service provision space utilization and, more specifically, to a square shaped service serving robot with an excellent service provision space utilization that it serves various services to be provided to customers, 5 provides a tableware return function depending on the situation, and maximizes driving safety and stability of the service serving robot by including sensors capable of detecting and recognizing obstacles and flow status information in real time, thereby promoting the usability and effectiveness of the service serving robot and the safety of the store.

The invention discloses an AI customer service robot automatic reply system based on a knowledge base, and relates to the technical field of intelligent customer service. Personal information of a user and a new question text sent by a user side are obtained, and user portrait analysis is carried out based on the personal information of the user; the emotion fluctuation state of the user is analyzed according to the historical dialogue record of the user, client classification is carried out, client classification results comprise emotion stable clients, pacifying demand clients and efficiency demand clients, and question feature words and emotion feature words are extracted based on the question text; querying a corresponding answer text in the knowledge base management unit according to the question feature words, judging the emotion of the user according to a customer classification result and the emotion feature words, correspondingly generating an emotion safety response text, and training based on a historical training text to obtain a text generation language model; and integrating the answer text and the emotion demand word into an answer text through a text generation language model, and outputting the answer text to a user side.

To make various kinds of improvements and refinements to service robots.SOLUTION: A service robot according to the present invention is a service robot that can travel and provides various kinds of services to a user, and includes a speaker that outputs voice to the user, a receiver that outputs voice to the user and can also input user's voice, a storage location for storing the receiver, and a control unit that performs voice control on the speaker and the receiver.SELECTED DRAWING: Figure 1