Multi-mode robot control method and device, electronic equipment and storage medium

Abstract

The invention provides a control method of a multi-mode robot. The method comprises the following steps: acquiring home environment data and personnel data of an area where the multi-mode robot is located; according to the home environment data and the personnel data, whether a target home task exists or not is determined, and the target home task is at least one of home tasks preset by the multi-mode robot; if it is determined that the target home task exists, determining a target mode of the multi-mode robot based on the target home task; and controlling the multi-mode robot to be switched to the target mode, so that the multi-mode robot executes the target home task in the target mode. Whether the target home task exists or not is determined by acquiring the home environment data and the personnel data of the located area, and the corresponding target home task is executed in the corresponding target mode under the condition that the target home task exists, so that a user does not need to input the corresponding task in advance, and the user experience is improved. And the autonomous ability and practicability of the multi-mode robot in the home environment are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a control method, device, electronic equipment and storage medium for a multi-mode robot. Background Art

[0002] With the rapid development of science and technology, smart home has become an important part of modern life. In order to further improve the convenience and comfort of home life, a new type of robot has emerged, such as butler robots and embodied robots. This type of robot not only has the function of flexible conversion between multiple modes, but also has its own unique actuators in different modes, which can easily cope with the execution of various household tasks.

[0003] In daily life, family members are faced with many household tasks, such as cleaning, cooking, and taking care of children and pets. Although these tasks are trivial, they take up a lot of time and energy. In order to reduce the burden on family members and improve the quality of life, multi-mode robots are designed. Users can choose corresponding modes to perform corresponding household tasks. For example, in cleaning mode, the multi-mode robot will pop out a cleaning mechanism or component, and then use the cleaning mechanism or component to perform corresponding vacuuming, sweeping, mopping and other functions. In baby care mode, the multi-mode robot will use the mechanical arm mechanism to shake the cradle, and use the audio-visual module to output corresponding audio-visual information to attract or comfort the baby.

[0004] However, the tasks of existing multimodal robots require input from home users, and they are unable to timely and effectively discover potential tasks for autonomous processing, making the task processing efficiency of multimodal robots low. Summary of the invention

[0005] The embodiment of the present invention provides a control method for a multi-mode robot, aiming to provide a control method for a multi-mode robot to improve the autonomous capability and practicality of the multi-mode robot in a home environment. By acquiring the home environment data and personnel data of the area, it is determined whether there is a target home task, thereby realizing autonomous discovery of the home task. When there is a target home task, the corresponding target home task is executed in the corresponding target mode, and the user does not need to input the corresponding task in advance, thereby improving the autonomous capability and practicality of the multi-mode robot in the home environment.

[0006] In a first aspect, an embodiment of the present invention provides a control method for a multi-mode robot, wherein the multi-mode robot has a conversion function between multiple modes, and the multi-mode robot in different modes has actuators of different forms, and the method comprises the following steps:

[0007] Acquire home environment data and personnel data of the area where the multi-mode robot is located;

[0008] Determining whether there is a target home task according to the home environment data and the personnel data, wherein the target home task is at least one of the home tasks preset by the multi-mode robot;

[0009] If it is determined that the target home task exists, determining the target mode of the multi-mode robot based on the target home task;

[0010] The multi-mode robot is controlled to switch to the target mode, so that the multi-mode robot performs the target household task in the target mode.

[0011] Optionally, the acquiring of home environment data and personnel data of an area where the multi-mode robot is located includes:

[0012] Acquiring multimodal data of an area where the multimodal robot is located;

[0013] Based on the multimodal data, home environment data and personnel data of the area where the multimodal robot is located are determined.

[0014] Optionally, determining whether there is a target home task according to the home environment data and the personnel data includes:

[0015] Processing the home environment data and the personnel data through a behavior detection model to obtain the behavior intention of each person;

[0016] Based on the behavioral intention, performing task matching processing in the household tasks preset by the multi-mode robot;

[0017] If the match is successful, it is determined that the target home task exists, and the successfully matched home task is determined as the target home task;

[0018] If the matching fails, it is determined that the target home task does not exist.

[0019] Optionally, performing task matching processing in household tasks preset by the multi-mode robot based on the behavioral intention includes:

[0020] If the behavior intention is an intention to switch from a first behavior to a second behavior, then based on the first behavior, task matching processing is performed in the home tasks preset by the multi-mode robot, and the first behavior is before the second behavior;

[0021] If the first behavior fails to match, task matching processing is performed in the home tasks preset by the multi-mode robot based on the second behavior.

[0022] Optionally, determining a target mode of the multi-mode robot based on the target household task includes:

[0023] If the number of the target home tasks is one, determining the target mode corresponding to the target home tasks based on the correspondence between preset home tasks and modes;

[0024] If there are multiple target home tasks, predict the revenue value when executing each target home task in each mode;

[0025] Sorting the target home tasks from high to low according to the benefit values ​​to obtain the target home task sequence;

[0026] Based on the target home task sequence, a target mode of the multi-mode robot is determined.

[0027] Optionally, the predicting of the revenue value when executing each of the target household tasks in each of the modes includes:

[0028] Based on at least one of the time sensitivity of the target home task, the comfort level of personnel, and the energy efficiency of the multi-mode robot, the profit value when performing each of the target home tasks in each of the modes is predicted.

[0029] Optionally, after determining the target mode of the multi-mode robot based on the target household task, the method further includes:

[0030] Determine whether there is a preset scheduled task;

[0031] If it is determined that there is a preset scheduled task, then determining whether the scheduled task conflicts with the target home task;

[0032] If it is determined that there is a conflict between the scheduled task and the target home task, searching or planning a conflict resolution method between the scheduled task and the target home task;

[0033] A target mode of the multimodal robot is adjusted based on the conflict resolution method.

[0034] In a second aspect, an embodiment of the present invention further provides a control device for a multi-mode robot, wherein the multi-mode robot has a conversion function between multiple modes, and the multi-mode robot in different modes has actuators of different forms, and the control device for the multi-mode robot includes:

[0035] An acquisition module, used to acquire home environment data and personnel data of an area where the multi-mode robot is located;

[0036] A first processing module, configured to determine whether there is a target home task according to the home environment data and the personnel data, wherein the target home task is at least one of the home tasks preset by the multi-mode robot;

[0037] A second processing module is configured to determine a target mode of the multi-mode robot based on the target home task if it is determined that the target home task exists;

[0038] The control module is used to control the multi-mode robot to switch to the target mode, so that the multi-mode robot performs the target household task in the target mode.

[0039] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the control method of the multimodal robot provided in an embodiment of the present invention when executing the computer program.

[0040] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the control method of the multi-mode robot provided in the embodiment of the invention are implemented.

[0041] In an embodiment of the present invention, home environment data and personnel data of the area where the multi-mode robot is located are obtained; based on the home environment data and personnel data, it is determined whether there is a target home task, and the target home task is at least one of the home tasks preset by the multi-mode robot; if it is determined that there is a target home task, the target mode of the multi-mode robot is determined based on the target home task; the multi-mode robot is controlled to switch to the target mode, so that the multi-mode robot performs the target home task in the target mode. By obtaining the home environment data and personnel data of the area where the robot is located to determine whether there is a target home task, autonomous discovery of home tasks is achieved. When there is a target home task, the corresponding target home task is performed in the corresponding target mode, and the user does not need to input the corresponding task in advance, thereby improving the autonomous ability and practicality of the multi-mode robot in the home environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1is a flow chart of a control method of a multi-mode robot provided by an embodiment of the present invention;

[0044] Figure 2 is a schematic structural diagram of a control device for a multi-mode robot provided by an embodiment of the present invention;

[0045] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] like Figure 1 As shown, Figure 1 1 is a flow chart of a control method of a multi-mode robot provided by an embodiment of the present invention. The multi-mode robot has a conversion function between multiple modes. The multi-mode robot in different modes has actuators of different forms. The control method of the multi-mode robot includes the following steps:

[0048] 101. Obtain home environment data and personnel data in the area where the multi-mode robot is located.

[0049] In an embodiment of the present invention, the control method of the multi-mode robot can be applied to a robot control system, wherein the robot control system includes a task discovery module and a control module, wherein the control module can be a robot control platform or an electronic device, wherein the electronic device can be an intelligent gateway, an intelligent networking device or a server, etc., and the robot control platform can be constructed based on a server or a distributed server. The control platform or electronic device of the multi-mode robot can include a robot interface, a data processing program, etc., wherein the robot control platform or electronic device can communicate with the multi-mode robot through the robot interface, thereby receiving the status data of the multi-mode robot and sending tasks to the multi-mode robot.

[0050] The multi-mode robot can be equipped with advanced sensors and cameras, which can scan and collect home environment data (such as temperature, humidity, light intensity, location of items, etc.) and personnel data (such as the location, activity status, health monitoring data of family members, etc.) in real time. The mode robot can perform environmental recognition in the home environment, walk autonomously in the home environment based on the recognized environmental data and personnel data, and perform corresponding tasks through corresponding functional structures. The multi-mode robot can be a robot integrated with corresponding actuators such as cleaning robots, educational robots, and nursing robots, or it can be an embodied robot with a humanoid structure that can perform more types of functions. The multi-mode robot has the function of switching between multiple modes, and multi-mode robots in different modes have actuators of different forms.

[0051] The above tasks can be set in advance by the user, or temporarily input by the user, or can be tasks discovered by the home robot based on autonomous discovery. Autonomous discovery tasks can be implemented based on image recognition or voice recognition algorithms. Image recognition can be used to identify tasks based on environmental data and personnel data in the current home environment. When tasks that need to be handled are identified in the current home environment, tasks can be automatically generated and uploaded.

[0052] The home environment data may include room temperature, humidity, light intensity, air quality, furniture layout, obstacle location, etc. The home environment data may be collected and analyzed by corresponding sensors installed inside or outside the multi-mode robot.

[0053] The above personnel data may include the location, behavior status (such as meeting, resting, working, and entertainment) and health status (such as whether special care is needed) of family members. The above personnel data may be obtained by collecting and analyzing corresponding sensors installed inside or outside the multi-mode robot.

[0054] The environmental data and personnel data of the area where the multi-mode robot is located can be collected at regular intervals, and then it can be determined whether there is a target home task in the area where the multi-mode robot is located based on the environmental data and personnel data of the area where the multi-mode robot is located.

[0055] 102. Determine whether there is a target home task based on the home environment data and personnel data.

[0056] In an embodiment of the present invention, a home task table can be preset for the multi-mode robot, and the home task table records all home tasks that the multi-mode robot can perform. In the home task table, different home tasks can correspond to different home environment data and personnel data. Therefore, after obtaining the home environment data and personnel data, it can be determined whether the target home task exists by looking up the table. If the table lookup is successful, it can be determined that the target home task exists, and the found home task is determined as the target home task; if the table lookup fails, it can be determined that the target home task does not exist, and the task discovery is terminated.

[0057] The target household task is at least one of the household tasks preset for the multi-mode robot.

[0058] 103. If it is determined that there is a target home task, a target mode of the multi-mode robot is determined based on the target home task.

[0059] In the embodiment of the present invention, different household tasks may require the robot to be performed in different forms or configurations, that is, different household tasks may require the corresponding actuators of the robot to be performed. For example, the cleaning task requires the robot to switch to vacuuming or mopping mode, while carrying heavy objects requires a stronger mechanical arm configuration, and the cradle task requires a timed swinging of the mechanical arm. The most suitable one can be selected from the multiple modes of the robot according to the target household task.

[0060] Specifically, a mode-task table can be preset for the multi-mode robot. The mode-task table records all household tasks that the multi-mode robot can perform and the recommended mode of each household task. One household task corresponds to at least one recommended mode. In the mode-task table, household tasks are recorded in the form of task tags, and recommended modes are recorded in the form of mode tags. One task tag can correspond to at least one mode tag. Therefore, after obtaining the target household task, the mode tag corresponding to the target household task can be determined by looking up the table, and then the mode recommended for performing the target household task can be determined, and the recommended mode can be used as the target mode of the multi-mode robot.

[0061] 104. Control the multi-mode robot to switch to the target mode, so that the multi-mode robot performs the target household task in the target mode.

[0062] In the embodiment of the present invention, after determining the target mode, the control module can send a command to the multi-mode robot with a switch command of the target mode, so that it can perform necessary morphological transformation or configuration adjustment to adapt to the task to be performed. Subsequently, the multi-mode robot will perform the task in this mode, such as cleaning, carrying, monitoring, etc., until the task is completed or a new command is received.

[0063] In an embodiment of the present invention, home environment data and personnel data of the area where the multi-mode robot is located are obtained; based on the home environment data and personnel data, it is determined whether there is a target home task, and the target home task is at least one of the home tasks preset by the multi-mode robot; if it is determined that there is a target home task, the target mode of the multi-mode robot is determined based on the target home task; the multi-mode robot is controlled to switch to the target mode, so that the multi-mode robot performs the target home task in the target mode. By obtaining the home environment data and personnel data of the area where the robot is located to determine whether there is a target home task, autonomous discovery of home tasks is achieved. When there is a target home task, the corresponding target home task is performed in the corresponding target mode, and the user does not need to input the corresponding task in advance, thereby improving the autonomous ability and practicality of the multi-mode robot in the home environment.

[0064] It can be understood that in the specific implementation of this application, related data such as robot data, home environment data, user data, family member data, multimodal data, etc. are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data, as well as the training, deployment and calling of the model, must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0065] Optionally, in the step of obtaining home environment data and personnel data of the area where the multimodal robot is located, multimodal data of the area where the multimodal robot is located may be obtained; based on the multimodal data, the home environment data and personnel data of the area where the multimodal robot is located may be determined.

[0066] In the embodiments of the present invention, multimodal data refers to data collected by various sensors and devices, and the multimodal data may include information in multiple dimensions such as vision, hearing, touch, temperature, humidity, etc. For a multimodal robot, multimodal data can be obtained by collecting and analyzing corresponding sensors set inside or outside the robot.

[0067] The above-mentioned visual data can be collected by a camera or an infrared sensor. By analyzing and processing the visual data, visual information such as the layout of the room, the location of furniture, and the activities of people can be captured.

[0068] The above auditory data can be collected by a microphone, which can collect sounds in the room, such as conversations between family members, calls of pets, playing sounds of TV or music, etc. The above auditory data can help determine the atmosphere of the current environment and whether there are any voice commands or abnormal situations that need to be responded to.

[0069] The above tactile data can be used to detect the hardness, temperature and other characteristics of the object through a tactile sensor. The tactile data can be used to perform tasks such as carrying and cleaning.

[0070] Environmental parameter data can include environmental parameters such as temperature, humidity, and light intensity. Environmental parameter data can analyze the factors that affect maintaining a comfortable home environment. Multimodal robots can obtain this data through built-in sensors or connected to smart home systems.

[0071] After the multimodal data is collected, the multimodal data can be processed and analyzed to determine specific home environment data and personnel data.

[0072] For home environment data, the cleanliness of the room, the placement of furniture, whether there are any debris that need to be cleaned, etc. can be identified through the analysis of visual data. Combined with environmental parameter data, the comfort level of the current environment can also be judged, such as whether the air conditioning temperature needs to be adjusted, the humidifier needs to be turned on, or the curtains need to be closed.

[0073] For personnel data, the location, behavior status (analyzed by behavior recognition algorithm) and possible emotional changes (analyzed by facial expression recognition algorithm) of family members can be identified through comprehensive analysis of visual and auditory data. For example, if a person is detected resting on the sofa and the TV volume is loud, the robot may determine that the person is taking a nap and generate a task to lower the TV volume or turn off the TV for execution.

[0074] In addition, machine learning algorithms can be used to conduct in-depth analysis of the collected multimodal data to discover potential patterns and regularities. These potential patterns and regularities can be used as feedback data to help robots better predict the needs and preferences of family members using reinforcement learning methods, thereby providing more personalized and efficient services.

[0075] Acquiring multimodal data and determining home environment data and personnel data based on the multimodal data can more accurately discover real-time home tasks in the current area.

[0076] Optionally, in the step of determining whether a target home task exists based on the home environment data and the personnel data, the home environment data and the personnel data can be processed by a behavior detection model to obtain the behavior intention of each person; based on the behavior intention, task matching processing is performed in the home tasks preset by the multi-mode robot; if the match is successful, it is determined that the target home task exists, and the successfully matched home task is determined as the target home task; if the match fails, it is determined that the target home task does not exist.

[0077] In an embodiment of the present invention, the above-mentioned behavior detection model can efficiently and accurately analyze the home environment data and personnel data. These behavior detection models can be based on deep learning algorithms, such as convolutional neural networks (CNN) or recurrent neural networks (RNN), which can identify complex behavior patterns and intentions.

[0078] Home environment data can include room layout, furniture location, item placement, etc., to understand the overall condition of the current environment. Home tasks related to the home environment, such as whether the floor is dirty and needs to be cleaned, or whether the window is open and needs to be closed.

[0079] Personnel data may include the person's location, movements, expressions, and voices, etc. Through intention recognition processing based on the person's location, movements, expressions, and voices, the person's behavioral intentions can be inferred, such as whether they are preparing to eat, whether they are about to leave the room, whether they need help, etc.

[0080] After obtaining the behavioral intention of the person, the multi-mode robot can match the preset household tasks in the table. These preset household tasks can include cleaning, care, monitoring, entertainment, etc., and each task corresponds to a specific trigger condition and execution mode.

[0081] For matching behavioral intentions with tasks, the identified behavioral intentions of people can be compared with preset tasks one by one to find the task that best fits the current situation. For example, if someone is identified as preparing food in the kitchen, the model may match tasks such as "assisting in cooking" or "kitchen cleaning".

[0082] If a matching task is found, it can be determined that there is a target home task, and the task is marked as the target task for the robot to execute later. If a matching task is not found, it can be determined that there is no target home task at present, and the robot will remain in standby state or execute other default tasks.

[0083] Through precise task matching, multimodal robots can provide assistance to users at the most appropriate time and in the most appropriate mode, thereby enhancing the user experience.

[0084] Through behavioral detection models and preset tasks of multimodal robots, as well as in-depth analysis of data to identify and determine the home tasks that need to be performed, not only the task discovery accuracy of multimodal robots is improved, but also a more convenient and intelligent home life experience is brought to users.

[0085] Optionally, in the step of performing task matching processing in home tasks preset for the multi-mode robot based on behavioral intention, if the behavioral intention is to switch from a first behavior to a second behavior, then task matching processing is performed in home tasks preset for the multi-mode robot based on the first behavior, and the first behavior is before the second behavior; if the first behavior fails to match, then task matching processing is performed in home tasks preset for the multi-mode robot based on the second behavior.

[0086] In the embodiment of the present invention, when the behavioral intention is manifested as a change from one behavior (first behavior) to another behavior (second behavior), this process reflects the conversion of the person's intention.

[0087] The change of behavioral intention, that is, the intention of the first behavior to the second behavior, can be understood as a behavior that the person is currently performing or about to end (the first behavior), and a behavior that the person intends to perform next (the second behavior). The intention change may be triggered by a variety of factors, such as time, environmental changes, personal needs, etc. For example, a person may change from taking care of a baby (the first behavior) to making milk powder (the second behavior), which may be because the baby is hungry, or because it is time to feed, and the person who takes care of the baby has to change his behavioral intention from taking care of the baby to making milk powder. At this time, it can be determined that the baby needs to be taken care of, so a home task of taking care of the baby can be generated.

[0088] When it is identified that the person's behavioral intention is to switch from the first behavior to the second behavior, task matching can be performed in the home tasks preset by the multi-modal robot based on the first behavior. Then, the task most directly related to the behavior currently being performed or about to end can be found. For example, if the first behavior is reading, tasks such as "adjust the lighting to suit reading", "provide reading material recommendations", or "play relaxing music to create a reading atmosphere" may be matched.

[0089] During the matching process, multiple factors should be considered, including the relevance, urgency, feasibility of the task, and whether it is consistent with the person's preferences and historical behavior. If the first behavior is successfully matched, that is, a task that is highly relevant to the current behavior and meets the execution conditions is found, then the execution of the task can be triggered immediately, or wait for a suitable time (such as after the person completes the current first behavior) before execution.

[0090] If the first behavior fails to match, the task matching is performed based on the second behavior. If the task matching based on the first behavior fails, that is, no task directly related to the current behavior or meeting the execution conditions is found, the task matching process can be performed based on the second behavior. The logic of this step is that since there is no directly corresponding task for the current behavior, the upcoming behavior can be considered to see if there are any tasks that can be prepared or assisted in advance.

[0091] Taking the previous reading example, if the first behavior is reading, but the system does not find a task directly related to it (perhaps because the current environment is already suitable enough for reading, or the person does not show a need for reading assistance), then when the system recognizes that the person is planning to prepare dinner next, it may match tasks such as "preheat the oven in advance", "prepare the ingredients needed for cooking", or "play a cooking tutorial video".

[0092] The continuity and smoothness of behavioral transitions can also be considered throughout the task matching process. Based on the identification of behavioral transitions, it is also possible to predict and adapt to environmental changes and demand changes that may be brought about by such transitions. For example, a transition from reading to preparing dinner may mean that a person needs more space, different lighting conditions, and a different sound environment. The system can automatically adjust its task execution strategy based on these changes to ensure the comfort and convenience of the home environment.

[0093] By deeply understanding people's behavioral intentions, accurately identifying the timing and nature of behavioral changes, and finding the tasks that best suit the current situation among the many home tasks preset by the multimodal robot, it can more intelligently adapt to people's needs and preferences, and provide more personalized, efficient and considerate services, thereby improving the comfort and convenience of home life.

[0094] Optionally, in the step of determining the target mode of the multi-mode robot based on the target home tasks, if the number of target home tasks is one, then based on the correspondence between preset home tasks and modes, the target mode corresponding to the target home task is determined; if the number of target home tasks is multiple, then the benefit value of executing each target home task in each mode is predicted; the target home tasks are sorted from high to low according to the benefit value to obtain a target home task sequence; based on the target home task sequence, the target mode of the multi-mode robot is determined.

[0095] In an embodiment of the present invention, when the number of target home tasks is one, the decision process is relatively straightforward. In this case, the target mode corresponding to the target home task can be quickly determined based on the correspondence between the preset home tasks and modes in the mode-task table. The correspondence can be a mapping relationship established based on the analysis of historical data and user preferences, and the correspondence can ensure that the robot can select the most appropriate working mode for a specific home task.

[0096] For example, if the target household task is to clean the floor, the robot may automatically switch to "cleaning mode", which is equipped with specialized cleaning tools and algorithms to complete the cleaning task in a corresponding manner. This direct mapping approach reduces decision complexity and enables the robot to respond and perform tasks quickly.

[0097] When there are multiple target home tasks, the decision-making process becomes more complicated. It is necessary to consider how to perform multiple target home tasks in the optimal order and mode under limited time and resources.

[0098] Specifically, for each possible working mode, the benefit value of executing each target home task in the mode can be predicted. The above benefit value can be determined based on the importance, urgency, execution difficulty, resource consumption, and user preference of the task. Alternatively, by comprehensively considering factors such as the importance, urgency, execution difficulty, resource consumption, and user preference of the task, the potential value of executing the task in different modes can be evaluated.

[0099] After obtaining the benefit value of executing each target home task in each mode, the target home tasks can be sorted according to the predicted benefit value to obtain a target home task sequence. The target home task sequence reflects the priority and order of task execution, ensuring that the multi-mode robot can first process the task with the highest benefit.

[0100] Based on the sorted target home task sequence, the target mode of the multi-mode robot can be determined. The target home task sequence is used to determine the switching and combination of multiple modes to meet the needs of different tasks. For example, the robot may first switch to "cleaning mode" to clean the floor, then switch to "cooking mode" to prepare dinner, and finally switch to "entertainment mode" to play music for the user.

[0101] In a possible embodiment, in the process of determining the target mode, some additional factors such as mode switching cost, task continuity and user real-time feedback can also be considered as factors affecting the overall benefit value. Among them, the switching cost refers to the time and resources required to switch from one mode to another, and the switching cost affects the overall efficiency of the multi-mode robot. The continuity of the task affects whether the multi-mode robot can maintain a certain degree of logical order and fluency when performing a series of related tasks. The real-time feedback of the user is an important basis for the multi-mode robot to adjust its working mode, which can help the robot better adapt to the needs and preferences of the user.

[0102] Whether it is a single or multiple target home tasks, multiple factors need to be considered comprehensively when determining the target mode, and corresponding optimization needs to be performed, which may include an in-depth understanding and analysis of the nature of the task, user needs, resource constraints, and environmental conditions. This will enable the multi-mode robot to adapt to the home environment more intelligently and be able to flexibly select and optimize its working mode according to the number and nature of the target home tasks.

[0103] Optionally, in the step of predicting the profit value when performing each target home task in each mode, the profit value when performing each target home task in each mode can be predicted based on at least one of the time sensitivity of the target home task, the comfort of the personnel, and the energy efficiency of the multi-mode robot.

[0104] In the embodiments of the present invention, time sensitivity can be understood as the urgency of household tasks and the time limit for execution. Different tasks have different time requirements. Some tasks need to be completed immediately, such as urgent cleaning needs or immediate security alarm response; while some tasks can be completed within a more relaxed time frame, such as daily home maintenance or scheduled home appliance control.

[0105] When predicting revenue values, for time-sensitive tasks, we can give priority to those tasks that can be responded to quickly and completed efficiently. These task modes can have faster execution speeds and higher flexibility, and can meet user needs in the shortest time. Therefore, when executing such tasks, we can give them higher revenue values ​​to reflect their advantages in time efficiency.

[0106] Human comfort can be understood as the impact of household tasks on the quality of life and feelings of the person. Human comfort can include environmental factors such as noise level, air quality, temperature and humidity, as well as the degree of interference with the user's daily life during the task execution.

[0107] When predicting revenue value, you can choose tasks that minimize the negative impact on occupant comfort while maintaining efficient execution. For example, when performing cleaning tasks at night, you can choose a low-noise mode to avoid disturbing the user's rest. Similarly, when adjusting the indoor environment, you can prioritize modes that provide comfortable temperature and air quality.

[0108] In order to accurately predict the benefit value, the potential impact on user comfort during task execution can be comprehensively considered, and personalized adjustments can be made based on user preferences and habits.

[0109] Energy efficiency can be understood as the ratio between the amount of energy used by the robot when performing household tasks and the effectiveness of its task completion. Efficient energy consumption means that the robot can minimize energy waste and impact on the home environment while completing tasks.

[0110] When predicting the revenue value, you can choose a task mode that can achieve the lowest energy consumption while ensuring the quality of the task.

[0111] In order to accurately evaluate the impact of energy efficiency on revenue, the energy consumption of multi-mode robots can be monitored and analyzed in real time, and intelligent selection can be made based on the energy consumption characteristics of different modes. In a possible embodiment, the multi-mode robot should also consider the impact of energy price fluctuations and peak and valley periods to further optimize its energy consumption strategy.

[0112] In practical applications, predicting the benefit value when executing each target home task in each mode is a process that comprehensively considers multiple factors. The embodiment of the present invention takes into account the specific requirements of the task and the personalized needs of the user, and weighs multiple factors such as time sensitivity, personnel comfort and energy efficiency to obtain the most reasonable benefit value prediction. It can be dynamically adjusted and optimized according to different situations and needs. By continuously optimizing its decision-making algorithms and models, the robot can more accurately predict the benefit value, thereby providing users with a more efficient, comfortable and environmentally friendly home service experience.

[0113] Optionally, after the step of determining the target mode of the multi-mode robot based on the target home task, it can also be determined whether there is a preset scheduled task; if it is determined that there is a preset scheduled task, it is determined whether there is a conflict between the scheduled task and the target home task; if it is determined that there is a conflict between the scheduled task and the target home task, a conflict resolution method between the scheduled task and the target home task is searched or planned; and the target mode of the multi-mode robot is adjusted based on the conflict resolution method.

[0114] In the embodiment of the present invention, in the daily operation of the multi-mode robot, the above-mentioned scheduled tasks can be set based on the user's living habits and preferences, such as scheduled cleaning, scheduled light switching, scheduled cooking, etc. Scheduled tasks are intended to provide users with scheduled and automated services to improve the convenience and comfort of life.

[0115] After determining the target home tasks, you can check whether there are any preset scheduled tasks that conflict with these tasks. Conflicts may manifest as time overlap, competition for resource use, or unreasonable order of task execution. For example, if the target home task is to urgently clean an area, and the preset scheduled task is also to clean the area, and the time is close, then there will be a time conflict. Similarly, if both tasks need to use the same resource (such as a vacuum cleaner), there will be a resource use conflict.

[0116] In a possible embodiment, in order to accurately determine the conflict, multiple factors may be considered comprehensively, such as the time, location, required resources, execution order, etc. of the task. By comparing and analyzing the time, location, required resources, execution order, etc. of the task, potential conflict points can be identified, providing a basis for subsequent conflict resolution.

[0117] After determining that a conflict exists, an effective solution can be searched for or planned.

[0118] The above solutions may include adjusting the time of scheduled tasks, changing the order of task execution, allocating different resources, or merging similar tasks to reduce conflicts. The order of target home tasks and scheduled tasks can be planned according to the priority of the tasks to resolve the conflicts between target home tasks and scheduled tasks. For example, if both the scheduled task and the target home task need to use the same resource, the robot may suggest adjusting the time of the scheduled task to after the target home task is completed, or finding alternative resources to complete the task.

[0119] In one possible embodiment, when searching for solutions, the robot can also use historical data and user feedback to optimize its decision-making. By analyzing the handling of similar conflicts in the past and user satisfaction, the robot can more accurately predict the effects of different solutions and select the solution that best meets user expectations.

[0120] Based on the selected conflict resolution method, the target mode of the multi-mode robot can be adjusted accordingly. Such adjustments include changing the robot's working mode, reallocating task resources, adjusting the order of task execution, etc. The purpose of the adjustment is to ensure that the robot can perform all tasks efficiently and orderly while meeting user needs.

[0121] In a possible embodiment, during the adjustment process, the user's real-time feedback and changing needs can also be considered. If the user is not satisfied with the new solution or has new requirements, the robot can respond and make corresponding adjustments. This is the key to ensuring that the robot can continue to provide high-quality services.

[0122] like Figure 2 As shown, an embodiment of the present invention provides a control device for a multi-mode robot, the control device for the multi-mode robot comprising:

[0123] An acquisition module 201 is used to acquire home environment data and personnel data of the area where the multi-mode robot is located;

[0124] A first processing module 202 is used to determine whether there is a target home task according to the home environment data and the personnel data, where the target home task is at least one of the home tasks preset by the multi-mode robot;

[0125] A second processing module 203 is configured to determine a target mode of the multi-mode robot based on the target home task if it is determined that the target home task exists;

[0126] The control module 204 is used to control the multi-mode robot to switch to the target mode, so that the multi-mode robot performs the target household task in the target mode.

[0127] Optionally, the acquisition module 201 is further used to acquire multimodal data of the area where the multimodal robot is located; based on the multimodal data, determine the home environment data and personnel data of the area where the multimodal robot is located.

[0128] Optionally, the first processing module 202 is also used to process the home environment data and the personnel data through a behavior detection model to obtain the behavior intention of each person; based on the behavior intention, perform task matching processing in the home tasks preset by the multi-mode robot; if the match is successful, it is determined that the target home task exists, and the successfully matched home task is determined as the target home task; if the match fails, it is determined that the target home task does not exist.

[0129] Optionally, the first processing module 202 is also used to perform task matching processing in the home tasks preset for the multi-mode robot based on the first behavior if the behavior intention is to switch from the first behavior to the second behavior, and the first behavior is before the second behavior; if the first behavior fails to match, then based on the second behavior, perform task matching processing in the home tasks preset for the multi-mode robot.

[0130] Optionally, the second processing module 203 is also used to determine the target mode corresponding to the target home task based on the corresponding relationship between the preset home tasks and the mode if the number of the target home tasks is one; if the number of the target home tasks is multiple, predict the benefit value when performing each target home task under each mode; sort the target home tasks from high to low according to the benefit value to obtain the target home task sequence; and determine the target mode of the multi-mode robot based on the target home task sequence.

[0131] Optionally, the second processing module 203 is also used to predict the profit value when performing each of the target home tasks in each of the modes based on at least one of the time sensitivity of the target home tasks, the comfort of personnel, and the energy efficiency of the multi-mode robot.

[0132] Optionally, the device further comprises:

[0133] The third processing module is used to determine whether there is a preset timed task;

[0134] A fourth processing module is used to determine whether there is a conflict between the scheduled task and the target home task if it is determined that there is a preset scheduled task;

[0135] A search module, for searching or planning a conflict resolution method between the scheduled task and the target home task if it is determined that the scheduled task and the target home task conflict;

[0136] An adjustment module is used to adjust the target mode of the multi-mode robot based on the conflict resolution method.

[0137] like Figure 3 As shown, an embodiment of the present invention further provides an electronic device, including a processor, and the processor can execute any one of the above-mentioned control methods for the multi-mode robot.

[0138] Specifically, it includes a processor 301 and a memory 302, and a computer program for executing the control method of the multi-mode robot stored in the memory 302 and capable of running on the processor 301, wherein:

[0139] The processor 301 runs the computer program of the control method of the multi-mode robot stored in the memory 302 to perform the following steps:

[0140] Acquire home environment data and personnel data of the area where the multi-mode robot is located;

[0141] Determining whether there is a target home task according to the home environment data and the personnel data, wherein the target home task is at least one of the home tasks preset by the multi-mode robot;

[0142] If it is determined that the target home task exists, determining the target mode of the multi-mode robot based on the target home task;

[0143] The multi-mode robot is controlled to switch to the target mode, so that the multi-mode robot performs the target household task in the target mode.

[0144] Optionally, the acquiring of home environment data and personnel data of an area where the multi-mode robot is located performed by the processor 301 includes:

[0145] Acquiring multimodal data of an area where the multimodal robot is located;

[0146] Based on the multimodal data, home environment data and personnel data of the area where the multimodal robot is located are determined.

[0147] Optionally, the determining whether there is a target home task according to the home environment data and the personnel data performed by the processor 301 includes:

[0148] Processing the home environment data and the personnel data through a behavior detection model to obtain the behavior intention of each person;

[0149] Based on the behavioral intention, performing task matching processing in the household tasks preset by the multi-mode robot;

[0150] If the match is successful, it is determined that the target home task exists, and the successfully matched home task is determined as the target home task;

[0151] If the matching fails, it is determined that the target home task does not exist.

[0152] Optionally, the processor 301 performs task matching processing in the household tasks preset by the multi-mode robot based on the behavioral intention, including:

[0153] If the behavior intention is an intention to switch from a first behavior to a second behavior, then based on the first behavior, task matching processing is performed in the home tasks preset by the multi-mode robot, and the first behavior is before the second behavior;

[0154] If the first behavior fails to match, task matching processing is performed in the home tasks preset by the multi-mode robot based on the second behavior.

[0155] Optionally, the determining of the target mode of the multi-mode robot based on the target household task performed by the processor 301 includes:

[0156] If the number of the target home tasks is one, determining the target mode corresponding to the target home tasks based on the correspondence between preset home tasks and modes;

[0157] If there are multiple target home tasks, predict the revenue value when executing each target home task in each mode;

[0158] Sorting the target home tasks from high to low according to the benefit values ​​to obtain the target home task sequence;

[0159] Based on the target home task sequence, a target mode of the multi-mode robot is determined.

[0160] Optionally, the prediction of the revenue value when performing each of the target home tasks in each of the modes performed by the processor 301 includes:

[0161] Based on at least one of the time sensitivity of the target home task, the comfort level of personnel, and the energy efficiency of the multi-mode robot, the profit value when performing each of the target home tasks in each of the modes is predicted.

[0162] Optionally, after determining the target mode of the multi-mode robot based on the target household task, the method executed by the processor 301 further includes:

[0163] Determine whether there is a preset scheduled task;

[0164] If it is determined that there is a preset scheduled task, then determining whether the scheduled task conflicts with the target home task;

[0165] If it is determined that there is a conflict between the scheduled task and the target home task, searching or planning a conflict resolution method between the scheduled task and the target home task;

[0166] A target mode of the multimodal robot is adjusted based on the conflict resolution method.

[0167] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the computer program implements the various processes of the control method of the multi-mode robot provided in the embodiment of the present invention or the control method of the multi-mode robot on the application side, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0168] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0169] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A control method for a multi-mode robot, characterized in that: The multi-mode robot has a conversion function between multiple modes, and the multi-mode robot in different modes has actuators of different forms. The method comprises the following steps: Acquire home environment data and personnel data of the area where the multi-mode robot is located; Determining whether there is a target home task according to the home environment data and the personnel data, wherein the target home task is at least one of the home tasks preset by the multi-mode robot; If it is determined that the target home task exists, determining the target mode of the multi-mode robot based on the target home task; The multi-mode robot is controlled to switch to the target mode, so that the multi-mode robot performs the target household task in the target mode.

2. The control method of the multi-mode robot according to claim 1, characterized in that: The acquiring of home environment data and personnel data of the area where the multi-mode robot is located includes: Acquiring multimodal data of an area where the multimodal robot is located; Based on the multimodal data, home environment data and personnel data of the area where the multimodal robot is located are determined.

3. The control method of the multi-mode robot according to claim 1, characterized in that: The determining whether there is a target home task according to the home environment data and the personnel data includes: Processing the home environment data and the personnel data through a behavior detection model to obtain the behavior intention of each person; Based on the behavioral intention, performing task matching processing in the household tasks preset by the multi-mode robot; If the match is successful, it is determined that the target home task exists, and the successfully matched home task is determined as the target home task; If the matching fails, it is determined that the target home task does not exist.

4. The control method of the multi-mode robot according to claim 3, characterized in that: The performing task matching processing in the household tasks preset by the multi-mode robot based on the behavioral intention includes: If the behavior intention is an intention to switch from a first behavior to a second behavior, then based on the first behavior, task matching processing is performed in the home tasks preset by the multi-mode robot, and the first behavior is before the second behavior; If the first behavior fails to match, task matching processing is performed in the home tasks preset by the multi-mode robot based on the second behavior.

5. The control method of the multi-mode robot according to claim 1, characterized in that: The step of determining a target mode of the multi-mode robot based on the target household task comprises: If the number of the target home tasks is one, determining the target mode corresponding to the target home tasks based on the correspondence between preset home tasks and modes; If there are multiple target home tasks, predict the revenue value when executing each target home task in each mode; Sorting the target home tasks from high to low according to the benefit values ​​to obtain the target home task sequence; Based on the target home task sequence, a target mode of the multi-mode robot is determined.

6. The control method of the multi-mode robot according to claim 5, characterized in that: The predicted benefit value when executing each of the target household tasks in each of the modes includes: Based on at least one of the time sensitivity of the target home task, the comfort level of personnel, and the energy efficiency of the multi-mode robot, the profit value when performing each of the target home tasks in each of the modes is predicted.

7. The control method of a multi-mode robot according to any one of claims 1 to 6, characterized in that: After determining the target mode of the multi-mode robot based on the target household task, the method further includes: Determine whether there is a preset scheduled task; If it is determined that there is a preset scheduled task, then determining whether the scheduled task conflicts with the target home task; If it is determined that there is a conflict between the scheduled task and the target home task, searching or planning a conflict resolution method between the scheduled task and the target home task; A target mode of the multimodal robot is adjusted based on the conflict resolution method.

8. A control device for a multi-mode robot, characterized in that: The multi-mode robot has a conversion function between multiple modes. The multi-mode robot in different modes has actuators of different forms. The control device of the multi-mode robot includes: An acquisition module, used to acquire home environment data and personnel data of an area where the multi-mode robot is located; A first processing module, configured to determine whether there is a target home task according to the home environment data and the personnel data, wherein the target home task is at least one of the home tasks preset by the multi-mode robot; A second processing module is configured to determine a target mode of the multi-mode robot based on the target home task if it is determined that the target home task exists; The control module is used to control the multi-mode robot to switch to the target mode, so that the multi-mode robot performs the target household task in the target mode.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the control method of the multi-mode robot according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the control method of the multi-mode robot according to any one of claims 1 to 7 are implemented.

Images