Work and rest assisting method, control device, equipment and storage medium of humanoid robot

Abstract

The present application discloses a work and rest assisting method, a control device, equipment and storage medium of a humanoid robot, the method comprising: determining a target time range for a target user to perform a target work and rest behavior based on the target user's historical work and rest data and schedule plan; if the current moment is within the target time range, obtaining a first user video for the target user; determining the first behavior of the target user based on the first user video; if the first behavior matches the target work and rest behavior, controlling the humanoid robot to execute a first behavior assisting strategy; if the first behavior does not match the target work and rest behavior, controlling the humanoid robot to play a first inquiry voice; obtaining the target user's feedback content for the first inquiry voice; sending a confirmation message to a parent terminal; if the response message indicates agreement with the first behavior, controlling the humanoid robot to execute a second behavior assisting strategy. The present application is conducive to meeting the personalized work and rest assisting needs of the target user and improving the user experience.

Description

Technical Field

[0001] The present application relates to the field of robots, and in particular to a work and rest assistance method, a control device, equipment and storage medium for a humanoid robot. Background Art

[0002] At present, more and more people develop the habit of staying up late and irregular work and rest, which leads to poor mental state the next day and affects people's work and study. For parents of students, they hope to help their children develop good work and rest habits to ensure their children's physical and mental health and improve their learning efficiency.

[0003] In some existing schedule assistance programs, parents can set specific times on some electronic devices. When the current time of the clock on the electronic device reaches the above time, a reminder sound corresponding to the time can be emitted to remind the child that the current time should be to get up or rest.

[0004] However, due to the differences in learning tasks and activity plans among different students, simply playing corresponding reminder sounds at specific times through electronic devices cannot meet the personalized schedule assistance needs of different students, and the user experience is poor. Summary of the invention

[0005] In order to solve the above-mentioned problems existing in the prior art, the embodiments of the present application provide a work and rest assistance method, a control device, equipment and storage medium of a humanoid robot, which determines the target time range for the target user to perform the target work and rest behavior according to the historical work and rest data and schedule plan of the target user, so as to determine the corresponding behavior assistance strategy according to the target user's behavior when the current moment is in the target time range to assist the user's work and rest, thereby meeting the target user's personalized work and rest assistance needs and improving the user experience.

[0006] In a first aspect, an embodiment of the present application provides a work and rest assistance method, which is applied to a controller of a humanoid robot, wherein the humanoid robot further includes a camera and a voice interaction module; the controller is communicatively connected to a parent terminal; and the method includes:

[0007] Acquire the historical work and rest data and schedule of the target user, wherein the schedule includes the activity plan and learning tasks of the target user after the current moment;

[0008] Based on the historical work and rest data and the schedule plan, determining a target time range for the target user to perform a target work and rest behavior;

[0009] If the current moment is within the target time range, obtaining a first user video of the target user within a first time period captured by the camera, wherein the first time period includes the current moment and a preset time period before the current moment;

[0010] Determining a first behavior of the target user at the current moment according to the first user video;

[0011] If the first behavior matches the target work-rest behavior, determining a first behavior assistance strategy corresponding to the target work-rest behavior, and controlling the humanoid robot to execute the first behavior assistance strategy;

[0012] If the first behavior does not match the target work and rest behavior, controlling the humanoid robot to play a first inquiry voice for the first behavior through the voice interaction module, wherein the first inquiry voice is used to inquire the target user about the reason for performing the first behavior;

[0013] Acquire feedback content of the target user in response to the first inquiry voice collected by the voice interaction module;

[0014] Sending confirmation information to the parent terminal, wherein the confirmation information is used to instruct the parent terminal to send response information corresponding to the feedback content;

[0015] If the response information indicates agreement with the first behavior, a second behavior assistance strategy corresponding to the first behavior is determined, and the humanoid robot is controlled to execute the second behavior assistance strategy.

[0016] In a second aspect, an embodiment of the present application provides a control device for a humanoid robot, which is applied to the humanoid robot, wherein the humanoid robot includes a camera and a voice interaction module; the device is communicatively connected with a parent terminal; the device includes an acquisition unit and a processing unit;

[0017] The acquisition unit is used to acquire the historical work and rest data and schedule of the target user, wherein the schedule includes the activity plan and learning tasks of the target user after the current moment;

[0018] The processing unit is used to determine a target time range for the target user to perform a target work and rest behavior based on the historical work and rest data and the schedule plan;

[0019] If the current moment is within the target time range, obtaining a first user video of the target user within a first time period captured by the camera, wherein the first time period includes the current moment and a preset time period before the current moment;

[0020] Determining a first behavior of the target user at the current moment according to the first user video;

[0021] If the first behavior matches the target work-rest behavior, determining a first behavior assistance strategy corresponding to the target work-rest behavior, and controlling the humanoid robot to execute the first behavior assistance strategy;

[0022] If the first behavior does not match the target work and rest behavior, controlling the humanoid robot to play a first inquiry voice for the first behavior through the voice interaction module, wherein the first inquiry voice is used to inquire the target user about the reason for performing the first behavior;

[0023] The acquisition unit is used to acquire the feedback content of the target user in response to the first inquiry voice collected by the voice interaction module;

[0024] The processing unit is used to send confirmation information to the parent terminal, wherein the confirmation information is used to instruct the parent terminal to send response information corresponding to the feedback content;

[0025] If the response information indicates agreement with the first behavior, a second behavior assistance strategy corresponding to the first behavior is determined, and the humanoid robot is controlled to execute the second behavior assistance strategy.

[0026] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory storing execution instructions, wherein the memory stores one or more programs; when the processor executes the execution instructions stored in the memory, the processor executes the method described in the first aspect.

[0027] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a control program for a humanoid robot, including execution instructions. When a processor of an electronic device executes the execution instructions, the processor executes the method described in the first aspect.

[0028] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute part or all of the steps described in the first aspect of the embodiment of the present application.

[0029] Implementing the embodiments of the present application has the following beneficial effects:

[0030] In an embodiment of the present application, the controller of the humanoid robot first obtains the historical work and rest data and schedule of the target user, and then, based on the historical work and rest data and schedule, determines the target time range for the target user to perform the target work and rest behavior. If the current moment is within the target time range, the first user video of the target user in the first time period collected by the camera is obtained, and the first time period includes the current moment and the preset time period before the current moment. Then, according to the first user video, the first behavior of the target user at the current moment is determined. If the first behavior matches the target work and rest behavior, the first behavior auxiliary strategy corresponding to the target work and rest behavior is determined, and the humanoid robot is controlled to execute the first behavior auxiliary strategy. If the first behavior does not match the target work and rest behavior, the humanoid robot is controlled to play the first inquiry voice for the first behavior through the voice interaction module to inquire the reason for the target user to perform the first behavior. Further, the feedback content of the target user for the first inquiry voice collected by the voice interaction module is obtained, and a confirmation message is sent to the parent terminal to instruct the parent terminal to send a response message corresponding to the feedback content. If the response information indicates agreement with the first behavior, the second behavior auxiliary strategy corresponding to the first behavior is determined, and the humanoid robot is controlled to execute the second behavior auxiliary strategy. In this way, based on the historical work and rest data and schedule of the target user, the personalized target time range for executing the target work and rest behavior can be determined. When the current time is within the target time range and the first behavior of the target user matches the target work and rest behavior, the humanoid robot is controlled to execute the first behavior assistance strategy corresponding to the target work and rest behavior to meet the personalized work and rest assistance of the target user. In addition, when the first behavior of the target user does not match the target work and rest behavior, the target user can be asked the reason for the first behavior, and based on the feedback content of the target user, confirmation information of the feedback content is sent to the parent terminal. When the response information returned by the parent terminal indicates agreement with the first behavior, the humanoid robot is controlled to execute the second behavior assistance strategy corresponding to the first behavior, thereby realizing personalized execution of the behavior assistance strategy based on the needs of the target user and the supervision confirmation of the parents, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 A schematic diagram of a work and rest assistance system provided in an embodiment of the present application;

[0033] Figure 2 A flowchart of a method for assisting a work and rest schedule provided in an embodiment of the present application;

[0034] Figure 3 A schematic diagram of a noise reduction method provided in an embodiment of the present application;

[0035] Figure 4 A schematic diagram of another noise reduction method provided in an embodiment of the present application;

[0036] Figure 5 A schematic diagram of a control device for a humanoid robot provided in an embodiment of the present application;

[0037] Figure 6 A schematic diagram of the structure of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or electronic device that includes a series of steps or units is not limited to the listed steps or units, but in an optional example also includes steps or units that are not listed, or in an optional example also includes other steps or units inherent to these processes, methods, products, or electronic devices.

[0040] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0041] First, see Figure 1 , Figure 1 This is a schematic diagram of a work and rest assistance system provided in an embodiment of the present application. Figure 1As shown, the work and rest assistance system includes: a humanoid robot 100, a database 110 and a user terminal 120. The humanoid robot 100 includes: a controller 101, a camera 102, a voice interaction module 103, an upper limb mechanical arm 104 and a lower limb mechanical leg 105.

[0042] The database 110 may be a server with a data storage function, such as a cloud server, a rack server, a data server, and the like. It may also be a memory with a data storage function, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, a compact disc read-only memory (CD-ROM) or other optical disc storage, a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, and the present application does not limit this. When the database 110 is a memory, the database 110 may be integrated into the humanoid robot 100, such as the controller 101 of the humanoid robot 100. In an embodiment of the present application, the database 110 is used to store the historical work and rest data of the target user, such as the daily work and rest activities and work and rest time of the target user.

[0043] Among them, the user terminal 120 can be an electronic device with communication functions, such as a smart phone, a personal computer, a tablet computer, and the like. The user terminal 120 can also be a remote control device of the humanoid robot 100, which can communicate with the controller 101 of the humanoid robot 100. The user terminal 120 sends a control instruction to the controller 101, instructing the controller 101 to control the humanoid robot 100 to perform the operation corresponding to the control instruction. In an embodiment of the present application, the user terminal 120 can be a parent terminal, and the controller 101 of the humanoid robot 100 can control the humanoid robot 100 to perform the corresponding operation by sending a confirmation message to the parent terminal and based on the response information of the parent terminal to the confirmation message. The user can also input a schedule on the user terminal 120, and the controller of the humanoid robot 100 can obtain the user's schedule from the user terminal 120.

[0044] Among them, the controller 101 of the humanoid robot 100 can be a processor with data processing and data caching functions, including: a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), etc. The controller 101 of the humanoid robot 100 can also be integrated with a communication processing unit for communicating with the database 110, the user terminal 120 and each unit module of the humanoid robot 100. The controller 101 can control the humanoid robot 100 to perform corresponding operations by issuing instructions and sensing information. The body trunk and structure of the humanoid robot 100 are similar to those of humans. The controller 101 of the humanoid robot 100 can control the mechanical joint movement of the upper limb mechanical arm 104 to simulate the grasping, stretching and other actions of human arms. The controller 101 of the humanoid robot 100 can control the mechanical joint movement of the lower limb mechanical leg 105 to simulate the walking, standing and other actions of humans. The camera 102 can collect video images within the field of view. Optionally, the camera 102 can also be provided with identification detection devices such as radar and infrared detectors. The voice interaction module 103 can include a microphone, a speaker, a sound receiving device, etc. The voice interaction module 103 is used to play audio and collect the user's voice to achieve voice interaction with the user. Optionally, the humanoid robot 100 also includes a variety of sensors, such as light sensors, temperature sensors, audio sensors, etc., which are not limited in this application.

[0045] It should be noted that in some existing work and rest assistance methods, parents usually set a specific time and a corresponding reminder tone on the electronic device. When the clock on the electronic device reaches a specific time, the electronic device will play the reminder tone to remind the child to perform the corresponding work and rest behavior. However, since each student may have different learning tasks and activity plans, and each study may also have additional plans at a specific time, simply playing the corresponding reminder tone at a specific time cannot meet the personalized work and rest assistance needs of each user, and the user experience is poor.

[0046] To this end, in an embodiment of the present application, a controller 101 of a humanoid robot 100 is applied to the above-mentioned work and rest assistance system, and the controller 101 of the humanoid robot 100 obtains the historical work and rest data and schedule of the target user; the controller 101 of the humanoid robot 100 determines the target time range for the target user to perform the target work and rest behavior based on the historical work and rest data and the schedule; if the current moment is within the target time range, the controller 101 of the humanoid robot 100 obtains the first user video of the target user within the first time period captured by the camera 102; the controller 101 of the humanoid robot 100 determines the first behavior of the target user at the current moment based on the first user video; if the first behavior matches the target work and rest behavior, the controller 101 of the humanoid robot 100 The controller 101 determines a first behavior assistance strategy corresponding to the target work and rest behavior, and controls the humanoid robot to execute the first behavior assistance strategy; if the first behavior does not match the target work and rest behavior, the controller 101 of the humanoid robot 100 controls the humanoid robot 100 to play a first inquiry voice for the first behavior through the voice interaction module 103; the controller 101 of the humanoid robot 100 obtains the feedback content of the target user for the first inquiry voice collected by the voice interaction module 103; the controller 101 of the humanoid robot 100 sends a confirmation message to the parent terminal; if the response message indicates agreement with the first behavior, the controller 101 of the humanoid robot 100 determines a second behavior assistance strategy corresponding to the first behavior, and controls the humanoid robot to execute the second behavior assistance strategy.

[0047] It can be seen that, applied to the above-mentioned work and rest assistance system, the controller of the humanoid robot can obtain the historical work and rest data of the target user from the database, and obtain the schedule of the target user from the user terminal, so as to determine the target time range for the target user to perform the target work and rest behavior based on the historical work and rest data and the schedule. Thus, when the current moment is in the target time range and the target user performs the target work and rest behavior, the controller of the humanoid robot can control the humanoid robot to perform the first behavior assistance strategy corresponding to the target work and rest behavior to meet the personalized work and rest assistance needs of the target user. In addition, when the first behavior of the target user at the current moment does not match the target work and rest behavior, the controller of the humanoid robot can also play the first inquiry voice to inquire the reason why the target user performs the first behavior, and send a confirmation message to the parent terminal. When the parent terminal agrees to the first behavior in response to the confirmation message, the humanoid robot is controlled to perform the second behavior assistance strategy corresponding to the first behavior, so as to further meet the personalized work and rest assistance needs of the target user and improve the user experience.

[0048] See also Figure 2 , Figure 2The flowchart of a method for assisting a work and rest schedule provided in an embodiment of the present application is shown in FIG. The method is applied to a controller of a humanoid robot, the humanoid robot further comprising a camera and a voice interaction module, and the controller is connected to a parent terminal for communication. Figure 2 As shown, the method includes but is not limited to the following steps:

[0049] 201: Obtain the historical work and rest data and schedule of the target user.

[0050] In an embodiment of the present application, the schedule includes the activity plan and learning tasks of the target user after the current moment. The activity plan may include, for example, math competitions, speeches, and the like. Learning tasks may include, for example, memorizing words, doing homework, course learning, and the like. The historical work and rest data includes the historical duration and historical time range of each work and rest behavior performed by the target user in a preset period before the current moment. For example, the preset period may be one month. Work and rest behaviors may include, for example, sleeping behavior, waking up behavior, activity behavior, learning behavior, and the like.

[0051] Specifically, the controller of the humanoid robot can obtain the historical work and rest data and schedule of the target user from the database. The controller of the humanoid robot can also obtain the schedule of the target user from the user terminal or the parent terminal.

[0052] 202: Based on the historical work and rest data and the schedule plan, determine a target time range for the target user to perform the target work and rest behavior.

[0053] In the embodiment of the present application, the target work-rest behavior may include, for example, sleeping behavior, waking up behavior, activity behavior, learning behavior, etc. The embodiment of the present application is mainly described by taking the target work-rest behavior as an example. The controller of the humanoid robot can determine the historical time range for the target user to perform the target work-rest behavior based on the historical work-rest data of the target user, and then adjust the historical time range according to the target user's schedule, thereby predicting the target time range for the target user to perform the target work-rest behavior.

[0054] Exemplarily, when the target work and rest behavior includes sleeping behavior, determining the target time range for the target user to perform the target work and rest behavior based on historical work and rest data and schedule plan may include:

[0055] Based on historical sleep and rest data, determine the historical sleep duration and historical sleep time range of the target user;

[0056] Based on the activity plan, determine the first fatigue value of the target user;

[0057] Determining a second fatigue value of the target user based on the learning task;

[0058] Determining a target sleep duration of a target user based on the first fatigue value and the second fatigue value;

[0059] Based on the historical sleep duration and the target sleep duration, the historical sleep time range is adjusted to obtain the target time range.

[0060] The first fatigue value is determined by a first preset difficulty of the activity plan and a first planned duration of the activity plan, and the second fatigue value is determined by a second preset difficulty of the learning task and a second planned duration of the learning task.

[0061] Specifically, the controller of the humanoid robot can determine the historical sleep duration and historical sleep time range of the target user from the historical work and rest data of the target user. Among them, the historical sleep duration can be, for example, the average daily sleep duration of the target user within a preset period. Optionally, before determining the historical sleep duration of the target user based on the historical work and rest data, the historical work and rest data needs to be preprocessed to remove abnormal data in the historical work and rest data. Specifically, the controller of the humanoid robot can obtain the initial average sleep duration of each day within a preset period, and remove the sleep duration whose difference with the initial average sleep duration within the preset period is greater than a first threshold, and the first threshold can be preset by the user. Determine the average sleep duration after removing the abnormal data, and obtain the above-mentioned daily average sleep duration, that is, the historical sleep duration. The historical sleep time range is the time range between the earliest sleep time and the latest sleep time of the target user within the preset period.

[0062] Then, the controller of the humanoid robot can predict the first fatigue value of the target user based on the first preset difficulty and the first planned duration of the activity plan. Specifically, based on the first planned duration, the first preset fatigue value can be determined, wherein the mapping relationship between the first planned duration and the first preset fatigue value can be preset. The first preset difficulty is used as the weight of the first preset fatigue value, and the first preset fatigue value is weighted to obtain the first fatigue value. For example, when the activity plan includes a math competition and a speech, assuming that the first preset difficulty of the math competition is D1 and the first planned duration is T1, and the first preset difficulty of the speech is D2 and the first planned duration is T2, if the first preset difficulty corresponding to T1 is k1, and the first preset difficulty corresponding to T2 is k2, then the first fatigue value is (D1 k1+D2 k2).

[0063] Similarly, the controller of the humanoid robot can predict the second fatigue value of the target user based on the second preset difficulty and the second planned duration of the learning task. Specifically, the second preset fatigue value can be determined based on the second planned duration, wherein the mapping relationship between the second planned duration and the second preset fatigue value can be preset. The second preset difficulty is used as the weight of the second preset fatigue value, and the second preset fatigue value is weighted to obtain the second fatigue value.

[0064] Furthermore, based on the first fatigue value and the second fatigue value, the target sleep duration of the target user can be predicted. Specifically, the humanoid robot can determine the fatigue value interval in which the sum of the first fatigue value and the second fatigue value is located, and based on the correspondence between the fatigue value interval and the sleep duration, determine the target sleep duration of the target user. The correspondence between the fatigue value interval and the sleep duration can be pre-set based on the test results of the user.

[0065] Finally, the controller of the humanoid robot can obtain the time difference between the historical sleep duration and the target sleep duration. Based on the time difference and the target user's schedule, the historical sleep time range is adjusted to obtain the target time range. Among them, if the schedule includes an early-rising plan, the maximum moment in the historical sleep time range is advanced by the above time difference to obtain the target time range. If the schedule does not include an early-rising plan, the maximum moment in the historical sleep time range is advanced by half of the above time difference, and the minimum moment in the historical sleep time range is delayed by half of the above time difference to obtain the target time range. For example, when the historical sleep duration is a hour, the target sleep duration is b hours, and the historical sleep time range is [x, y], if the schedule includes an early-rising plan, the maximum moment in the target time range is advanced by (ab) hours to obtain the target time range [x, y-(ab)]. If the schedule does not include a plan to get up early, advance the maximum time in the target time range by (ab) / 2 hours and delay the minimum time in the target time range by (ab) / 2 hours to obtain the target time range [x+(ab), y-(ab)].

[0066] It can be seen that the controller of the humanoid robot can determine the historical sleep duration and historical sleep time range of the target user based on the historical work and rest data of the target user. Then, based on the first preset difficulty and the first planned duration of the activity plan, the first fatigue value of the target user is determined, and based on the second preset difficulty and the second planned duration of the learning task, the second fatigue value of the target user is determined. Further, based on the first fatigue value and the second fatigue value, the target sleep duration of the target user can be determined, and then based on the historical sleep duration and the target sleep duration, the historical sleep time range is adjusted to obtain the target time range. Thus, the controller of the humanoid robot can determine a personalized target time range based on the historical work and rest data and schedule plan of the target user, so as to provide personalized work and rest assistance to the target user and improve the user experience.

[0067] 203: If the current time is within the target time range, obtain a first user video of the target user in the first time period captured by the camera.

[0068] In an embodiment of the present application, the first time period includes the current moment, and a preset time period before the current moment. The camera of the humanoid robot can collect image data within the field of view in real time. Optionally, the controller of the humanoid robot can perform target detection on the video collected by the camera to determine the position of the target user in the video. The target detection can be detected, for example, by a target detection model (You Only Look Once, YOLO). Based on the position of the target user in the first user's video, the rotation direction and rotation angle of the humanoid robot's head are determined, and based on the rotation direction and rotation angle, the rotation of the humanoid robot's head is controlled so that the target user is in the center area of ​​the video collected by the camera. For example, when the field of view angle of the camera is r degrees, and the resolution of the video collected by the camera is m When n is equal to 0, a coordinate system is established with the central pixel point of the video image as the origin, the horizontal right direction as the horizontal axis, and the vertical upward direction as the vertical axis. If the center of gravity of the target user in the video is located at the pixel coordinate (m / 2, 0), the rotation direction of the humanoid robot head is determined to be horizontal right direction, the rotation angle is r / 4 degrees, and the head of the humanoid robot is controlled to rotate horizontally to the right r / 4 degrees, so that the center of the target user is at the center of the video captured by the camera.

[0069] When the current moment is within the target time range, the controller of the humanoid robot can obtain a first user video of the target user in the first time period captured by the camera. Thus, the target user is in the central area of ​​the first user video in the obtained first user video, and based on the first user video, the first behavior of the target user can be determined more accurately.

[0070] 204: Determine a first behavior of the target user at the current moment according to the first user video.

[0071] In an embodiment of the present application, the controller of the humanoid robot performs behavior detection on the target user in the first user video to determine the first behavior of the target user at the current moment.

[0072] Specifically, the controller of the humanoid robot can determine the location of the target user by performing target detection on the first user video. The target detection can be performed using yolo detection, for example. Then, the controller of the humanoid robot will perform target tracking on the location to continuously mark the location of the target user. The target tracking can be achieved, for example, using a real-time online tracking algorithm deepsort. Finally, the controller of the humanoid robot will perform action recognition on the location to identify the first behavior of the target user. The action recognition can be achieved, for example, using a behavior recognition algorithm slowfast.

[0073] 205: If the first behavior matches the target work and rest behavior, a first behavior assistance strategy corresponding to the target work and rest behavior is determined, and the humanoid robot is controlled to execute the first behavior assistance strategy.

[0074] In an embodiment of the present application, when the current moment is within the target time range, if the first behavior matches the target work and rest behavior, it means that the target user is performing the target work and rest behavior. At this time, the controller of the humanoid robot will determine the first behavior assistance strategy corresponding to the target work and rest behavior, and control the humanoid robot to execute the first behavior assistance strategy to assist the target user in performing the target work and rest behavior.

[0075] Exemplarily, the humanoid robot further includes: a light sensor, a temperature sensor, an audio sensor, an upper limb mechanical arm, and a lower limb mechanical leg. Determining a first behavior assistance strategy corresponding to the target work and rest behavior, and controlling the humanoid robot to execute the first behavior assistance strategy, for example, may include:

[0076] Obtain target environmental parameters corresponding to target work and rest behaviors;

[0077] Get the current environment parameters of the room where the target user is located;

[0078] If the current light brightness is not within the target light brightness range, the humanoid robot is controlled to adjust the light switch through the upper limb mechanical arm to adjust the light brightness of the room to the target light brightness;

[0079] If the current temperature is not within the target temperature range, an adjustment signal is sent to the air conditioner in the room to adjust the room temperature to the target ambient temperature;

[0080] If the current noise volume value is greater than the noise threshold, the humanoid robot is controlled to move to the window position of the room through the lower limb mechanical legs;

[0081] Get the window opening angle captured by the camera;

[0082] Determine the target angle based on the current noise volume value, noise threshold and window opening angle;

[0083] The upper limb robotic arm of the humanoid robot is controlled to adjust the window of the room to a target angle so that the noise volume value in the room is less than or equal to the noise threshold.

[0084] In the embodiment of the present application, the target environment parameters include: target light brightness range, target temperature range and noise threshold. The current environment parameters include: current light brightness, current temperature and current noise volume value. The current light brightness is collected by a light sensor, the current temperature is collected by a temperature sensor, and the current noise volume value is collected by an audio sensor. The target light brightness is the light brightness within the target light brightness range, and the target environment temperature is the temperature within the target temperature range.

[0085] Specifically, the humanoid robot can obtain the target environment parameters corresponding to the target work and rest behavior, and the correspondence between the target work and rest behavior and the target environment parameters can be pre-set based on the user's historical work and rest data. Then, the current light brightness of the room where the target user is located collected by the light sensor, the current temperature of the room where the target user is located collected by the temperature sensor, and the current noise volume value of the room where the target user is located collected by the audio sensor are obtained, thereby obtaining the current environment parameters of the room.

[0086] It should be noted that the target environmental parameters are the range of environmental parameters that assist the target user in performing the target work and rest behaviors. When any of the current environmental parameters is not within the range of the environmental parameters, the controller of the humanoid robot will control the humanoid robot to adjust the environmental parameters.

[0087] If the current light brightness of the room is not within the target light brightness range, the controller of the humanoid robot will obtain the position of the light switch in the room, and control the lower limbs of the humanoid robot to drive the humanoid robot to move to the position of the light switch. After the humanoid robot reaches the position of the light switch, the upper limbs of the humanoid robot and the light are controlled to adjust the light brightness of the room. During the adjustment process, the light brightness of the room is collected in real time through the light sensor until the light brightness of the room is adjusted to the target light brightness, that is, the light brightness of the room is within the target light brightness range.

[0088] If the current temperature of the room is not within the target temperature range, the controller of the humanoid robot may send an adjustment signal to the air conditioner in the room, the adjustment signal including the target ambient temperature, which is the temperature closest to the current temperature within the target temperature range. For example, when the target temperature range is [23°C, 26°C], if the current temperature of the room is 30°C, the target ambient temperature is 26°C. Thus, after receiving the adjustment signal, the air conditioner may output the output power corresponding to the target ambient temperature to adjust the temperature of the room to the target ambient temperature.

[0089] If the current noise volume value of the room is greater than the noise threshold, the controller of the humanoid robot obtains the window position of the window in the room. Figure 3 As shown, the target user is sleeping, and the controller of the humanoid robot controls the humanoid robot to move to the window position, collects the window image in real time through the camera, and determines the window opening angle based on the window image. Then, based on the current noise volume value, the noise threshold and the window opening angle, the target angle is determined, wherein the noise volume value corresponding to the target angle can be any value less than or equal to the noise threshold. The ratio of the current noise volume value to the window opening angle is equal to the noise volume value corresponding to the target angle and the target angle.

[0090] Further, after determining the target angle, as Figure 3 As shown, the controller of the humanoid robot controls the upper limb mechanical arm of the humanoid robot to adjust the window opening angle. During the adjustment process, the current window opening angle of the window is obtained in real time through the camera until the window of the room is adjusted to the target angle so that the noise volume value of the room is less than or equal to the noise threshold.

[0091] It can be seen that in the embodiment of the present application, the controller of the humanoid robot can obtain the target environmental parameters of the target work and rest behavior, and obtain the current environmental parameters of the room where the target user is located. When the current environmental parameters do not match the target environmental parameters, the humanoid robot can be controlled to adjust the current environmental parameters to assist the target user in performing the target work and rest behavior, thereby meeting the target user's personalized work and rest assistance needs and improving the user experience.

[0092] In a possible embodiment, the method may further include:

[0093] If the windows of the room are closed and the current noise volume value is greater than the noise threshold, the noise parameters and the source direction of the noise are obtained;

[0094] Based on the noise parameter, determining an anti-phase sound wave having a phase opposite to that of the noise;

[0095] Determine the target direction relative to the source direction;

[0096] Determine the sleeping position of the target user when performing the sleeping behavior;

[0097] Determine the sound wave emission position based on the sleeping position and the target direction;

[0098] The humanoid robot is controlled to move to the sound wave emission position and emit anti-phase sound waves in the target direction to reduce the noise volume value in the room to a value less than or equal to the noise threshold.

[0099] The noise parameters include the phase and amplitude of the noise.

[0100] Specifically, when the camera detects that the window of the room is closed and the noise volume value collected by the audio sensor is greater than the noise threshold, the controller of the humanoid robot can continuously collect the noise parameters of the current noise and the source direction of the noise through the audio sensor. Then, based on the noise parameters, an anti-phase sound wave with an opposite phase and equal amplitude to the noise is determined. It should be noted that since the anti-phase sound wave has an opposite phase and equal amplitude to the noise, the anti-phase sound wave and the noise will cancel each other out and attenuate when they meet in the air, thereby achieving the effect of reducing the noise volume value.

[0101] It should be noted that since anti-phase sound waves also need to propagate in the air, in order to ensure that the anti-phase sound waves do not disturb the target user's target work and rest behaviors, the controller of the humanoid robot needs to first determine the sound wave emission position based on the source direction of the noise and the sleeping position of the target user, and control the humanoid robot to emit anti-phase sound waves through the voice interaction module at the sound wave emission position.

[0102] Specifically, the controller of the humanoid robot first determines the target direction relative to the source direction of the noise. Then, the sleeping position of the target user in the sleeping phase is obtained. If the sleeping position is not in the target direction of the humanoid robot, the controller of the humanoid robot uses the current position as the sound wave emission position. The voice interaction module emits an anti-phase sound wave in the target direction to reduce the noise volume value in the room to a value less than or equal to the noise threshold.

[0103] If the target user's sleeping position is in the target direction of the humanoid robot, the controller of the humanoid robot will determine the sound wave emission position according to the sleeping position and the target direction, so that when the humanoid robot is in the sound wave emission position, the target user's sleeping position is not in the target direction of the humanoid robot. Figure 4 As shown, after the humanoid robot determines the target direction relative to the source direction of the noise, the target user's sleeping position is now located in the target direction of the current position of the humanoid robot. In order not to disturb the target user's sleeping behavior, the controller of the humanoid robot can determine the sound wave emission position in the target direction and control the humanoid robot to move to the sound wave emission position so that the target user's sleeping position is not in the target direction of the humanoid robot. Finally, the humanoid robot is controlled to emit anti-phase sound waves in the target direction through the voice interaction module to reduce the noise volume value in the room to a value less than or equal to the noise threshold.

[0104] It can be seen that when the windows of the room are closed and the current noise volume value is still greater than the noise threshold, the controller of the humanoid robot can determine the anti-phase sound wave based on the noise parameters, and determine the sound wave emission position according to the source direction of the noise and the sleeping position of the target user, and control the humanoid robot to emit the anti-phase sound wave at the sound wave emission position to reduce the noise volume value in the room, thereby assisting the target user in sleeping behavior, meeting the target user's personalized work and rest assistance needs, and improving the user experience.

[0105] Exemplarily, controlling the humanoid robot to execute the first behavior assistance strategy may also include:

[0106] Get information about items in the room;

[0107] If there is a user terminal device in the room, obtain the power of the user terminal device;

[0108] If the power level is less than the power threshold, the upper limb mechanical arm of the humanoid robot is controlled to grab the user terminal device and connect the user terminal device to the charger;

[0109] If the power level is greater than or equal to the power threshold, the upper limb mechanical arm of the humanoid robot is controlled to grab the user terminal device and place the user terminal device in a preset storage location;

[0110] If no covering object is identified on the torso of the target user, the position information of the covering object is obtained through the camera;

[0111] Based on the position information of the covering object, the upper limb mechanical arm of the humanoid robot is controlled to cover the covering object on the torso of the target user.

[0112] Specifically, the controller of the humanoid robot performs target detection on the room video captured by the camera to obtain information about items in the room. If a user terminal device, such as a mobile phone or tablet computer, is identified in the room, a power acquisition instruction is sent to the user terminal device, and the return information of the user terminal device is received, which includes the power level of the user terminal device.

[0113] If the power level of the user terminal device is less than the power threshold, the humanoid robot is controlled to move to the location of the user terminal device through the lower limb mechanical legs, and the user terminal device at the location is grabbed through the upper limb mechanical arms. Then, the location of the charger matching the user terminal device in the room is obtained, and the humanoid robot is controlled to move to the location of the charger through the lower limb mechanical legs, and the user terminal device is connected to the charger to charge the user terminal device.

[0114] If the power level of the user terminal device is greater than or equal to the power threshold, the humanoid robot is controlled to move to the location of the user terminal device through the lower limb mechanical legs, and the user terminal device at that location is grabbed through the upper limb mechanical arms. Then, the humanoid robot is controlled to move to the preset storage location, and the user terminal device is placed in the preset storage location through the upper limb mechanical arms to prevent the radiation of the user terminal device from affecting the sleep of the target user, and further assist the target user in sleeping behavior.

[0115] If no covering object is identified on the torso of the target user, the covering object may include, for example, a quilt, a blanket, etc., the head of the humanoid robot is controlled to rotate to obtain the position information of the covering object in the room through the camera on the head. Then, based on the position information of the covering object, the upper limb mechanical arm of the humanoid robot is controlled to grab the covering object and cover the torso of the target user with the covering object, further assisting the target user in sleeping.

[0116] It can be seen that the controller of the humanoid robot can obtain information about items in the room. After identifying the user terminal device, the controller obtains the user terminal device. When the power is less than the power threshold, the controller controls the humanoid robot to connect the user terminal device to the charger to assist the target user in sleeping. When the power is greater than or equal to the power threshold, the controller controls the humanoid robot to place the user terminal device at a preset interview position to prevent the radiation generated by the user terminal device from affecting the target user's sleep, further assisting the target user in sleeping. When the target user's torso does not identify a covering object, the controller controls the humanoid robot to cover the target user's torso with the covering object, further assisting the target user in sleeping and improving the user experience.

[0117] Exemplarily, the humanoid robot further includes a respiratory rate sensor. The method further includes:

[0118] Obtain the breathing rate of the target user collected by the breathing rate sensor;

[0119] Determine the target user's sleep state based on breathing frequency;

[0120] If the sleep state is deep sleep, a prompt message is sent to the parent terminal;

[0121] If the sleep state is light sleep, the humanoid robot is controlled to play sleep-aiding audio.

[0122] In the embodiment of the present application, the sleep state includes deep sleep and light sleep, the prompt information includes the sleep state, and the sleep-aiding audio is used to assist the target user to have a deep sleep.

[0123] Specifically, when the target user is sleeping, the breathing frequency sensor of the humanoid robot can collect the breathing frequency of the target user in real time. Based on the breathing frequency of the target user, the sleeping state of the target user can be determined. For example, when the breathing frequency of the target user is greater than the second threshold, the sleeping state of the target user is determined to be light sleep. When the breathing frequency of the target user is less than or equal to the second threshold, the sleeping state of the target user is determined to be deep sleep.

[0124] If the sleep state is deep sleep, a prompt message is sent to the parent terminal to prompt the parent terminal that the target user is in the deep sleep state.

[0125] If the sleep state is light sleep, the controller of the humanoid robot will control the humanoid robot to play sleep-aiding audio to assist the target user to enter a deep sleep state.

[0126] It can be seen that in the embodiment of the present application, the controller of the humanoid robot can obtain the breathing frequency of the target user when the target user is performing sleep behavior, and determine the sleep state of the target user based on the breathing frequency of the target user, so as to control the humanoid robot to execute corresponding auxiliary strategies based on the sleep state of the target user to assist the target user in performing sleep behavior and improve the user experience.

[0127] 206: If the first behavior does not match the target work and rest behavior, control the humanoid robot to play a first inquiry voice for the first behavior through the voice interaction module.

[0128] In the embodiment of the present application, the first inquiry voice is used to inquire the target user about the reason for performing the first behavior. When the first behavior does not match the target work and rest behavior, the controller of the humanoid robot controls the humanoid robot to play the first inquiry voice for the first behavior through the voice interaction module to inquire the target user about the reason for performing the first behavior.

[0129] 207: Acquire feedback content of the target user collected by the voice interaction module in response to the first inquiry voice.

[0130] In an embodiment of the present application, the voice interaction module can receive in real time the feedback content of the target user in response to the first inquiry voice. The feedback content can be voice feedback. The target user can also input the feedback content through the user terminal device, which is sent by the user terminal device to the controller of the humanoid robot.

[0131] 208: Send confirmation information to the parent terminal.

[0132] In the embodiment of the present application, the confirmation information is used to instruct the parent terminal to send response information corresponding to the feedback content. The humanoid robot can send the confirmation information to the parent terminal to confirm whether the parent terminal agrees with the first behavior of the target user.

[0133] 209: If the response information indicates agreement with the first behavior, a second behavior auxiliary strategy corresponding to the first behavior is determined, and the humanoid robot is controlled to execute the second behavior auxiliary strategy.

[0134] In an embodiment of the present application, when the response information sent by the parent terminal indicates agreement with the first behavior, the controller of the humanoid robot can determine a second behavior assistance strategy corresponding to the first behavior, and control the humanoid robot to execute the second behavior assistance strategy to assist the target user in performing the first behavior.

[0135] Exemplarily, determining a second behavior auxiliary strategy corresponding to the first behavior and controlling the humanoid robot to execute the second behavior auxiliary strategy may include:

[0136] Controlling the humanoid robot to play a second inquiry voice through the voice interaction module;

[0137] Obtaining the first reply content of the target user to the second inquiry voice;

[0138] If the first reply indicates that the massage operation is agreed, the third inquiry voice is played through the voice interaction module;

[0139] Obtaining a second reply content of the target user to the third inquiry voice;

[0140] In response to the second reply content, the finger joints of the upper limb mechanical arm of the humanoid robot are controlled to move so as to massage the target part of the target user.

[0141] In the embodiment of the present application, the second inquiry voice is used to inquire the target user whether to perform a massage operation, the third inquiry voice is used to inquire the target user about the body part to be massaged, and the second reply content includes the target part. The humanoid robot can ask the target user whether to perform a massage operation to assist the target user in performing the first behavior.

[0142] Specifically, the controller of the humanoid robot can control the voice interaction module to play the second inquiry voice to inquire whether the target user performs the massage operation. The voice interaction module receives the first reply content of the target user to the second inquiry voice. The first reply content is semantically recognized.

[0143] If the first reply indicates that the massage operation is approved, a third inquiry voice is played through the voice interaction module to inquire the target user about the body part to be massaged. A second reply from the target user to the third inquiry voice is received through the voice interaction module.

[0144] In response to the second reply content, semantic recognition is performed on the second reply content to determine the target part of the target user to be massaged. The finger joints of the upper limb mechanical arm of the humanoid robot are controlled to move to massage the target part of the target user with a preset strength. Optionally, a fourth inquiry voice can be played through the voice interaction module to inquire about the massage strength required by the target user. After receiving the third reply content of the target user, the finger joints of the humanoid robot are controlled to massage the target part with the massage strength corresponding to the third reply content.

[0145] It can be seen that the controller of the humanoid robot can ask the target user whether to perform a massage operation when the target user performs the first behavior. After the target user agrees, the controller can ask the target user which part needs to be massaged so as to massage the target part of the target user, thereby assisting the target user to perform the first behavior, meeting the target user's personalized work and rest assistance needs, and improving the user experience.

[0146] Optionally, if the response information sent by the parent terminal indicates opposition to the first behavior, a prompt voice is played through the voice interaction module, and the prompt voice includes the response information sent by the parent terminal to prompt the target user to perform the target work and rest behavior. The second user video of the second time period captured by the camera is obtained, and the second time period is the playback time of the prompt voice, and the preset time period after the playback time. Based on the second user video, the second behavior of the target user is determined. If the second behavior does not match the target work and rest behavior, a persuasion message is sent to the parent terminal to prompt the parent to persuade the target user to perform the target work and rest behavior to improve the user experience.

[0147] In summary, in the embodiment of the present application, the controller of the humanoid robot first obtains the historical work and rest data and schedule of the target user, and then, based on the historical work and rest data and schedule, determines the target time range for the target user to perform the target work and rest behavior. If the current moment is within the target time range, the first user video for the target user in the first time period collected by the camera is obtained, and the first time period includes the current moment and the preset time period before the current moment. Then, according to the first user video, the first behavior of the target user at the current moment is determined. If the first behavior matches the target work and rest behavior, the first behavior auxiliary strategy corresponding to the target work and rest behavior is determined, and the humanoid robot is controlled to execute the first behavior auxiliary strategy. If the first behavior does not match the target work and rest behavior, the humanoid robot is controlled to play the first inquiry voice for the first behavior through the voice interaction module to inquire the reason for the target user to perform the first behavior. Further, the feedback content of the target user for the first inquiry voice collected by the voice interaction module is obtained, and a confirmation message is sent to the parent terminal to instruct the parent terminal to send a response message corresponding to the feedback content. If the response information indicates agreement with the first behavior, the second behavior auxiliary strategy corresponding to the first behavior is determined, and the humanoid robot is controlled to execute the second behavior auxiliary strategy. In this way, based on the historical work and rest data and schedule of the target user, the personalized target time range for executing the target work and rest behavior can be determined. When the current time is within the target time range and the first behavior of the target user matches the target work and rest behavior, the humanoid robot is controlled to execute the first behavior assistance strategy corresponding to the target work and rest behavior to meet the personalized work and rest assistance of the target user. In addition, when the first behavior of the target user does not match the target work and rest behavior, the target user can be asked the reason for the first behavior, and based on the feedback content of the target user, confirmation information of the feedback content is sent to the parent terminal. When the response information returned by the parent terminal indicates agreement with the first behavior, the humanoid robot is controlled to execute the second behavior assistance strategy corresponding to the first behavior, thereby realizing personalized execution of the behavior assistance strategy based on the needs of the target user and the supervision confirmation of the parents, thereby improving the user experience.

[0148] See also Figure 5 , Figure 5The schematic diagram of a control device of a humanoid robot provided in an embodiment of the present application, the control device 500 of the humanoid robot can be the controller of the humanoid robot in any of the above embodiments. The control device 500 of the humanoid robot is applied to the humanoid robot, and the humanoid robot also includes a camera and a voice interaction module; the control device 500 of the humanoid robot is connected to the parent terminal for communication; the control device 500 of the humanoid robot includes an acquisition unit 501 and a processing unit 502;

[0149] The acquisition unit 501 is used to acquire the historical schedule data and schedule plan of the target user, where the schedule plan includes the activity plan and learning tasks of the target user after the current moment;

[0150] Processing unit 502, for determining a target time range for a target user to perform a target work and rest behavior based on historical work and rest data and schedule plan;

[0151] If the current moment is within the target time range, a first user video of the target user within a first time period captured by the camera is obtained, where the first time period includes the current moment and a preset time period before the current moment;

[0152] Determine the first behavior of the target user at the current moment according to the first user video;

[0153] If the first behavior matches the target work-rest behavior, determining a first behavior assistance strategy corresponding to the target work-rest behavior, and controlling the humanoid robot to execute the first behavior assistance strategy;

[0154] If the first behavior does not match the target work and rest behavior, the humanoid robot is controlled to play a first inquiry voice for the first behavior through the voice interaction module, where the first inquiry voice is used to inquire the target user about the reason for performing the first behavior;

[0155] An acquisition unit 501 is used to acquire the feedback content of the target user in response to the first inquiry voice collected by the voice interaction module;

[0156] The processing unit 502 is used to send confirmation information to the parent terminal, where the confirmation information is used to instruct the parent terminal to send response information corresponding to the feedback content;

[0157] If the response information indicates agreement with the first behavior, a second behavior assistance strategy corresponding to the first behavior is determined, and the humanoid robot is controlled to execute the second behavior assistance strategy.

[0158] In a possible embodiment, when the target work and rest behavior includes sleeping behavior, in determining the target time range for the target user to perform the target work and rest behavior based on the historical work and rest data and the schedule plan, the processing unit 502 is specifically configured to:

[0159] Based on historical sleep and rest data, determine the historical sleep duration and historical sleep time range of the target user;

[0160] Based on the activity plan, determining a first fatigue value of the target user, where the first fatigue value is determined by a first preset difficulty of the activity plan and a first planned duration of the activity plan;

[0161] Based on the learning task, determining a second fatigue value of the target user, where the second fatigue value is determined by a second preset difficulty of the learning task and a second planned duration of the learning task;

[0162] Determining a target sleep duration of the target user based on the first fatigue value and the second fatigue value;

[0163] Based on the historical sleep duration and the target sleep duration, the historical sleep time range is adjusted to obtain the target time range.

[0164] In a possible embodiment, the humanoid robot further includes: a light sensor, a temperature sensor, an audio sensor, an upper limb mechanical arm, and a lower limb mechanical leg; in determining a first behavior assistance strategy corresponding to the target work and rest behavior and controlling the humanoid robot to execute the first behavior assistance strategy, the processing unit 502 is specifically used to:

[0165] Obtain target environment parameters corresponding to the target work and rest behavior, the target environment parameters including: target light brightness range, target temperature range and noise threshold;

[0166] Obtain the current environmental parameters of the room where the target user is located. The current environmental parameters include: current light brightness, current temperature, and current noise volume value. The current light brightness is collected by a light sensor, the current temperature is collected by a temperature sensor, and the current noise volume value is collected by an audio sensor;

[0167] If the current light brightness is not within the target light brightness range, the humanoid robot is controlled to adjust the light switch through the upper limb mechanical arm to adjust the light brightness of the room to the target light brightness, and the target light brightness is the light brightness within the target light brightness range;

[0168] If the current temperature is not within the target temperature range, an adjustment signal is sent to the air conditioner in the room to adjust the temperature of the room to the target ambient temperature, which is a temperature within the target temperature range;

[0169] If the current noise volume value is greater than the noise threshold, the humanoid robot is controlled to move to the window position of the room through the lower limb mechanical legs;

[0170] Get the window opening angle captured by the camera;

[0171] Determine the target angle based on the current noise volume value, noise threshold and window opening angle;

[0172] The upper limb robotic arm of the humanoid robot is controlled to adjust the window of the room to a target angle so that the noise volume value in the room is less than or equal to the noise threshold.

[0173] In a possible embodiment, the processing unit 502 is further configured to:

[0174] If the windows of the room are closed and the current noise volume value is greater than the noise threshold, the noise parameters and the source direction of the noise are obtained. The noise parameters include the phase and amplitude of the noise.

[0175] Based on the noise parameter, determining an anti-phase sound wave having a phase opposite to that of the noise;

[0176] Determine the target direction relative to the source direction;

[0177] Determine the sleeping position of the target user when performing the sleeping behavior;

[0178] Determine the sound wave emission position based on the sleeping position and the target direction;

[0179] The humanoid robot is controlled to move to the sound wave emission position and emit anti-phase sound waves in the target direction to reduce the noise volume value in the room to a value less than or equal to the noise threshold.

[0180] In a possible embodiment, in controlling the humanoid robot to execute the first behavior assistance strategy, the processing unit 502 is further configured to:

[0181] Get information about items in the room;

[0182] If there is a user terminal device in the room, obtain the power of the user terminal device;

[0183] If the power level is less than the power threshold, the upper limb mechanical arm of the humanoid robot is controlled to grab the user terminal device and connect the user terminal device to the charger;

[0184] If the power level is greater than or equal to the power threshold, the upper limb mechanical arm of the humanoid robot is controlled to grab the user terminal device and place the user terminal device in a preset storage location;

[0185] If no covering object is identified on the torso of the target user, the position information of the covering object is obtained through the camera;

[0186] Based on the position information of the covering object, the upper limb mechanical arm of the humanoid robot is controlled to cover the covering object on the torso of the target user.

[0187] In a possible embodiment, the humanoid robot further includes a respiratory rate sensor; and the processing unit 502 is further configured to:

[0188] Obtain the breathing rate of the target user collected by the breathing rate sensor;

[0189] Based on the breathing frequency, determine the target user's sleep state, which includes deep sleep and light sleep;

[0190] If the sleep state is deep sleep, a prompt message is sent to the parent terminal, and the prompt message includes the sleep state;

[0191] If the sleep state is light sleep, the humanoid robot is controlled to play sleep-aiding audio, and the sleep-aiding audio is used to assist the target user to have a deep sleep.

[0192] In a possible embodiment, in determining a second behavior auxiliary strategy corresponding to the first behavior and controlling the humanoid robot to execute the second behavior auxiliary strategy, the processing unit 502 is specifically configured to:

[0193] Controlling the humanoid robot to play a second inquiry voice through the voice interaction module, where the second inquiry voice is used to inquire the target user whether to perform a massage operation;

[0194] Obtaining the first reply content of the target user to the second inquiry voice;

[0195] If the first reply indicates that the massage operation is agreed, a third inquiry voice is played through the voice interaction module, and the third inquiry voice is used to inquire the target user about the body part to be massaged;

[0196] Obtaining a second reply content of the target user to the third inquiry voice, wherein the second reply content includes the target part;

[0197] In response to the second reply content, the finger joints of the upper limb mechanical arm of the humanoid robot are controlled to move so as to massage the target part of the target user.

[0198] See also Figure 6 , Figure 6 A schematic diagram of the structure of an electronic device proposed in an embodiment of the present application is shown in FIG. Figure 6 As shown, the electronic device 600 includes a processor 610, a memory 620, a communication interface 630, and one or more programs 621. The one or more programs 621 are stored in the memory 620 and are configured to be executed by the processor 610. The processor 610, the memory 620, and the communication interface 630 are interconnected and perform communication work with each other. Among them, the electronic device 600 may include a humanoid robot in any of the above embodiments.

[0199] The memory 620 may be a volatile memory such as a dynamic random access memory DRAM, or a non-volatile memory such as a mechanical hard disk. The memory 620 is used to store a set of executable program codes, and the processor 610 is used to call one or more programs 621 stored in the memory 620, and may execute part or all of the steps of any work and rest assisting method recorded in the control method embodiment of the humanoid robot.

[0200] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps of any method described in the above method embodiments, and the above computer includes an electronic device.

[0201] The embodiment of the present application also provides a computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, the computer program is operable to cause a computer to execute some or all of the steps of any method described in the method embodiment. The computer program product may be a software installation package, and the computer includes an electronic device.

[0202] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0203] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0204] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0205] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0206] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0207] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions to enable a computer electronic device (which can be a personal computer, electronic device or network electronic device, etc.) to perform all or part of the steps of the above-mentioned methods in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0208] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable memory, which may include a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0209] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for assisting a person in doing something, characterized in that: A controller applied to a humanoid robot, the humanoid robot also comprising a camera and a voice interaction module; The controller is connected to the parent terminal for communication; the method comprises: Acquire the historical work and rest data and schedule of the target user, wherein the schedule includes the activity plan and learning tasks of the target user after the current moment; Based on the historical work and rest data and the schedule plan, determine the target time range for the target user to perform the target work and rest behavior, specifically including: based on the historical work and rest data, determine the historical sleep duration and historical sleep time range of the target user; based on the activity plan, determine the first fatigue value of the target user, the first fatigue value is determined by the first preset difficulty of the activity plan and the first planned duration of the activity plan; based on the learning task, determine the second fatigue value of the target user, the second fatigue value is determined by the second preset difficulty of the learning task and the second planned duration of the learning task; based on the first fatigue value and the second fatigue value, determine the target sleep duration of the target user; based on the historical sleep duration and the target sleep duration, adjust the historical sleep time range to obtain the target time range; the target work and rest behavior includes sleep behavior; If the current moment is within the target time range, obtaining a first user video of the target user within a first time period captured by the camera, wherein the first time period includes the current moment and a preset time period before the current moment; Determining a first behavior of the target user at the current moment according to the first user video; If the first behavior matches the target work-rest behavior, determining a first behavior assistance strategy corresponding to the target work-rest behavior, and controlling the humanoid robot to execute the first behavior assistance strategy; If the first behavior does not match the target work and rest behavior, controlling the humanoid robot to play a first inquiry voice for the first behavior through the voice interaction module, wherein the first inquiry voice is used to inquire the target user about the reason for performing the first behavior; Acquire feedback content of the target user in response to the first inquiry voice collected by the voice interaction module; Sending confirmation information to the parent terminal, wherein the confirmation information is used to instruct the parent terminal to send response information corresponding to the feedback content; If the response information indicates agreement with the first behavior, a second behavior assistance strategy corresponding to the first behavior is determined, and the humanoid robot is controlled to execute the second behavior assistance strategy.

2. The method according to claim 1, characterized in that The humanoid robot further includes: a light sensor, a temperature sensor, an audio sensor, an upper limb mechanical arm, and a lower limb mechanical leg; the first behavior assistance strategy corresponding to the target work and rest behavior is determined, and the humanoid robot is controlled to execute the first behavior assistance strategy, including: Acquire target environment parameters corresponding to the target work and rest behavior, the target environment parameters including: target light brightness range, target temperature range and noise threshold; Acquire current environmental parameters of the room where the target user is located, the current environmental parameters including: current light brightness, current temperature and current noise volume value, the current light brightness is collected by the light sensor, the current temperature is collected by the temperature sensor, and the current noise volume value is collected by the audio sensor; If the current light brightness is not within the target light brightness range, controlling the humanoid robot to adjust the light switch through the upper limb mechanical arm to adjust the light brightness of the room to the target light brightness, wherein the target light brightness is the light brightness within the target light brightness range; If the current temperature is not within the target temperature range, sending an adjustment signal to the air conditioner in the room to adjust the temperature of the room to a target ambient temperature, the target ambient temperature being a temperature within the target temperature range; If the current noise volume value is greater than the noise threshold, controlling the humanoid robot to move to the window position where the window of the room is located through the lower limb mechanical legs; Obtaining the opening angle of the window captured by the camera; Determining a target angle based on the current noise volume value, the noise threshold and the window opening angle; The upper limb mechanical arm of the humanoid robot is controlled to adjust the window of the room to a target angle so that the noise volume value in the room is less than or equal to the noise threshold.

3. The method according to claim 2, characterized in that The method further comprises: If the window of the room is closed and the current noise volume value is greater than the noise threshold, then obtaining noise parameters and the source direction of the noise, wherein the noise parameters include the phase and amplitude of the noise; Based on the noise parameter, determining an anti-phase sound wave having a phase opposite to that of the noise; determining a target direction relative to the source direction; determining a sleeping position of the target user when performing the sleeping behavior; determining a sound wave emission position according to the sleeping position and the target direction; The humanoid robot is controlled to move to the sound wave emission position, and emit the anti-phase sound wave in the target direction, so as to reduce the noise volume value of the room to a value less than or equal to the noise threshold.

4. The method according to claim 2 or 3, characterized in that: The controlling the humanoid robot to execute the first behavior assistance strategy also includes: Obtaining information about items in the room; If there is a user terminal device in the room, obtaining the power of the user terminal device; If the power level is less than the power threshold, controlling the upper limb mechanical arm of the humanoid robot to grab the user terminal device and connect the user terminal device to the charger; If the power level is greater than or equal to the power threshold, control the upper limb mechanical arm of the humanoid robot to grab the user terminal device and place the user terminal device in a preset storage position; If no covering object is identified on the torso of the target user, obtaining the position information of the covering object through a camera; Based on the position information of the covering object, the upper limb mechanical arm of the humanoid robot is controlled to cover the torso of the target user with the covering object.

5. The method according to claim 4, characterized in that The humanoid robot further includes a respiratory rate sensor; and the method further includes: Acquire the breathing frequency of the target user collected by the breathing frequency sensor; Based on the breathing frequency, determining a sleep state of the target user, wherein the sleep state includes deep sleep and light sleep; If the sleep state is the deep sleep, sending a prompt message to the parent terminal, the prompt message including the sleep state; If the sleeping state is the light sleep, the humanoid robot is controlled to play sleep-aiding audio, where the sleep-aiding audio is used to assist the target user to achieve the deep sleep.

6. The method according to claim 1, characterized in that The determining a second behavior auxiliary strategy corresponding to the first behavior, and controlling the humanoid robot to execute the second behavior auxiliary strategy, comprises: Controlling the humanoid robot to play a second inquiry voice through the voice interaction module, where the second inquiry voice is used to inquire the target user whether to perform a massage operation; Obtaining a first reply content of the target user to the second inquiry voice; If the first reply content indicates agreement with the massage operation, a third inquiry voice is played through the voice interaction module, where the third inquiry voice is used to inquire about the body part of the target user to be massaged; Acquire a second reply content of the target user to the third inquiry voice, wherein the second reply content includes a target part; In response to the second reply content, the finger joints of the upper limb mechanical arm of the humanoid robot are controlled to move so as to massage the target part of the target user.

7. A control device for a humanoid robot, characterized in that: Applied to a humanoid robot, the humanoid robot includes a camera and a voice interaction module; the device is connected to a parent terminal for communication; the device includes an acquisition unit and a processing unit; The acquisition unit is used to acquire the historical work and rest data and schedule of the target user, wherein the schedule includes the activity plan and learning tasks of the target user after the current moment; The processing unit is used to determine the target time range for the target user to perform the target work and rest behavior based on the historical work and rest data and the schedule plan, and is specifically used to: determine the historical sleep duration and historical sleep time range of the target user based on the historical work and rest data; determine the first fatigue value of the target user based on the activity plan, and the first fatigue value is determined by the first preset difficulty of the activity plan and the first planned duration of the activity plan; Based on the learning task, determining a second fatigue value of the target user, where the second fatigue value is determined by a second preset difficulty of the learning task and a second planned duration of the learning task; based on the first fatigue value and the second fatigue value, determining a target sleep duration of the target user; Based on the historical sleep duration and the target sleep duration, adjusting the historical sleep time range to obtain the target time range; the target work and rest behavior includes sleep behavior; If the current moment is within the target time range, obtaining a first user video of the target user within a first time period captured by the camera, wherein the first time period includes the current moment and a preset time period before the current moment; Determining a first behavior of the target user at the current moment according to the first user video; If the first behavior matches the target work-rest behavior, determining a first behavior assistance strategy corresponding to the target work-rest behavior, and controlling the humanoid robot to execute the first behavior assistance strategy; If the first behavior does not match the target work and rest behavior, controlling the humanoid robot to play a first inquiry voice for the first behavior through the voice interaction module, wherein the first inquiry voice is used to inquire the target user about the reason for performing the first behavior; The acquisition unit is used to acquire the feedback content of the target user in response to the first inquiry voice collected by the voice interaction module; The processing unit is used to send confirmation information to the parent terminal, wherein the confirmation information is used to instruct the parent terminal to send response information corresponding to the feedback content; If the response information indicates agreement with the first behavior, a second behavior assistance strategy corresponding to the first behavior is determined, and the humanoid robot is controlled to execute the second behavior assistance strategy.

8. An electronic device, characterized in that: The method comprises a processor and a memory storing execution instructions, wherein the memory stores one or more programs; when the processor executes the execution instructions stored in the memory, the processor executes the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: A control program for a humanoid robot is stored, including execution instructions. When a processor of an electronic device executes the execution instructions, the processor executes the method as described in any one of claims 1-6.

Images