Humanoid robot control method

Abstract

The invention relates to a control technology of a humanoid robot, in particular to a humanoid robot control method which comprises the following steps: acquiring image information in a target area, identifying the position and posture of a to-be-grabbed object through a convolutional neural network model, and determining the position and posture of the to-be-grabbed object; and the robot arm is controlled to move to the position above the to-be-grabbed object, the posture of the end effector is adjusted to be matched with the object, the grabbing action is executed after the object state is captured and confirmed, the grabbed object is carried to a designated placement area, the optimal placement position is determined in the placement area, and the objects are placed according to a preset sequence. The robot achieves the effects of accurate recognition, stable grabbing and efficient carrying, and is particularly suitable for operation requirements in a complex environment.

Description

Technical Field

[0001] The present application relates to the technical field of humanoid robot control, and in particular to a humanoid robot control method. Background Art

[0002] Currently, in the automated warehousing industry, the introduction of robotic systems has greatly improved work efficiency and accuracy, especially in the handling and sorting of large-scale goods. These robots not only reduce labor costs, but also improve the safety and reliability of operations. However, with the increasing diversification and complexity of market demand, existing robot control technology faces new challenges, especially in terms of increasing demand for flexibility and adaptability.

[0003] Traditional solutions mainly include pre-programmed paths and sensor-assisted object detection mechanisms. Specifically, these robots usually navigate using pre-set routes and rely on sensors (such as lidar, cameras, etc.) to detect and identify objects in the working environment. In actual operation, these robots will complete specific tasks according to preset instructions, such as moving along a fixed trajectory, grabbing items at a specified location and placing them at another specified location. In addition, some advanced systems will also combine machine learning algorithms to improve the accuracy of object recognition and grasping by training models.

[0004] Although the above methods have solved the basic automation needs to a certain extent, they still have obvious limitations when facing scenarios with uncertain positions of items in dynamic environments. Especially in complex logistics sorting centers or manufacturing production lines, due to the wide variety of items and their non-fixed placement, traditional robots often find it difficult to accurately identify and grasp each target item, let alone place them in an orderly manner according to the predetermined order. This leads to low efficiency and prone to errors when performing complex tasks, thus affecting the overall operational effect. Summary of the invention

[0005] The purpose of this application is to overcome the above technical problems and provide a humanoid robot control method.

[0006] A humanoid robot control method comprises the following steps: a) acquiring image information in a target area; b) processing the acquired image information to identify the position and posture of an object to be grasped; c) controlling the robot arm to move above the object to be grasped, and adjusting the posture of an end effector to match the posture of the object to be grasped; d) capturing and confirming the state of the object to be grasped, and if the state meets a preset condition, executing the next operation; otherwise, returning to step b) to continue recognition; e) executing a grasping action to lift the object to be grasped; f) carrying the grasped object to a designated placement area; g) determining a suitable placement position in the designated placement area, and placing the carried object in a predetermined order.

[0007] By adopting the above technical solutions, humanoid robots can more flexibly adapt to the needs of object recognition and grasping in different environments. First, by processing the image information in the target area, the robot can accurately identify the position and posture of the object to be grasped, thereby avoiding grasping failures caused by uncertain position. Secondly, by adjusting the posture of the end effector in real time, it is ensured that each grasping can accurately match the actual state of the object, thereby improving the success rate and stability of grasping. In addition, this method also takes into account the path planning and placement selection during the handling process, ensuring that the objects will not be damaged during the handling process, and are placed in an orderly manner in the designated area according to the predetermined order, which significantly improves the overall work efficiency and accuracy. Preferably, the specific method of processing the image information in step b) is: using a convolutional neural network model to extract feature points in the image, and calculating the position and posture of the object to be grasped through the geometric relationship between the feature points, wherein the distance and angle relationship between the feature points are calculated by the formulas \( d = \sqrt{(x_2 x_1)^2 + (y_2 y_1)^2} \) and \( \theta = \arctan\left(\frac{y_2 y_1}{x_2 x_1}\right) \) , and the specific steps are as follows: i) inputting the image information into the convolutional neural network model to extract key feature points; ii) using the above formula to calculate the distance and angle between each feature point; iii) determining the precise position and posture of the object to be grasped based on the calculation results.

[0008] By adopting the above technical solutions, the humanoid robot's ability to recognize objects and judge postures in complex environments is improved, especially in cases where light changes greatly or there are obstructions. The robot can more accurately determine the position and posture of the object to be grasped, thereby improving the success rate and stability of grasping.

[0009] Preferably, the convolutional neural network model is trained at multiple levels and can recognize changes in various types of objects and their different postures, and is particularly suitable for environments with large changes in light. The specific training steps are as follows: i) Collect a large number of different types of object images, covering various lighting conditions and pose changes; ii) Use annotation tools to identify the feature points in each image; iii) Input the labeled images into the convolutional neural network model for training, and continuously iterate and optimize the model parameters; iv) Test the model’s performance on new images to ensure its accuracy and robustness.

[0010] By adopting the above technical solutions, after multi-level training, the convolutional neural network model can not only recognize the changes of various types of objects and their different postures, but also has strong robustness, especially in environments with large changes in light, it can still maintain a high recognition accuracy. This enables the robot to accurately identify and grasp target objects under complex lighting conditions, improving the reliability and adaptability of the overall operation.

[0011] Preferably, the method of adjusting the posture of the end effector in step c) is: based on the visual feedback system, the actual position and posture of the object to be grasped are monitored in real time, and the angle and height of the end effector are dynamically adjusted according to the monitoring results. Specifically, the dynamic adjustment is achieved through a PID controller, and the controller parameters \( K_p, K_i, K_d \) are set to 0.5, 0.1, and 0.2 respectively. The specific steps are as follows: i) Install cameras on the head and hands of the robot to collect multi-view image information; ii) Transmit image information to the central processing unit in real time; iii) The central processing unit calculates the actual position and posture of the object to be grasped based on the image information; iv) Adjust the angle and height of the end effector to match the actual situation of the object to be grasped; v) Regulate the movement of the end effector through the PID controller to ensure smoothness and precision.

[0012] By adopting the above technical solution, the actual position and posture of the object to be grasped are monitored in real time based on the visual feedback system, and the angle and height of the end effector are dynamically adjusted according to the monitoring results, so as to ensure that the robot arm always maintains the best posture during the grasping process. Specifically, multi-view image information is collected by cameras installed on the head and hands of the robot, and transmitted to the central processor for calculation in real time, so as to accurately obtain the position and posture of the object to be grasped. Subsequently, the action of the end effector is adjusted by the PID controller so that it matches the actual position and posture of the object to be grasped, ensuring the stability and accuracy of the grasping process. This not only improves the success rate of grasping, but also reduces the risk of grasping failure or damage to items due to posture mismatch.

[0013] Preferably, the visual feedback system includes multiple high-resolution cameras, which are respectively installed on the head and hands of the robot to capture image information from different perspectives to ensure all-round monitoring. The specific configuration is as follows: i) the head camera is responsible for long-distance and large-range shooting to provide a global view; ii) the hand camera is responsible for close-range local shooting to provide detailed information; iii) the data of each camera is transmitted synchronously to ensure no delay.

[0014] By adopting the above technical solutions, the multi-view visual feedback system significantly improves the perception ability of humanoid robots in complex environments. The global view provided by the head camera enables the robot to fully understand the overall situation of the target area, while the close-up shooting of the hand camera ensures the precise grasp of details. The synchronous transmission of data from these cameras ensures the timeliness and consistency of information, thereby enhancing the robot's real-time response capability and operational accuracy. In particular, during the process of grasping and placing objects, the comprehensive analysis of multi-view image information helps the robot to more accurately judge the position and posture of objects, further improving operational efficiency and reliability.

[0015] Preferably, the method for capturing and confirming the state of the object to be grasped in step d) is: using a force sensor to detect the force applied during the grasping process to ensure that the grasping force is moderate and will not damage the object, and at the same time, combining a displacement sensor to detect small displacement changes during the grasping process to improve the grasping accuracy. The specific steps are as follows: i) The force sensor is installed inside the robot finger to measure the pressure distribution of the contact surface in real time; ii) When the pressure distribution is between 0.5N and 10N, the grasping is considered successful; iii) If it exceeds this range, the grasping posture is readjusted; iv) The displacement sensor detects small displacement changes during the grasping process to ensure stable grasping.

[0016] By adopting the above technical solution, accurate grasping and stable handling of objects can be achieved. Specifically, the force sensor can monitor the force applied during the grasping process in real time to ensure that the grasping force is moderate and will not damage the object, thereby improving the success rate of grasping. At the same time, the displacement sensor detects the slight displacement changes during the grasping process, further enhancing the stability of the grasping, reducing the misoperation caused by external interference, and improving the reliability and safety of the entire system. Preferably, the force sensor is installed inside the robot finger, and can measure the pressure distribution of the contact surface in real time during grasping. The pressure distribution range is set to between 0.5N and 10N. If it exceeds this range, the grasping posture is readjusted. The specific implementation steps are as follows: i) Set the upper limit of the threshold to 10N and the lower limit to 0.5N; ii) When the pressure is detected to exceed 10N, automatically release the finger and reposition the grasping point; iii) When the pressure is detected to be lower than 0.5N, increase the gripping force until it reaches the appropriate range.

[0017] By adopting the above technical solution, the force sensor is installed inside the robot finger, which can measure the pressure distribution of the contact surface in real time during grasping, ensuring that the grasping force is moderate and will not damage the object. When the pressure detected exceeds 10N, the finger is automatically released and the grasping point is repositioned; when the pressure detected is less than 0.5N, the gripping force is increased until it reaches the appropriate range, thereby improving the stability and reliability of grasping.

[0018] Preferably, the process of moving the grasped items to the designated placement area in step f) includes: planning the optimal path, avoiding obstacles, and ensuring that the items arrive at the destination safely. The path planning takes into account the kinematic constraints of the robot, minimizes the turning radius, and optimizes the walking speed. The specific steps are as follows: i) reading map data to determine the starting point and the end point; ii) applying an AI search algorithm to generate the optimal path; iii) avoiding known obstacles to ensure that the path is unobstructed; iv) minimizing the turning radius, optimizing the walking speed, and ensuring efficient operation.

[0019] By adopting the above technical solutions, the navigation ability of humanoid robots in complex environments can be significantly improved. Specifically, the optimal path is generated by combining AI search algorithms with map data, which not only avoids known obstacles but also ensures the smoothness and safety of the path. In addition, considering the kinematic constraints of the robot, minimizing the turning radius and optimizing the walking speed make the robot more efficient and stable when carrying items, thereby improving the operational efficiency of the entire logistics system.

[0020] Preferably, the optimal path is generated by an AI search algorithm combined with map data, and the algorithm weight function is \( f(n) = g(n) + h(n) \), where \( g(n) \) represents the actual cost from the starting point to node n, and \( h(n)\) represents the estimated cost from node n to the end point. Manhattan distance is used as the heuristic function \( h(n) = |x_{goal} x_n| + |y_{goal} y_n| \). The specific steps are as follows: i) Initialize the open list and the closed list; ii) Add the starting node to the open list; iii) Repeat the following steps until the end point is found or the open list is empty: Select the node with the lowest \( f(n) \) value from the open list; Move the node to the closed list; Check the adjacent nodes and update their \( g(n) \) and \( h(n) \) values; Add the adjacent nodes that meet the conditions to the open list; iv) Backtrack the path to generate the final route map.

[0021] By adopting the above technical solutions, humanoid robots can efficiently and accurately complete the tasks of identifying, grabbing, and moving objects from a designated area and placing them in order in another designated area. Specifically, the convolutional neural network model is used to extract image feature points, and the precise position and posture of the object to be grabbed are calculated through geometric relationships, which improves the accuracy and robustness of object recognition; the posture of the end effector is adjusted in real time through the visual feedback system to ensure the smoothness and accuracy of the grabbing action; the combined use of force sensors and displacement sensors further enhances the safety and stability of grabbing; the AI ​​search algorithm combines map data to generate the optimal path, avoid obstacles, minimize turning radius, optimize walking speed, and ensure that the objects reach their destination safely and efficiently; the genetic algorithm optimizes the placement strategy, maximizes the stacking density, reduces gaps, and achieves efficient object placement.

[0022] Preferably, the method of determining the appropriate placement position in step g) is: according to the spatial layout of the placed objects, the optimal utilization rate of the free space is calculated, and the most appropriate placement position is selected. Specifically, the placement strategy is optimized by a genetic algorithm to maximize the stacking density and reduce the gaps. The specific steps are as follows: i) determine the boundary of the placement area and the position of the placed objects; ii) initialize the population and randomly generate several feasible solutions; iii) calculate the objective function value of each solution, that is, the stacking density; iv) select excellent individuals to enter the next generation and eliminate poor solutions; v) perform crossover and mutation operations to generate new offspring; vi) repeat the above steps until convergence to obtain the optimal solution.

[0023] By adopting the above technical solutions, the humanoid robot can identify, grab, and carry objects from a designated area, and place them in order in another designated area. Specifically, the convolutional neural network model is used to extract feature points in the image, calculate the position and posture of the object to be grabbed, and ensure accurate identification. Subsequently, the posture of the end effector is adjusted in real time through the visual feedback system so that the robot arm accurately matches the state of the object to be grabbed. During the grasping process, the force sensor and the displacement sensor work together to ensure that the grasping force is moderate and stable. After the object is grasped, the AI ​​search algorithm is used to plan the optimal path, avoid obstacles, and ensure that the objects are safely and efficiently transported to the designated placement area. Finally, the placement strategy is optimized through the genetic algorithm to maximize the stacking density, reduce the gaps, and achieve orderly placement of objects.

[0024] In summary, the present application includes at least one of the following beneficial technical effects: 1. The convolutional neural network model is used to extract feature points in the image and use geometric relationships to calculate the position and posture of the object to be grasped, which improves the accuracy and robustness of object recognition, especially in environments with large light changes; 2. Combine the visual feedback system and PID controller to dynamically adjust the posture of the end effector to ensure that the robot arm can accurately match the actual position and posture of the object to be grasped, thereby improving the success rate and stability of grasping; 3. The AI ​​search algorithm is combined with map data to generate the optimal path, avoid obstacles and optimize walking speed, ensuring that items are safely and efficiently transported to the designated placement area, improving the reliability and efficiency of the entire transportation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of a humanoid robot control method according to an embodiment of the present application.

[0026] Figure 2 It is a structural schematic diagram of a humanoid robot control method in an embodiment of the present application. DETAILED DESCRIPTION

[0027] Embodiment 1: Reference Figure 1 and Figure 2 , a humanoid robot control method provided in an embodiment of the present application comprises the following steps: a) Obtain image information within the target area.

[0028] Specifically, in this embodiment, multiple high-resolution cameras are installed on the head and hands of the robot. The head camera is responsible for long-distance and large-scale shooting to provide a global view; the hand camera is responsible for close-range local shooting to provide detailed information. The data of each camera is transmitted synchronously to ensure no delay. In this way, the target area can be fully covered to ensure the integrity of the image information.

[0029] b) Process the acquired image information to identify the position and posture of the object to be grasped.

[0030] Specifically, in this embodiment, a convolutional neural network model is used to extract feature points in an image, and the position and posture of the object to be grasped are calculated based on the geometric relationship between the feature points. The specific steps are as follows: i) Input image information into the convolutional neural network model and extract key feature points: The convolutional neural network model has been trained at multiple levels and can recognize various types of objects and their different postures, especially suitable for environments with large changes in light. For example, the model can recognize bags, pallets and small items of various shapes.

[0031] ii) Use the formula to calculate the distance and angle between each feature point**: The distance \( d \) and angle \( \theta \) between feature points are calculated by the formulas \( d = \sqrt{(x_2 - x_1)^2 + (y_2 - y_1)^2} \) and \( \theta = \arctan\left(\frac{y_2 - y_1}{x_2 - x_1}\right) \).

[0032] iii) Determine the exact position and posture of the object to be grasped based on the calculation results: Through the above calculations, the position and posture of the object to be grasped can be accurately located, providing an accurate basis for subsequent operations. For example, if a square box is identified, its exact position and direction can be determined by calculating the distance and angle of its four corner points.

[0033] c) Control the robot arm to move above the object to be grasped and adjust the posture of the end effector to match the posture of the object to be grasped Specifically, in this embodiment, the actual position and posture of the object to be grasped are monitored in real time based on the visual feedback system, and the angle and height of the end effector are dynamically adjusted according to the monitoring results. Specifically, the dynamic adjustment is achieved through the PID controller, and the controller parameters \( K_p, K_i, K_d \) are set to 0.5, 0.1, and 0.2 respectively. The specific steps are as follows: i) Install cameras on the robot's head and hands to collect multi-view image information ii) Real-time transmission of image information to the central processing unit iii) The CPU calculates the actual position and posture of the object to be grasped based on the image information iv) Adjust the angle and height of the end effector to match the actual situation of the object to be grasped v) Regulate the motion of the end effector through the PID controller to ensure smooth and precise grasping: For example, when the robot approaches a tilted box, the PID controller will adjust the angle and height of the end effector according to the actual position and posture to ensure smooth grasping.

[0034] d) Capture and confirm the status of the object to be grasped. If the status meets the preset conditions, execute the next step; otherwise, return to step b) to continue identification Specifically, in this embodiment, a force sensor is used to detect the force applied during the grasping process to ensure that the grasping force is moderate and will not damage the item. At the same time, a displacement sensor is used to detect the slight displacement changes during the grasping process to improve the grasping accuracy. The specific steps are as follows: i) The force sensor is installed inside the robot finger to measure the pressure distribution on the contact surface in real time ii) When the pressure distribution is between 0.5N and 10N, the grasping is considered successful iii) If it exceeds this range, readjust the grasping posture iv) Displacement sensors detect tiny displacement changes during the grasping process to ensure stable grasping: For example, when the robot tries to grasp a thin cardboard box, the force sensor detects the appropriate grasping force to prevent the box from deforming or falling.

[0035] e) Perform the grabbing action and lift the object to be grabbed Specifically, when all conditions are met, the robot arm performs the grasping action and lifts the object to be grasped. This process requires fine coordination and control to ensure the smooth progress of the grasping action.

[0036] f) Move the grabbed items to the designated placement area Specifically, in this embodiment, the optimal path is planned to avoid obstacles and ensure that the items reach the destination safely. The path planning takes into account the robot's kinematic constraints, minimizes the turning radius, and optimizes the walking speed. The specific steps are as follows: i) Read map data and determine the starting and ending points ii) Applying A* search algorithm to generate optimal path: For example, when a robot needs to move from one corner of the warehouse to another, the A* search algorithm will generate a shortest path that bypasses the middle shelves.

[0037] iii) Avoid known obstacles and ensure a clear path iv) Minimize turning radius and optimize travel speed to ensure efficient operation: For example, the robot will slow down in narrow passages and speed up in open areas to ensure safety and efficiency.

[0038] g) Determine the appropriate placement location within the designated placement area and place the items to be transported in the predetermined order Specifically, the method for determining the appropriate placement position in this embodiment is: according to the spatial layout of the placed objects, the optimal utilization rate of the free space is calculated, and the most appropriate placement position is selected. Specifically, the placement strategy is optimized by genetic algorithm to maximize the stacking density and reduce the gaps. The specific steps are as follows: i) Determine the boundaries of the placement area and the locations of placed items.

[0039] ii) Initialize the population and randomly generate several feasible solutions.

[0040] iii) Calculate the objective function value, i.e., packing density, for each solution.

[0041] iv) Select outstanding individuals to enter the next generation and eliminate poor solutions.

[0042] v) Perform crossover and mutation operations to generate new offspring.

[0043] vi) Repeat the above steps until convergence to the optimal solution: For example, when the robot needs to place multiple boxes in a small space, the genetic algorithm will choose the most suitable location to ensure maximum space utilization.

[0044] In addition, an embodiment of the present application also discloses a humanoid robot for executing a humanoid robot control method, including a robot body 1, and the robot body is suitable for areas such as warehouses and houses.

[0045] The implementation principle of this embodiment is: through all-round monitoring of multiple cameras, image processing of convolutional neural networks, dynamic adjustment of visual feedback systems, precise detection of force sensors and displacement sensors, path planning of A* search algorithms, and placement optimization of genetic algorithms, efficient and precise control of humanoid robots in complex environments is achieved. This method not only improves the accuracy of object recognition and grasping, but also can flexibly adapt in dynamic environments, significantly improving the efficiency and safety of automated warehousing and logistics.

[0046] Embodiment 2: The difference between this embodiment and the above-mentioned embodiment is that in order to further improve the robot's adaptability in complex environments, this embodiment adds an environmental perception module and a self-learning function.

[0047] a) Obtain image information within the target area.

[0048] In addition to the multiple cameras, this embodiment also adds infrared sensors and ultrasonic sensors to detect the temperature, humidity and obstacles of the surrounding environment. The data of these sensors are combined with the camera data to form richer environmental information.

[0049] b) Process the acquired image information to identify the position and posture of the object to be grasped.

[0050] In this embodiment, the convolutional neural network model can not only recognize objects, but also other objects in the environment, such as walls, the ground, etc. By adding more training samples, the model can better understand the environment and improve the robustness of recognition. For example, the model can recognize the sign on the wall to help the robot avoid unnecessary collisions.

[0051] c) Control the robot arm to move above the object to be grasped and adjust the posture of the end effector to match the posture of the object to be grasped.

[0052] In this embodiment, the parameters of the PID controller can be dynamically adjusted according to changes in the environment. For example, on a slippery surface, the controller parameters may be appropriately reduced to avoid slipping. In addition, the controller parameters can be optimized through machine learning algorithms so that the robot can perform optimally in different environments. For example, when the robot moves on an icy surface, the controller parameters will be automatically adjusted down to ensure stability and safety.

[0053] d) Capture and confirm the status of the object to be grasped. If the status meets the preset conditions, execute the next step; otherwise, return to step b) to continue identification.

[0054] In this embodiment, the force sensor and displacement sensor can not only detect the force and displacement during the grasping process, but also record each successful grasping case for the robot to learn and refer to. Through continuous self-learning, the robot can make better decisions when encountering similar situations in the future. For example, when the robot grasps a heavy object, the force sensor will record the maximum grasping force required, and the same force can be directly used next time when encountering an object of similar weight.

[0055] e) Perform the grasping action and lift the object to be grasped.

[0056] In this embodiment, the grasping action is not just a simple pick up and put down, but also includes a variety of actions such as rotation and tilting to adapt to objects of different shapes and sizes. By simulating human finger movements, the robot can complete the grasping task more naturally. For example, when the robot grasps a round bottle, it will straighten it before grasping it to prevent it from rolling.

[0057] f) Move the grabbed items to the designated placement area.

[0058] In this embodiment, path planning should not only consider static obstacles, but also dynamic obstacles, such as moving people or other robots. By updating map data in real time, the robot can react quickly and avoid obstacles in time to ensure safe and efficient driving. For example, when the robot detects that there is a worker carrying items in front of it, it will actively stop aside and wait for the worker to pass before continuing to move forward.

[0059] g) Determine appropriate placement locations within the designated placement area and place the items being transported in the predetermined order.

[0060] In this embodiment, placement optimization not only considers the stacking density, but also the stability of the items. Through physical simulation technology, the robot can predict the stability of each placement method and select the most stable solution for placement. For example, when placing fragile items, the robot will give priority to places with a flat bottom to reduce damage caused by vibration.

[0061] The implementation principle of this embodiment is: by adding an environmental perception module and a self-learning function, the humanoid robot of this embodiment can autonomously adapt to more complex environments and improve the success rate and efficiency of tasks. In particular, for emergencies in dynamic environments, the robot can continuously optimize its own control strategy through self-learning, so as to perform better in future work.

[0062] Embodiment 3: The difference between this embodiment and the above-mentioned embodiment is that, in response to the application requirements of special occasions, this embodiment adds explosion-proof and fire-proof functions.

[0063] a) Obtain image information within the target area.

[0064] In this embodiment, both the camera and the sensor are explosion-proof and suitable for use in flammable and explosive environments. The camera housing is made of high-strength material, and the sensor has a built-in anti-static device to ensure normal operation in dangerous environments. For example, the camera lens and sensor interface are both metal shielded to prevent electric sparks from causing explosions.

[0065] b) Process the acquired image information to identify the position and posture of the object to be grasped.

[0066] In this embodiment, the convolutional neural network model is specially trained for anti-interference, and can maintain good recognition effect in strong electromagnetic fields and high temperature environments. By labeling and training data in extreme environments, the model has higher robustness. For example, the model can identify burning flames and help the robot avoid fire sources.

[0067] c) Control the robot arm to move above the object to be grasped and adjust the posture of the end effector to match the posture of the object to be grasped.

[0068] In this embodiment, the hardware part of the PID controller is also explosion-proof to ensure that it can still work reliably in an environment with a high risk of explosion. At the same time, the controller software part has added a fault diagnosis function. Once an abnormality is detected, measures are taken immediately to prevent the accident from expanding. For example, when a sensor failure is detected, the controller will sound an alarm and switch to a backup sensor to continue working.

[0069] d) Capture and confirm the status of the object to be grasped. If the status meets the preset conditions, execute the next step; otherwise, return to step b) to continue identification.

[0070] In this embodiment, the force sensor and displacement sensor are also explosion-proof to ensure accurate measurement in high temperature and high pressure environments. In addition, the sensor signal transmission line is also shielded to prevent external interference. For example, the force sensor is filled with flame-retardant materials to prevent electrical failures caused by high temperatures.

[0071] e) Perform the grasping action and lift the object to be grasped.

[0072] In this embodiment, both the robot arm and the end effector are made of high temperature resistant materials and can work for a long time in a high temperature environment. In addition, the gripping component is also equipped with a cooling system to ensure that it will not overheat and fail under high temperature conditions. For example, a heat sink is provided inside the robot arm to help quickly cool down.

[0073] f) Move the grabbed items to the designated placement area.

[0074] In this embodiment, the path planning takes into account the fire source and other potential dangerous factors, and gives priority to paths away from dangerous areas. At the same time, the robot is equipped with a fire extinguishing device. Once a fire is detected, the fire extinguishing procedure is immediately initiated to ensure the safety of the robot and the surrounding environment. For example, when the robot is close to the fire source, the water sprinkler system will be activated to extinguish the initial fire.

[0075] g) Determine appropriate placement locations within the designated placement area and place the items being transported in the predetermined order.

[0076] In this embodiment, the placement optimization not only considers the stacking density, but also the unobstructed fire passage. Through a reasonable placement strategy, it is ensured that rescue vehicles and personnel can arrive at the scene quickly in an emergency. For example, when the robot stacks items, it will reserve enough space for the fire truck to pass.

[0077] The implementation principle of this embodiment is: by adding explosion-proof and fire-proof functions, the humanoid robot of this embodiment can work reliably in extremely harsh environments to ensure the safety of production and storage. Especially in chemical plants, oil depots and other places, this specially designed robot can greatly reduce the risk of accidents and ensure the safety of personnel and property.

[0078] Embodiment 4: The difference between this embodiment and the above-mentioned embodiment is that, for outdoor working scenes, this embodiment adds waterproof and dustproof functions.

[0079] a) Obtain image information within the target area.

[0080] In this embodiment, the camera and sensor are both sealed, and the protection level reaches the IP67 standard, which can be used normally in harsh environments such as rain, snow, and sandstorms. The surface of the camera lens is coated with a hydrophobic coating to prevent rain residue from affecting the image quality. For example, a layer of nano-level waterproof film is coated on the outside of the camera lens, which can provide clear images even in heavy rain.

[0081] b) Process the acquired image information to identify the position and posture of the object to be grasped In this embodiment, the convolutional neural network model is specially trained to maintain good recognition results in low contrast and blurry conditions. By enhancing the image processing algorithm, the model can filter out noise and interference and improve recognition accuracy. For example, the model can recognize the outline of an object on muddy grass to ensure accurate recognition in complex terrain.

[0082] c) Control the robot arm to move above the object to be grasped and adjust the posture of the end effector to match the posture of the object to be grasped.

[0083] In this embodiment, the hardware of the PID controller adopts a waterproof design to ensure that it will not short-circuit or corrode in a humid environment. At the same time, the controller software has added a humidity compensation function to automatically adjust the control parameters according to different environmental conditions to ensure stable operation. For example, a desiccant is set inside the controller to prevent moisture from corroding electronic components.

[0084] d) Capture and confirm the status of the object to be grasped. If the status meets the preset conditions, execute the next step; otherwise, return to step b) to continue identification.

[0085] In this embodiment, the force sensor and displacement sensor are also waterproof, and the signal transmission line also uses a waterproof connector to prevent water intrusion and cause malfunctions. In addition, the sensor is wrapped with a layer of anti-corrosion material to extend its service life. For example, the cable socket of the force sensor uses a rubber sealing ring to prevent water intrusion and cause a short circuit.

[0086] e) Perform the grasping action and lift the object to be grasped.

[0087] In this embodiment, the robot arm and the end effector are made of waterproof materials, and sealing rings are provided at the joints to ensure normal operation in water. The gripping parts are also equipped with drainage holes to prevent water accumulation from affecting the gripping effect. For example, waterproof gaskets are provided at the connection parts of the robot arm to prevent water from penetrating into the internal mechanical structure.

[0088] f) Move the grabbed items to the designated placement area.

[0089] In this embodiment, the path planning takes into account the characteristics of outdoor terrain and gives priority to flat and barrier-free roads. At the same time, the robot is equipped with a GPS positioning system, which can accurately navigate even in the absence of obvious signs. For example, when the robot crosses the forest, it will use the satellite positioning system to move along the predetermined track to avoid getting lost.

[0090] g) Determine appropriate placement locations within the designated placement area and place the items being transported in the predetermined order.

[0091] In this embodiment, the placement optimization not only considers the stacking density, but also the influence of outdoor wind speed. Through reasonable design, it is ensured that the stacked items are not easy to fall over in strong winds, thereby improving the overall stability. For example, when placing items, the robot will try to choose the leeward side to reduce the influence of wind pressure on the stacking stability.

[0092] The implementation principle of this embodiment is: by adding waterproof and dustproof functions, the humanoid robot of this embodiment can work stably for a long time in harsh outdoor environments and meet the needs of various outdoor operations. Whether it is farming or construction, this multifunctional robot can play an important role and improve work efficiency and safety.

[0093] Embodiment 5: The difference between this embodiment and the above-mentioned embodiments is that, in response to the needs of the medical field, this embodiment adds disinfection and sterilization functions.

[0094] a) Obtain image information within the target area.

[0095] In this embodiment, both the camera and the sensor are made of antibacterial materials to prevent bacterial growth. The surface of the camera lens is coated with an antibacterial coating to ensure normal operation in an environment with high hygiene requirements such as a hospital. For example, the camera lens is coated with a layer of silver ion antibacterial film to inhibit bacterial growth.

[0096] b) Process the acquired image information to identify the position and posture of the object to be grasped.

[0097] In this embodiment, the convolutional neural network model is professionally trained to identify medical supplies and equipment, especially those that require strict sterilization. By enhancing the image processing algorithm, the model can distinguish between medical items of different types and states, improving recognition accuracy. For example, the model can recognize the difference between a scalpel and a hemostat to ensure correct grasping.

[0098] c) Control the robot arm to move above the object to be grasped and adjust the posture of the end effector to match the posture of the object to be grasped.

[0099] In this embodiment, the hardware part of the PID controller is made of medical grade materials to ensure that it can still work normally after multiple disinfections. At the same time, the controller software part has added a disinfection cycle management function to regularly remind staff to perform maintenance and cleaning. For example, a timer is set inside the controller to automatically check once a day whether there is a need for disinfection.

[0100] d) Capture and confirm the status of the object to be grasped. If the status meets the preset conditions, execute the next step; otherwise, return to step b) to continue identification.

[0101] In this embodiment, the force sensor and displacement sensor are also made of antibacterial materials, and the signal transmission line is also disinfected to ensure that it will not become a source of contamination in the hospital environment. In addition, the sensor is wrapped with an antibacterial film to extend its service life. For example, the surface of the force sensor is coated with an antibacterial coating to prevent bacteria from attaching.

[0102] e) Perform the grasping action and lift the object to be grasped.

[0103] In this embodiment, the robot arm and end effector are made of easy-to-clean materials, and the joints are equipped with self-cleaning functions to ensure that they can be easily disinfected before and after use. The gripping parts are also equipped with ultraviolet lamps to quickly disinfect gloves, tweezers and other tools before grasping. For example, the joints of the robot arm are equipped with micro brushes to automatically remove dirt after each use.

[0104] f) Move the grabbed items to the designated placement area.

[0105] In this embodiment, the path planning takes into account the dense flow of people in the hospital, giving priority to spacious corridors with few people walking. At the same time, the robot is equipped with a positioning system to ensure accurate navigation in a busy medical environment. For example, when the robot passes through the waiting hall, it will move slowly along the wall to avoid disturbing patients.

[0106] g) Determine appropriate placement locations within the designated placement area and place the items being transported in the predetermined order.

[0107] In this embodiment, placement optimization not only considers the stacking density, but also the storage specifications of medical supplies. Through reasonable design, it is ensured that each medical item is correctly placed, which is convenient for medical staff to quickly use. For example, when placing medicines, the robot will strictly follow the classification rules of the pharmacy to ensure that the same type of medicines are stored together.

[0108] The implementation principle of this embodiment is: by adding disinfection and sterilization functions, the humanoid robot of this embodiment can play an important role in the medical field to ensure the safety and hygiene of medical supplies. Whether in the operating room or the ward, this professional robot can effectively reduce the burden on medical staff and improve the quality and efficiency of medical services.

[0109] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for controlling a humanoid robot, characterized in that: The following steps are involved: a) Acquire image information in the target area; b) Process the acquired image information to identify the position and posture of the object to be grasped; c) Control the robot arm to move above the object to be grasped, and adjust the posture of the end effector to match the posture of the object to be grasped; d) Capture and confirm the state of the object to be grasped. If the state meets the preset conditions, execute the next step; otherwise, return to step b) to continue identification; e) Perform the grabbing action to lift the object to be grabbed; f) Carry the grabbed items to the designated placement area; g) Determine the appropriate placement location within the designated placement area and place the items being transported in the predetermined order.

2. The humanoid robot control method according to claim 1, characterized in that: The specific method of processing the image information in step b) is: using a convolutional neural network model to extract feature points in the image, and calculating the position and posture of the object to be grasped through the geometric relationship between the feature points, wherein the distance and angle relationship between the feature points are calculated by the formulas \( d =\sqrt{(x_2 x_1)^2 + (y_2 y_1)^2} \) and \( \theta = \arctan\left(\frac{y_2 y_1}{x_2 x_1}\right) \). The specific steps are as follows: i) Input image information into the convolutional neural network model to extract key feature points; ii) Use the above formula to calculate the distance and angle between each feature point; iii) The precise position and posture of the object to be grasped are determined based on the calculation results.

3. The humanoid robot control method according to claim 2, characterized in that: The convolutional neural network model is trained at multiple levels and can recognize various types of objects and their different posture changes. It is particularly suitable for environments with large light changes. The specific training steps are as follows: i) Collect a large number of different types of object images, covering various lighting conditions and pose variations; ii) Use annotation tools to identify the feature points in each image; iii) Input the labeled images into the convolutional neural network model for training, and continuously iterate and optimize the model parameters; iv) Test the model on new images to ensure its accuracy and robustness.

4. The humanoid robot control method according to claim 1, characterized in that: The method of adjusting the posture of the end effector in step c) is: based on the visual feedback system, the actual position and posture of the object to be grasped are monitored in real time, and the angle and height of the end effector are dynamically adjusted according to the monitoring results. Specifically, the dynamic adjustment is achieved through the PID controller, and the controller parameters \( K_p, K_i, K_d \) are set to 0.5, 0.1, and 0.2 respectively. The specific steps are as follows: i) Install cameras on the robot's head and hands to collect multi-view image information; ii) Transmit image information to the central processor in real time; iii) The central processing unit calculates the actual position and posture of the object to be grasped based on the image information; iv) Adjust the angle and height of the end effector to match the actual situation of the object to be grasped; v) The movement of the end effector is adjusted by the PID controller to ensure smoothness and precision.

5. The humanoid robot control method according to claim 4, characterized in that: The visual feedback system includes multiple high-resolution cameras, which are installed on the head and hands of the robot to capture image information from different perspectives to ensure all-round monitoring. The specific configuration is as follows: i) The head camera is responsible for long-distance and wide-range shooting, providing a global view; ii) The hand camera is responsible for close-up local shooting and provides detailed information; iii) The data from each camera is transmitted synchronously to ensure no delay.

6. The humanoid robot control method according to claim 1, characterized in that: The method for capturing and confirming the state of the object to be grasped in step d) is: using a force sensor to detect the force applied during the grasping process to ensure that the grasping force is moderate and will not damage the object. At the same time, combining a displacement sensor to detect the slight displacement changes during the grasping process to improve the grasping accuracy. The specific steps are as follows: i) The force sensor is installed inside the robot finger to measure the pressure distribution on the contact surface in real time; ii) When the pressure distribution is between 0.5N and 10N, the grasping is considered successful; iii) If it exceeds this range, readjust the grasping posture; iv) The displacement sensor detects tiny displacement changes during the grasping process to ensure stable grasping.

7. The humanoid robot control method according to claim 6, characterized in that: The force sensor is installed inside the robot finger and can measure the pressure distribution of the contact surface in real time during grasping. The pressure distribution range is set between 0.5N and 10N. If it exceeds this range, the grasping posture is readjusted. The specific implementation steps are as follows: i) The upper and lower limits of the threshold are set to 10N and 0.5N; ii) When the pressure detected exceeds 10N, the fingers are automatically released and the gripping point is repositioned; iii) When the pressure is detected to be lower than 0.5N, increase the grip strength until it reaches the appropriate range.

8. The humanoid robot control method according to claim 1, characterized in that: The process of moving the grasped object to the designated placement area in step f) includes: planning the optimal path, avoiding obstacles, and ensuring that the object arrives safely at the destination. The path planning takes into account the robot's kinematic constraints, minimizes the turning radius, and optimizes the walking speed. The specific steps are as follows: i) Read map data to determine the starting point and end point; ii) Apply AI search algorithms to generate optimal paths; iii) Avoid known obstacles and ensure the path is clear; iv) Minimize turning radius and optimize travel speed to ensure efficient operation.

9. The humanoid robot control method according to claim 8, characterized in that: The optimal path is generated by combining AI search algorithm with map data. The algorithm weight function is \( f(n) = g(n) + h(n) \), where \( g(n) \) represents the actual cost from the starting point to node n, and \( h(n) \) represents the estimated cost from node n to the end point. Manhattan distance is used as the heuristic function \( h(n) = |x_{goal} x_n| + |y_{goal} y_n| \). The specific steps are as follows: i) Initialize open list and closed list; ii) add the starting node to the open list; iii) Repeat the following steps until the end is found or the open list is empty: Select the node with the lowest \( f(n) \) value from the open list; Move the node to the closed list; Check the adjacent nodes and update their \( g(n) \) and \( h(n) \) values; Add the qualified adjacent nodes to the open list; iv) Backtrack the path to generate the final route map.

10. The humanoid robot control method according to claim 1, characterized in that: The method of determining the appropriate placement position in step g) is: according to the spatial layout of the placed objects, the optimal utilization rate of the free space is calculated, and the most appropriate placement position is selected. Specifically, the placement strategy is optimized by a genetic algorithm to maximize the stacking density and reduce the gaps. The specific steps are as follows: i) Determine the boundaries of the placement area and the locations of placed items; ii) Initialize the population and randomly generate several feasible solutions; iii) Calculate the objective function value, i.e., packing density, for each solution; iv) Select outstanding individuals to enter the next generation and eliminate poor solutions; v) Perform crossover and mutation operations to generate new offspring; vi) Repeat the above steps until convergence to the optimal solution.

Images