Method and device for generating passable area of humanoid robot and humanoid robot

Abstract

The invention belongs to the technical field of robots, and provides a humanoid robot passable area generation method and device and a humanoid robot, and the method comprises the steps: obtaining an original image of a to-be-passed area of the humanoid robot when it is detected that the humanoid robot starts to move after being electrified; defocusing recovery processing is carried out on the original image, and a depth image is generated based on the distance from each pixel point in the image after defocusing recovery to the monocular camera; determining a first region formed by pixel points of which the depth values are greater than a distance threshold in the depth image, and mapping the first region into the original image to obtain a second region; determining an impassable area of the original image; determining a passable area of the humanoid robot based on the impassable area of the original image; and the passing direction angle of the humanoid robot is determined, and the humanoid robot is controlled to pass through the passable area based on the passing direction angle. According to the invention, the passable area of the humanoid robot can be accurately generated in real time, and real-time navigation is carried out.

Description

Technical Field

[0001] The present invention belongs to the technical field of robots, and in particular relates to a method and device for generating a traversable area for a humanoid robot, and the humanoid robot. Background Art

[0002] A humanoid robot is a robot that is designed to imitate human appearance and behavior. Because humanoid robots can assist people in completing relatively heavy and complex tasks, and their work efficiency and quality are high, humanoid robots are widely used in many industries such as medical care, military, education, production and life.

[0003] In the process of generating traversable areas, existing humanoid robots rely too much on hardware, such as lidar, binocular cameras or structured light sensors. If the image clarity collected by the hardware is low, the depth estimation accuracy will be low, which will in turn lead to low accuracy of the generated traversable areas. At the same time, because existing humanoid robots use traditional image processing algorithms, the real-time performance of existing humanoid robot vision systems is poor, that is, existing humanoid robots cannot accurately generate traversable areas in real time. Summary of the invention

[0004] The purpose of the present invention is to accurately generate a traversable area for a humanoid robot in real time and to ensure that the humanoid robot can still work stably in complex environments and scenes such as complex lighting conditions.

[0005] In a first aspect, an embodiment of the present invention provides a method for generating a traversable area for a humanoid robot, which is applied to a humanoid robot including a monocular camera, and the method includes:

[0006] When it is detected that the humanoid robot is powered on and starts to move, an original image of the area to be passed by the humanoid robot is obtained, wherein the original image is acquired in real time by the monocular camera;

[0007] Performing defocus recovery processing on the original image to obtain a defocus-recovered image, and generating a depth image based on the distance from each pixel in the defocus-recovered image to the monocular camera;

[0008] Determine a first region composed of pixel points in the depth image whose depth values ​​are greater than a distance threshold, and map the first region to the original image to obtain a second region; the distance threshold is determined based on the current travel speed of the humanoid robot and a preset time length;

[0009] Determine an impassable area of ​​the original image, where the impassable area is: a rectangular area determined based on a width of the second area and a height of the original image;

[0010] Determining a traversable area of ​​the humanoid robot in the original image based on the intraversable area of ​​the original image;

[0011] The passing direction angle of the humanoid robot is determined, and based on the passing direction angle, the humanoid robot is controlled to pass through the passable area.

[0012] Optionally, determining a first area consisting of pixels in the depth image whose depth values ​​are greater than a distance threshold includes:

[0013] Acquire the current travel speed of the humanoid robot, and determine the product of the current travel speed and the preset time as the distance threshold;

[0014] Searching for target pixel points whose distance is greater than the distance threshold in the depth image, and determining the width of the area formed by the target pixel points;

[0015] Based on the width of the region composed of the target pixel points and the height of the depth image, a first region composed of pixel points having depth values ​​greater than a distance threshold in the depth image is determined.

[0016] Optionally, determining the traversable area of ​​the humanoid robot in the original image based on the impassable area of ​​the original image includes:

[0017] Calculating the width of each rectangular area in the original image except the impassable area;

[0018] Acquire the width of the humanoid robot, and determine the product of the width of the humanoid robot and a preset multiple as a width threshold;

[0019] Compare the width of each rectangular area with the width threshold respectively;

[0020] If there is a target rectangular area with a width greater than the width threshold, the rectangular area with the largest width in the target rectangular area is determined as the passable area of ​​the humanoid robot.

[0021] Optionally, determining the passing direction angle of the humanoid robot, and controlling the humanoid robot to pass through the passable area based on the passing direction angle includes:

[0022] Determine the coordinates of a first central pixel point of the passable area, and determine the coordinates of a second central pixel point of the original image;

[0023] Determine a depth value of the first central pixel, where the depth value of the first central pixel is the depth value of the central pixel of the depth image;

[0024] Calculating a passing direction angle of the humanoid robot based on the coordinates of the first central pixel point, the coordinates of the second central pixel point, and the depth value of the first central pixel point;

[0025] According to the traveling direction angle and current traveling speed of the humanoid robot, the humanoid robot is controlled to travel in the travelable area, and the step of obtaining the original image of the area to be traveled by the humanoid robot is returned to be executed.

[0026] Optionally, the method further includes:

[0027] If the traversable area of ​​the humanoid robot does not exist in the original image, controlling the humanoid robot to rotate on the spot by a preset angle, and controlling the traversing speed of the humanoid robot to be zero;

[0028] Return to the step of acquiring the original image of the area to be passed by the humanoid robot.

[0029] Optionally, performing defocus recovery processing on the original image to obtain a defocus-recovered image includes:

[0030] Inputting the original image into a pre-trained defocus image restoration model to perform defocus restoration processing on the original image through the defocus image restoration model, and outputting the defocus restored image;

[0031] The defocused image restoration model is obtained by training with a pre-collected defocused image and a clear image corresponding to the defocused image.

[0032] Optionally, generating a depth image based on the distance from each pixel in the defocused restored image to the monocular camera includes:

[0033] Inputting the defocused image into a pre-trained image depth prediction model, so that the image depth prediction model extracts image features of the defocused image, obtains the distance from each pixel in the defocused image to the monocular camera based on the relationship between the image features and the depth distance, and outputs a depth image with the distance as the depth;

[0034] The image depth distance prediction model is obtained by training with clear images in an image library.

[0035] In a second aspect, an embodiment of the present invention further provides a device for generating a traversable area for a humanoid robot, which is applied to a humanoid robot including a monocular camera, and the device includes:

[0036] An original image acquisition module, used for acquiring an original image of the area to be passed by the humanoid robot when it is detected that the humanoid robot is powered on and starts to move, wherein the original image is acquired in real time by the monocular camera;

[0037] A depth image generation module, used to perform defocus recovery processing on the original image to obtain a defocus-recovered image, and generate a depth image based on the distance from each pixel in the defocus-recovered image to the monocular camera;

[0038] A region determination module, configured to determine a first region composed of pixels in the depth image whose depth values ​​are greater than a distance threshold, and map the first region to the original image to obtain a second region; the distance threshold is determined based on the current travel speed of the humanoid robot and a preset duration;

[0039] An impassable area determination module, used to determine an impassable area of ​​the original image, wherein the impassable area is: a rectangular area determined based on a width of the second area and a height of the original image;

[0040] A traversable area determination module is used to determine a traversable area of ​​the humanoid robot in the original image based on the intraversable area of ​​the original image.

[0041] The humanoid robot control module is used to determine the passing direction angle of the humanoid robot and control the humanoid robot to pass through the passable area based on the passing direction angle.

[0042] In a third aspect, an embodiment of the present invention further provides a humanoid robot, comprising:

[0043] A monocular camera, used to collect original images of the area to be passed by the humanoid robot in real time;

[0044] at least one processor, configured to acquire the original image from the monocular camera when detecting that the humanoid robot is powered on and starts to move;

[0045] a memory for storing the at least one processor-executable instruction;

[0046] The at least one processor is configured to execute the instructions to implement the method described in the first aspect.

[0047] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method described in the first aspect.

[0048] The technical solution provided by the embodiment of the present invention uses the monocular camera of the humanoid robot to collect the original image of the area to be passed in real time. Since the original image may be a defocused image with low clarity, the original image is defocused to obtain a defocused image, and a depth image is generated based on the distance from each pixel in the defocused image to the monocular camera. Since the image clarity after defocusing is high, the accuracy of the distance from each pixel to the monocular camera in the defocused image is also high, and the accuracy of the generated depth image is also high, that is, the accuracy of the depth value of each pixel in the generated depth image is high.

[0049] Then, the first area composed of pixels whose depth values ​​are greater than the distance threshold in the depth image is determined. Since the area whose depth value is greater than the distance threshold is far away from the humanoid robot, the humanoid robot will not move to the area far away from it at present. Therefore, the first area is the inaccessible area determined by the depth image. In addition, in order to make the finally determined traversable area correspond to the actual spatial area, it is necessary to map the first area in the depth image to the original image to obtain the second area corresponding to the first area; since the humanoid robot passes vertically, the rectangular area determined by the width of the second area and the height of the original image is determined as the inaccessible area of ​​the original image, and based on the inaccessible area of ​​the original image, the traversable area of ​​the humanoid robot is determined in the original image. Finally, the passing direction angle of the humanoid robot is determined, and according to the passing direction angle, the humanoid robot is controlled to pass in the traversable area.

[0050] It can be seen that the technical solution of the embodiment of the present invention can accurately generate the traversable area of ​​the humanoid robot in real time, accurately determine the traversable direction angle in real time, and control the humanoid robot to pass through the traversable area in real time according to the traversal direction angle, thereby realizing real-time navigation. In addition, the humanoid robot can still work stably in complex environments and scenes such as complex lighting conditions, thereby enhancing robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of a partial hardware structure of a humanoid robot provided by an embodiment of the present invention;

[0052] Figure 2 A flow chart of the overall technical solution of a method for generating a traversable area for a humanoid robot provided by an embodiment of the present invention;

[0053] Figure 3 A flowchart of a method for generating a traversable area for a humanoid robot provided in an embodiment of the present invention;

[0054] Figure 4 A flowchart of a specific implementation of S360 provided in an embodiment of the present invention;

[0055] Figure 5 A schematic structural diagram of a device for generating a traversable area for a humanoid robot provided by an embodiment of the present invention;

[0056] Figure 6 A schematic structural diagram of a humanoid robot provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The present invention will be described in detail below through examples.

[0058] With the rapid development of science and technology, humanoid robots are widely used in many industries such as medicine, military, education, production and life. The existing methods for generating and navigating the traversable area of ​​humanoid robots are overly dependent on hardware, such as lidar, binocular cameras or structured light sensors, which not only increases the weight of the humanoid robot's head, but is also not conducive to improving the visual resolution of the humanoid robot, and the overall cost of the humanoid robot is relatively high.

[0059] In addition, existing humanoid robot navigation systems do not have integrated image restoration processing functions. For example, existing humanoid robots do not have image defocus recovery functions, which makes the system unable to work effectively when faced with blurred images or defocused images, greatly reducing the accuracy of the extracted depth information, resulting in low accuracy of the generated humanoid robot's traversable area and problems with the humanoid robot's path planning.

[0060] At the same time, because the existing humanoid robots use traditional image processing algorithms, the real-time performance of the existing humanoid robot vision system is insufficient and cannot meet the needs of humanoid robots to respond quickly in dynamic environments, that is, the existing humanoid robots cannot accurately generate traversable areas in real time. To this end, the existing humanoid robots also rely on the supplement and assistance of other sensors such as six-dimensional force sensors, which may cause the humanoid robots to be unable to accurately generate traversable areas in a timely manner in an emergency, and thus unable to accurately navigate.

[0061] In summary, existing humanoid robots do not effectively integrate image restoration, depth estimation and path planning, and the efficiency of generating traversable areas is low, and the accuracy of the generated traversable areas is low, resulting in problems such as slow response and difficult maintenance of existing humanoid robots.

[0062] In view of the above problems existing in the existing humanoid robots, the embodiments of the present invention provide a method and device for generating a traversable area for a humanoid robot and a humanoid robot.

[0063] The method for generating a passable area for a humanoid robot provided by the present invention generates a passable area for a humanoid robot in real time by combining the defocus recovery technology of a monocular image with the depth estimation technology. The efficiency of generating the passable area is high, and the accuracy of the generated passable area is high. Moreover, even in application scenarios with complex environments or insufficient lighting, the humanoid robot can generate the passable area efficiently and accurately. In other words, the humanoid robot provided by the present invention can respond quickly to environmental changes and provide accurate spatial information for path planning, which solves the limitations of the prior art in humanoid robot navigation and path planning, and improves the environmental perception ability, navigation accuracy and path planning efficiency of the humanoid robot. Moreover, by using a monocular camera, the weight of the head of the humanoid robot can be reduced, and costs can be saved.

[0064] In order to describe the solution clearly, the overall technical solution of the embodiment of the present invention will be elaborated in detail below in combination with specific application scenarios.

[0065] First, the hardware structure of the humanoid robot provided in the embodiment of the present invention is introduced.

[0066] like Figure 1 As shown, the hardware structure of the humanoid robot provided by the embodiment of the present invention includes: a humanoid robot head vision mounting base 101, a communication line routing structure 102 and a monocular camera 103, all of which are installed on a flat plate on the chest of the humanoid robot.

[0067] The humanoid robot head vision mounting chassis 101 is used for overall installation and locking, and is adapted to the upper chest plane of various types of humanoid robots. The communication line routing structure 102 is used for built-in communication lines to achieve beautiful line management. The monocular camera 103 is a universal monocular array industrial camera of various types, with no special requirements for specifications and models. It is used to collect monocular images of the area to be driven by the humanoid robot in real time during the passage of the humanoid robot, so that in the subsequent steps, the monocular image can be used to efficiently and accurately generate the passable area of ​​the humanoid robot.

[0068] It should be explained that when the monocular camera 103 is fastened and locked to the humanoid robot head visual mounting base 101, the installation and fastening is performed so that the monocular camera 103 does not fall off, and no special calibration steps and installation angle requirements are required; and the camera communication line is buried in the communication line routing structure 102, and there is no special requirement for the burying method. It can be seen that the solution of the embodiment of the present invention has the advantage of easy installation.

[0069] After introducing the hardware structure of the humanoid robot provided in the embodiment of the present invention, the overall technical solution of a method for generating a traversable area for a humanoid robot provided in the embodiment of the present invention will be introduced below. The execution subject of the method for generating a traversable area for a humanoid robot can be Figure 1The humanoid robot of the embodiment is provided with a humanoid robot traversable area generating system for performing the following steps: Figure 2 The method for generating a traversable area for a humanoid robot is shown in FIG. Figure 2 As shown, the method for generating a traversable area for a humanoid robot may include the following steps:

[0070] S210, when it is detected that the humanoid robot is powered on and starts to move, start the monocular camera.

[0071] Specifically, after detecting that the humanoid robot is powered on and starts to move, it is necessary to generate the passable area of ​​the humanoid robot in real time so as to control the humanoid robot to pass according to the generated passable area. Therefore, it is necessary to start the monocular camera so that in the subsequent steps, the monocular camera can be used to collect images of the area to be passed.

[0072] S220, determining whether the monocular camera communication is normal.

[0073] Specifically, after starting the monocular camera through S210, in order to ensure that the monocular camera can collect images in real time, it is necessary to determine whether the monocular camera communication is normal. If the monocular camera communication is normal, it means that the monocular camera can work normally, that is, it can collect images in real time. At this time, the next step S230 is executed. If the monocular camera communication is abnormal, it means that the monocular camera cannot work normally, that is, it cannot collect images. At this time, S220 is continued until it is determined that the monocular camera communication is normal.

[0074] S230, when it is determined that the monocular camera communication is normal, the original image captured by the monocular camera is acquired in real time, and a defocus recovery algorithm is called to restore the clarity of the original image to obtain a defocus restored image.

[0075] Specifically, when it is determined that the monocular camera communication is normal, the monocular camera can collect images of the area to be passed in real time. In order to clearly describe the solution, the images collected by the monocular camera can be called original images. Since the original images directly collected by the monocular camera may be defocused images with low clarity, in order to accurately generate the passable area of ​​the humanoid robot in subsequent steps, the defocus recovery algorithm can be called to perform defocus recovery processing on the original image, that is, to restore the clarity of the original image and obtain the defocus recovered image.

[0076] In this embodiment, the specific implementation method of calling the defocus recovery algorithm to perform defocus recovery processing on the original image can be: directly input the original image into a pre-trained defocus image recovery model, the defocus image recovery model performs defocus recovery processing on the defocus image, and outputs the defocus restored image.

[0077] The training process of the defocused image restoration model may be as follows: collecting a large number of defocused images of various defocused types as negative samples; and collecting a large number of clear images that have a mapping relationship with the defocused images in the negative samples as positive samples; inputting the negative samples and the positive samples into the neural network model to be trained, and training the neural network model; after the negative samples are input into the neural network model, when the accuracy of the clear image output from the neural network model is greater than the preset accuracy, it means that the trained neural network model can restore the defocused image to a clear image, and at this time the trained neural network model is determined as the defocused image restoration model. The preset accuracy can be determined according to the actual situation and is not specifically limited here.

[0078] It can be seen that the technical solution of the embodiment of the present invention does not perform preprocessing and image enhancement processing on the original image during the process of restoring the clarity of the defocused image, but directly inputs the original image into the pre-trained defocused image restoration model, that is, a clear image after defocus restoration can be obtained efficiently, and the obtained clear image has a high accuracy.

[0079] S240, calling a depth estimation algorithm to perform depth calculation on the image after defocus restoration, obtaining the distance from each pixel in the image after defocus restoration to the monocular camera, and forming a depth image with the distance as the depth.

[0080] Specifically, after obtaining the image after defocus restoration, the depth estimation algorithm can be called to perform depth calculation on the image after defocus restoration, obtain the distance from each pixel in the image after defocus restoration to the monocular camera, and form a depth image with the distance as the depth. Since the image after defocus restoration has a high definition, the distance from each pixel to the monocular camera obtained by performing depth calculation on the image after defocus restoration has a high accuracy, and thus the accuracy of the generated depth image is also high, which helps to accurately determine the traversable area of ​​the humanoid robot in the subsequent steps.

[0081] In this embodiment, the specific implementation method of S240 can be: input the defocus restored image into a pre-trained image depth prediction model, the image depth prediction model extracts the image features of the defocus restored image, and based on the relationship between the image features and the depth distance obtained in advance, obtains the depth distance of each pixel point in the defocus restored image, and generates a depth image based on the depth distance.

[0082] In practical applications, the process of training an image depth prediction model can be as follows: a large number of super-resolution images in an image library are input into the image depth prediction model to be trained, the image depth prediction model to be trained is trained, and the image depth prediction model to be trained extracts image features of the super-resolution image, and determines the depth distance of each pixel based on the image features. If the accuracy of the depth distance of the pixel output from the image depth prediction model to be trained is greater than the preset accuracy, it means that the trained image depth prediction model can accurately predict the depth distance of each pixel of the image. At this time, the trained image depth prediction model is determined as the trained image depth prediction model. Among them, the preset accuracy can be determined according to the actual situation and is not specifically limited here.

[0083] It can be seen that the technical solution of the embodiment of the present invention, when calculating the distance from each pixel point in the defocus restored image to the monocular camera, inputs the defocus restored image into a pre-trained defocus image restoration model, thereby efficiently and accurately obtaining the distance from each pixel point in the defocus restored image to the monocular camera.

[0084] It can be understood that the pixels of the defocused restored image have a one-to-one mapping relationship with the pixels of the original image, and the pixels of the depth image also have a one-to-one mapping relationship with the pixels of the defocused restored image, and then the pixels of the depth image also have a one-to-one mapping relationship with the pixels of the original image. After obtaining the distance from each pixel of the depth image to the monocular camera, by mapping each pixel of the depth image to the pixel of the original image, the distance from each pixel of the original image to the monocular camera can be accurately obtained.

[0085] S250: Determine a first region in the depth image where the depth value of the pixel is greater than the distance threshold.

[0086] The distance threshold is determined by multiplying the current speed of the humanoid robot by the preset time. Assuming the preset time is 5 seconds, the distance threshold is the current speed*5 seconds. Of course, the specific value of the preset time can also be determined according to the actual time, which is not specifically limited in the embodiment of the present invention. Moreover, when the humanoid robot is just started, the current running speed of the humanoid robot is the preset initial running speed, which can also be determined according to the actual situation, which is not specifically limited in the embodiment of the present invention.

[0087] Since the area with a distance greater than the distance threshold is far away from the humanoid robot at the current position, the humanoid robot will not move to the area far away from it. Therefore, in order to improve the response speed of the humanoid robot in determining the passable area, the first area with a pixel depth value greater than the distance threshold can be determined in the depth image. The first area is the impassable area determined by the depth image. In practical applications, the impassable area is a rectangular area, the width of the rectangular area is the width of the area composed of pixels with a pixel depth value greater than the distance threshold, and the height of the rectangular area can be the height of the depth image. The impassable area in the depth image can be one or more.

[0088] S260, based on the mapping relationship between the pixel points of the depth image and the pixel points of the original image, map each first area obtained in step S250 to the original image to obtain a second area corresponding to the first area; determine the first rectangular area with the height of the original image as the rectangular height and the width of each second area as the rectangular width; and subtract the first rectangular area from the entire area of ​​the original image to obtain the second rectangular area.

[0089] Specifically, in order to make the finally determined passable area correspond to the actual spatial area, after marking the impassable area in the depth image, it is also necessary to map the width of the impassable area in the depth image to the original image, that is, map the first area to the original image to obtain the second area, and the width of the second area is the width of the impassable area in the original image.

[0090] In practical applications, the size of the depth image may be the same as the size of the original image, and the size of the depth image may be different from the size of the original image. If the size of the depth image is the same as the size of the original image, the width of the second area in the mapped original image is the same as the width of the first area obtained in step S250. If the size of the depth image is different from the size of the original image, the width of the second area in the mapped original image is the same as the width of the first area obtained in step S250. Since the humanoid robot passes vertically, after obtaining the width of each second area in the original image, the width of each second area is used as the rectangular width, and the height of the original image is used as the rectangular height, and a first rectangular area is formed in the original image, which is the impassable area in the original image. Then, the first rectangular area is subtracted from the entire area of ​​the original image to obtain the second rectangular area. In practical applications, the number of second rectangular areas may be one or more.

[0091] S270, obtain the width of the humanoid robot, and compare the width of the second rectangular area obtained in step S260 with the width threshold, mark the rectangular area whose width of the second rectangular area in step S260 is less than the width threshold as a new inaccessible area, and add it to the set of inaccessible areas marked in step S260.

[0092] The width threshold is determined by the width of the humanoid robot and a preset multiple, for example, the preset multiple may be 2, in which case the width threshold is the width of the humanoid robot * 2. Of course, in practical applications, the specific value of the preset multiple may be determined based on actual conditions, as long as the humanoid robot can safely pass through the actual space area corresponding to the passable rectangular area.

[0093] If the width of a second rectangular area determined in step S260 is smaller than the width threshold, it means that a collision may occur when the humanoid robot passes through the actual space area corresponding to the second rectangular area, that is, it may not be able to pass through. Therefore, in order to ensure that the humanoid robot can pass safely, it is necessary to determine the second rectangular area with a smaller width as an inaccessible area and add it to the set of inaccessible rectangular areas marked in step S260.

[0094] S280, determining whether there is a third rectangular area whose width is greater than a width threshold in the second rectangular area excluding the impassable area in the original image.

[0095] Specifically, through steps S260 and S270, all the intraversable rectangular areas in the original image can be determined. In order to generate a traversable area for the humanoid robot, it is necessary to determine whether there is a third rectangular area with a width greater than a width threshold in the second rectangular area other than the intraversable area in the original image.

[0096] S290, if the third rectangular area does not exist in the original image, the humanoid robot rotates on the spot by a preset angle, determines the moving speed to be 0, and returns to execute step S230.

[0097] Specifically, if the original image contains only inaccessible areas, that is, the number of the remaining third rectangular areas in the original image is 0, then it means that the current humanoid robot has no accessible areas. At this time, the humanoid robot can be controlled to rotate in situ by a preset angle, and the preset angle can also be set according to the actual situation, and is not specifically limited here. For example, the preset angle can be +45°, indicating that the humanoid robot can turn 45° clockwise in situ, and the preset angle can be -45°, indicating that the humanoid robot can turn 45° counterclockwise in situ. After controlling the humanoid robot to rotate in situ by the preset angle, return to execute step S220, that is, regenerate the accessible area of ​​the humanoid robot.

[0098] S2100, if there is a third rectangular area in the original image, adjust the passing angle of the humanoid robot, and control the humanoid robot to pass in the space area corresponding to the third rectangular area according to the adjusted passing angle and the current passing speed, and return to execute step S320.

[0099] Specifically, if the number of the remaining third rectangular areas in the original image is greater than 1, it means that there is a passable area in the original image. At this time, the rectangular area with the largest width among all the third rectangular areas is selected as the passable area, and the coordinates (x, y) of the center point of the passable area are calculated. The distance d from the center point of the passable area to the monocular camera is obtained in the depth image; and the coordinates (x 0 ,y 0 ), arcsin(xx 0 ) / d as the travel direction angle of the humanoid robot, and keep the current travel speed unchanged. The humanoid robot is controlled to travel in the passable area with the largest width according to the travel direction angle and the current travel speed, and the process returns to step S220. The passable area of ​​the humanoid robot is determined in real time, and the humanoid robot is controlled to travel in the passable area, thereby realizing the real-time navigation function.

[0100] The technical solution provided by the embodiment of the present invention uses the monocular camera of the humanoid robot to collect the original image of the area to be passed in real time. Since the original image may be a defocused image with low clarity, the original image is defocused to obtain a defocused image, and a depth image is generated based on the distance from each pixel in the defocused image to the monocular camera. Since the image clarity after defocusing is high, the accuracy of the distance from each pixel to the monocular camera in the defocused image is also high, and the accuracy of the generated depth image is also high, that is, the accuracy of the depth value of each pixel in the generated depth image is high.

[0101] Then, the first area composed of pixels whose depth values ​​are greater than the distance threshold in the depth image is determined. Since the area whose depth value is greater than the distance threshold is far away from the humanoid robot, the humanoid robot will not move to the area far away from it at present. Therefore, the first area is the inaccessible area determined by the depth image. In addition, in order to make the finally determined traversable area correspond to the actual spatial area, it is necessary to map the first area in the depth image to the original image to obtain the second area corresponding to the first area; since the humanoid robot passes vertically, the rectangular area determined by the width of the second area and the height of the original image is determined as the inaccessible area of ​​the original image, and based on the inaccessible area of ​​the original image, the traversable area of ​​the humanoid robot is determined in the original image. Finally, the passing direction angle of the humanoid robot is determined, and according to the passing direction angle, the humanoid robot is controlled to pass in the traversable area.

[0102] It can be seen that the technical solution of the embodiment of the present invention can accurately generate the traversable area and traversable direction angle of the humanoid robot in real time, and control the humanoid robot to pass through the traversable area in real time according to the traversable direction angle, thereby achieving the purpose of real-time navigation. In addition, the humanoid robot can still work stably in complex environments and scenes such as complex lighting conditions, thereby enhancing robustness.

[0103] After describing the overall technical solution of this solution, a method for generating a traversable area for a humanoid robot provided by an embodiment of the present invention will be introduced below. Figure 3 As shown, a method for generating a traversable area for a humanoid robot provided in an embodiment of the present invention is applied to Figure 1 The humanoid robot shown includes a monocular camera, and the humanoid robot is equipped with a humanoid robot traversable area generation system. The humanoid robot traversable area generation method may include the following steps:

[0104] S310, when it is detected that the humanoid robot is powered on and starts to move, an original image of the area to be passed by the humanoid robot is obtained.

[0105] The original image is acquired in real time by a monocular camera, and the original image is an image of the area to be passed by the humanoid robot taken by the monocular camera, that is, the shooting direction of the monocular camera is toward the forward direction of the humanoid robot.

[0106] Specifically, when it is detected that the humanoid robot is powered on and starts to move, the monocular camera is started, and it is determined whether the monocular camera communication is normal; if the monocular camera communication is normal, it means that the monocular camera can work normally, that is, it can collect images in real time, and at this time, the next step S320 is executed. If the monocular camera communication is abnormal, it means that the monocular camera cannot work normally, that is, it cannot collect images, and at this time, it continues to determine whether the monocular camera communication is normal until it is determined that the monocular camera communication is normal.

[0107] When it is determined that the monocular camera communication is normal, the monocular camera can collect images of the area to be passed in real time. In order to describe the solution clearly, the image collected by the monocular camera can be called the original image.

[0108] S320, performing defocus recovery processing on the original image to obtain a defocus-recovered image, and generating a depth image based on the distance from each pixel in the defocus-recovered image to the monocular camera.

[0109] Since the original image directly captured by the monocular camera may be a defocused image with low clarity, in order to accurately generate the traversable area of ​​the humanoid robot in subsequent steps, the defocus recovery algorithm can be called to perform defocus recovery processing on the original image, that is, to restore the clarity of the original image and obtain the defocus recovered image.

[0110] After obtaining the defocused image, the depth estimation algorithm can be called to perform depth calculation on the defocused image to obtain the distance from each pixel in the defocused image to the monocular camera, and form a depth image with the distance as the depth. Since the image after defocusing is of high definition, the distance from each pixel to the monocular camera obtained by performing depth calculation on the defocused image is of high accuracy, and thus the generated depth image is of high accuracy, which is helpful in the subsequent steps to accurately determine the traversable area of ​​the humanoid robot.

[0111] In one implementation, performing defocus recovery processing on the original image to obtain a defocus-recovered image may include the following steps:

[0112] Inputting the original image into a pre-trained defocus image restoration model to perform defocus restoration processing on the original image through the defocus image restoration model, and outputting the defocus restored image;

[0113] Among them, the defocused image restoration model is trained by pre-collected defocused images and clear images corresponding to the defocused images.

[0114] It should be noted that the specific implementation method of "performing defocus recovery processing on the original image to obtain a defocus restored image" is as follows: Figure 2This has been explained in the illustrated embodiment and will not be described in detail here.

[0115] In one embodiment, generating a depth image based on the distance from each pixel in the defocused image to the monocular camera may include the following steps:

[0116] The defocused image is input into a pre-trained image depth prediction model, so that the image depth prediction model extracts image features of the defocused image, and based on the relationship between the image features and the depth distance, obtains the distance from each pixel in the defocused image to the monocular camera, and outputs a depth image with the distance as the depth;

[0117] Among them, the image depth distance prediction model is trained through clear images in the image library.

[0118] It should be noted that the specific implementation method of "performing defocus recovery processing on the original image to obtain a defocus restored image" is as follows: Figure 2 This has been explained in the illustrated embodiment and will not be described in detail here.

[0119] S330, determining a first region composed of pixel points having depth values ​​greater than a distance threshold in the depth image, and mapping the first region to the original image to obtain a second region.

[0120] The distance threshold is determined based on the current travel speed of the humanoid robot and a preset time duration.

[0121] Since the area with a distance greater than the distance threshold is far away from the humanoid robot at the current position, the humanoid robot will not move to the area far away from it. Therefore, in order to improve the response speed of the humanoid robot in determining the passable area, the first area with a pixel depth value greater than the distance threshold can be determined in the depth image. The first area is the impassable area determined by the depth image. In practical applications, the impassable area is a rectangular area, the width of the rectangular area is the width of the area composed of pixels with a pixel depth value greater than the distance threshold, and the height of the rectangular area can be the height of the depth image. The impassable area in the depth image can be one or more.

[0122] The distance threshold is determined by multiplying the current speed of the humanoid robot by the preset time. Assuming the preset time is 5 seconds, the distance threshold is the current speed*5 seconds.

[0123] In order to make the finally determined passable area correspond to the actual spatial area, after the first area is determined in the depth image, that is, after the impassable area is determined in the depth image, it is also necessary to map the first area in the depth image to the original image, by Figure 2From the description of the illustrated embodiment, it can be seen that: the pixel points of the depth image and the pixel points of the original image have a one-to-one mapping relationship. Therefore, after obtaining the first area in the depth image, the first area can be mapped to the original image according to the mapping relationship between the pixel points of the depth image and the pixel points of the original image to obtain a second area corresponding to the first area, which helps to accurately determine the inaccessible area in the original image in the subsequent steps.

[0124] As an implementation of the embodiment of the present disclosure, S330, determining a first area composed of pixels in the depth image whose depth values ​​are greater than a distance threshold, may include the following steps a1 to a3:

[0125] Step a1, obtaining the current travel speed and preset time of the humanoid robot, and determining the product of the current travel speed and the preset time as the distance threshold.

[0126] Step a2, searching the depth image for target pixels whose distance is greater than a distance threshold, and determining the width of the region formed by the target pixels.

[0127] Step a3: determining a first area composed of pixels having depth values ​​greater than a distance threshold in the depth image based on the width of the area composed of the target pixels and the height of the depth image.

[0128] Specifically, the distance threshold is determined by the product of the current speed of the humanoid robot and the preset time. Assuming the preset time is 5 seconds, the distance threshold is the current speed * 5 seconds. Since the area with a distance greater than the distance threshold is far away from the humanoid robot at the current position, the humanoid robot will not move to an area far away from it. Therefore, in order to improve the response speed of the humanoid robot in determining the passable area, the target pixel points with a depth value greater than the distance threshold can be found in the depth image, and the rectangular area determined by the width of the area composed of the target pixel points and the height of the depth image is determined as the first area, and the first area is the impassable area determined in the depth image.

[0129] S340: Determine an inaccessible area of ​​the original image.

[0130] The impassable area is a rectangular area determined based on the width of the second area and the height of the original image.

[0131] Since the first area is the impassable area in the depth image, and the second area is obtained by mapping the first area, the width of the second area is the width of the impassable area in the original image. And the humanoid robot passes vertically, so the rectangular area determined by the width of the second area and the height of the original image is the impassable area in the original image. In practical applications, the impassable area can be one or more.

[0132] S350, determining a traversable area of ​​the humanoid robot in the original image based on the intraversable area of ​​the original image.

[0133] Specifically, after obtaining the impassable area of ​​the original image, the impassable area can be subtracted from the entire area of ​​the original image to obtain one or more rectangular areas. The passable area can be determined from the obtained rectangular area. In practical applications, some rectangular areas may have a small width, and the humanoid robot cannot pass through the area with a small width. Therefore, it is necessary to determine whether there is a rectangular area with a width greater than a width threshold in the rectangular area.

[0134] The width threshold is determined by the width of the humanoid robot and a preset multiple, for example, the preset multiple may be 2, in which case the width threshold is the width of the humanoid robot * 2. Of course, in practical applications, the specific value of the preset multiple may be determined based on actual conditions, as long as the humanoid robot can safely pass through the passable area.

[0135] In one embodiment, based on the impassable area of ​​the original image, determining the traversable area of ​​the humanoid robot in the original image may include the following steps, namely, step b1 to step b4:

[0136] Step b1, calculating the width of each rectangular area in the original image except the impassable area.

[0137] Step b2, obtaining the width of the humanoid robot, and determining the product of the width of the humanoid robot and a preset multiple as a width threshold.

[0138] Step b3, comparing the width of each rectangular area obtained in step b1 with the width threshold.

[0139] Step b4: If there is a target rectangular area whose width is greater than the width threshold, the rectangular area with the largest width in the target rectangular area is determined as the passable area of ​​the humanoid robot.

[0140] Specifically, the width of each rectangular area except the impassable area in the original image can be calculated, and the width of each rectangular area can be compared with the width threshold. If there is a target rectangular area with a width greater than the width threshold, it means that the humanoid robot can safely and smoothly pass through the target rectangular area. Therefore, the target rectangular area can be determined as a passable area for the humanoid robot.

[0141] In practical applications, the number of target rectangular areas may be one or more. If the number of target rectangular areas is one, then the target rectangular area may be directly determined as the passable area of ​​the humanoid robot. If the number of target rectangular areas is more than one, then the rectangular area with the largest width among the target rectangular areas may be determined as the passable area of ​​the humanoid robot, so that in subsequent steps, the humanoid robot passes through the spacious passable area.

[0142] S360, determining the passing direction angle of the humanoid robot, and based on the passing direction angle, controlling the humanoid robot to pass through the passable area.

[0143] Specifically, after determining the passable area of ​​the humanoid robot, it is also necessary to determine the passing direction angle of the humanoid robot. In this way, the humanoid robot can be controlled to pass through the passable area according to the passing direction angle. It can be seen that real-time navigation can be achieved by accurately determining the passable area and passing direction angle of the humanoid robot in real time.

[0144] In order to describe the solution clearly, the specific implementation of S360 will be described in detail in the following embodiments.

[0145] The technical solution provided by the embodiment of the present invention uses the monocular camera of the humanoid robot to collect the original image of the area to be passed in real time. Since the original image may be a defocused image with low clarity, the original image is defocused to obtain a defocused image, and a depth image is generated based on the distance from each pixel in the defocused image to the monocular camera. Since the image clarity after defocusing is high, the accuracy of the distance from each pixel to the monocular camera in the defocused image is also high, and the accuracy of the generated depth image is also high, that is, the accuracy of the depth value of each pixel in the generated depth image is high.

[0146] Then, the first area composed of pixels whose depth values ​​are greater than the distance threshold in the depth image is determined. Since the area whose depth value is greater than the distance threshold is far away from the humanoid robot, the humanoid robot will not move to the area far away from it at present. Therefore, the first area is the inaccessible area determined by the depth image. In addition, in order to make the finally determined traversable area correspond to the actual spatial area, it is necessary to map the first area in the depth image to the original image to obtain the second area corresponding to the first area; since the humanoid robot passes vertically, the rectangular area determined by the width of the second area and the height of the original image is determined as the inaccessible area of ​​the original image, and based on the inaccessible area of ​​the original image, the traversable area of ​​the humanoid robot is determined in the original image. Finally, the passing direction angle of the humanoid robot is determined, and according to the passing direction angle, the humanoid robot is controlled to pass in the traversable area.

[0147] It can be seen that the technical solution of the embodiment of the present invention can accurately generate the traversable area and traversable direction angle of the humanoid robot in real time, and control the humanoid robot to pass through the traversable area in real time according to the traversable direction angle, thereby achieving the purpose of real-time navigation. In addition, the humanoid robot can still work stably in complex environments and scenes such as complex lighting conditions, thereby enhancing robustness.

[0148] exist Figure 3 Based on the embodiment shown, Figure 4 As shown, in one embodiment, S360, determining the passing direction angle of the humanoid robot, and controlling the humanoid robot to pass through the passable area based on the passing direction angle, may include the following steps:

[0149] S361, determine the coordinates of a first central pixel point of the passable area, and determine the coordinates of a second central pixel point of the original image.

[0150] Specifically, in order to calculate the passing direction angle of the humanoid robot, after determining the passable area, it is necessary to determine the coordinates of the central pixel point of the passable area. In order to clearly describe the scheme, the central pixel point is called the first central pixel point, and the coordinates of the first central pixel point are marked as (x, y); and determine the coordinates of the central pixel point of the original image. In order to clearly describe the scheme, the central pixel point is called the second central pixel point, and the coordinates of the second central pixel point are marked as (x 0 ,y 0 ).

[0151] S362: Determine the depth value of the first central pixel.

[0152] The depth value of the first central pixel is the depth value of the central pixel of the depth image.

[0153] Since the pixels of the original image and the pixels of the depth image have a one-to-one mapping relationship, and the first central pixel and the central pixel of the depth image have a mapping relationship, the depth value of the central pixel of the depth image can be determined as the depth value of the first central pixel, and the depth value can be recorded as d.

[0154] S363, calculating the passing direction angle of the humanoid robot based on the coordinates of the first central pixel point, the coordinates of the second central pixel point, and the depth value of the first central pixel point.

[0155] After obtaining the coordinates (x, y) of the first central pixel point and the coordinates (x 0 ,y 0 ), and the depth value d of the first central pixel, arcsin(xx 0) / d is used as the travel direction angle of the humanoid robot.

[0156] S364, controlling the humanoid robot to pass through the passable area according to the passing direction angle and current passing speed of the humanoid robot, and returning to execute the step of obtaining the original image of the area to be passed by the humanoid robot.

[0157] After the travel direction angle of the humanoid robot is determined, the travel direction of the humanoid robot can be determined. In addition, the humanoid robot can maintain the current travel speed. In other words, the humanoid robot is controlled to travel in the passable area according to the travel direction angle and the current travel speed, and the process returns to S310 to obtain the original image of the area to be passed by the humanoid robot.

[0158] It can be seen that the technical solution of the embodiment of the present invention can generate the traversable area of ​​the humanoid robot in real time and accurately, and can also determine the passing angle and passing speed of the humanoid robot in real time and accurately, thereby achieving the purpose of real-time navigation.

[0159] exist Figure 3 Based on the illustrated embodiment, in one implementation, the method for generating a traversable area for a humanoid robot may further include the following steps:

[0160] If there is no passable area for the humanoid robot in the original image, the humanoid robot is controlled to rotate at a preset angle on the spot, and the passing speed of the humanoid robot is controlled to be zero.

[0161] Return to the step of obtaining the original image of the area to be passed by the humanoid robot.

[0162] Specifically, if the original image contains only inaccessible areas, that is, the number of remaining accessible areas in the original image is 0, indicating that the current humanoid robot has no accessible areas. At this time, the humanoid robot can be controlled to rotate in situ by a preset angle, and the preset angle can also be set according to actual conditions, and is not specifically limited here. For example, the preset angle can be +45°, indicating that the humanoid robot can turn 45° clockwise in situ, and the preset angle can be -45°, indicating that the humanoid robot can turn 45° counterclockwise in situ. After controlling the humanoid robot to rotate in situ by the preset angle, return to step S220, that is, regenerate the accessible area of ​​the humanoid robot.

[0163] It can be seen that through the technical solution provided by this embodiment, if there is currently no passable area, the humanoid robot can be controlled to rotate on the spot and the passable area of ​​the humanoid robot can be re-determined, that is, the route for the humanoid robot can be re-planned.

[0164] In a second aspect, an embodiment of the present invention provides a device for generating a traversable area for a humanoid robot, which is applied to a humanoid robot including a monocular camera, such as Figure 5 As shown, the device comprises:

[0165] The original image acquisition module 510 is used to acquire the original image of the area to be passed by the humanoid robot when it is detected that the humanoid robot is powered on and starts to move, wherein the original image is acquired in real time by the monocular camera;

[0166] A depth image generation module 520 is used to perform defocus recovery processing on the original image to obtain a defocus-recovered image, and to generate a depth image based on the distance from each pixel in the defocus-recovered image to the monocular camera;

[0167] The region determination module 530 is used to determine a first region composed of pixel points in the depth image whose depth values ​​are greater than a distance threshold, and map the first region to the original image to obtain a second region; the distance threshold is determined based on the current travel speed of the humanoid robot and a preset time length;

[0168] An impassable area determination module 540 is used to determine an impassable area of ​​the original image, where the impassable area is a rectangular area determined based on the width of the second area and the height of the original image;

[0169] The traversable area determination module 550 is used to determine the traversable area of ​​the humanoid robot in the original image based on the intraversable area of ​​the original image.

[0170] The humanoid robot control module 560 is used to determine the passing direction angle of the humanoid robot, and based on the passing direction angle, control the humanoid robot to pass through the passable area.

[0171] Optionally, the area determination module is specifically used to:

[0172] Acquire the current travel speed of the humanoid robot, and determine the product of the current travel speed and the preset time as the distance threshold;

[0173] Searching for target pixel points whose distance is greater than the distance threshold in the depth image, and determining the width of the area formed by the target pixel points;

[0174] Based on the width of the region composed of the target pixel points and the height of the depth image, a first region composed of pixel points having depth values ​​greater than a distance threshold in the depth image is determined.

[0175] Optionally, the traversable area determination module is specifically used to:

[0176] Calculating the width of each rectangular area in the original image except the impassable area;

[0177] Acquire the width of the humanoid robot, and determine the product of the width of the humanoid robot and a preset multiple as a width threshold;

[0178] Compare the width of each rectangular area with the width threshold respectively;

[0179] If there is a target rectangular area with a width greater than the width threshold, the rectangular area with the largest width in the target rectangular area is determined as the passable area of ​​the humanoid robot.

[0180] Optionally, the humanoid robot control module is specifically used to:

[0181] Determine the coordinates of a first central pixel point of the passable area, and determine the coordinates of a second central pixel point of the original image;

[0182] Determine a depth value of the first central pixel, where the depth value of the first central pixel is the depth value of the central pixel of the depth image;

[0183] Calculating a passing direction angle of the humanoid robot based on the coordinates of the first central pixel point, the coordinates of the second central pixel point, and the depth value of the first central pixel point;

[0184] According to the traveling direction angle and current traveling speed of the humanoid robot, the humanoid robot is controlled to travel in the travelable area, and the step of obtaining the original image of the area to be traveled by the humanoid robot is returned to be executed.

[0185] Optionally, the humanoid robot control module is further used to:

[0186] If the traversable area of ​​the humanoid robot does not exist in the original image, controlling the humanoid robot to rotate on the spot by a preset angle, and controlling the traversing speed of the humanoid robot to be zero;

[0187] Return to the step of acquiring the original image of the area to be passed by the humanoid robot.

[0188] Optionally, the depth image generation module is specifically used to:

[0189] Inputting the original image into a pre-trained defocus image restoration model to perform defocus restoration processing on the original image through the defocus image restoration model, and outputting the defocus restored image;

[0190] The defocused image restoration model is obtained by training with a pre-collected defocused image and a clear image corresponding to the defocused image.

[0191] Optionally, the depth image generation module is specifically used to:

[0192] Inputting the defocused image into a pre-trained image depth prediction model, so that the image depth prediction model extracts image features of the defocused image, obtains the distance from each pixel in the defocused image to the monocular camera based on the relationship between the image features and the depth distance, and outputs a depth image with the distance as the depth;

[0193] The image depth distance prediction model is obtained by training with clear images in an image library.

[0194] In a third aspect, an embodiment of the present invention provides a humanoid robot 600, such as Figure 6 As shown, including:

[0195] The monocular camera 601 is used to collect the original image of the area to be passed by the humanoid robot in real time;

[0196] At least one processor 602, configured to acquire the original image from the monocular camera when detecting that the humanoid robot is powered on and starts to move;

[0197] A memory 603 for storing the at least one processor executable instruction;

[0198] Wherein, the at least one processor is configured to execute the instructions to implement the traversable area generation method described in the above embodiment of the humanoid robot.

[0199] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to execute the method for generating a traversable area for a humanoid robot described in the above embodiment.

[0200] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention without departing from the principles and intent of the present invention.

Claims

1. A method for generating a traversable area for a humanoid robot, characterized in that: Applied to a humanoid robot including a monocular camera, the method comprises: When it is detected that the humanoid robot is powered on and starts to move, an original image of the area to be passed by the humanoid robot is obtained, wherein the original image is acquired in real time by the monocular camera; Performing defocus recovery processing on the original image to obtain a defocus-recovered image, and generating a depth image based on the distance from each pixel in the defocus-recovered image to the monocular camera; Determine a first region composed of pixel points in the depth image whose depth values ​​are greater than a distance threshold, and map the first region to the original image to obtain a second region; the distance threshold is determined based on the current travel speed of the humanoid robot and a preset time length; Determine an impassable area of ​​the original image, where the impassable area is: a rectangular area determined based on a width of the second area and a height of the original image; Determining a traversable area of ​​the humanoid robot in the original image based on the intraversable area of ​​the original image; The passing direction angle of the humanoid robot is determined, and based on the passing direction angle, the humanoid robot is controlled to pass through the passable area.

2. The method according to claim 1, characterized in that The determining a first area composed of pixel points having depth values ​​greater than a distance threshold in the depth image includes: Acquire the current travel speed and preset time of the humanoid robot, and determine the product of the current travel speed and the preset time as the distance threshold; Searching for target pixel points whose distance is greater than the distance threshold in the depth image, and determining the width of the area formed by the target pixel points; Based on the width of the region composed of the target pixel points and the height of the depth image, a first region composed of pixel points having depth values ​​greater than a distance threshold in the depth image is determined.

3. The method according to claim 1, characterized in that The determining of the traversable area of ​​the humanoid robot in the original image based on the impassable area of ​​the original image comprises: Calculating the width of each rectangular area in the original image except the impassable area; Acquire the width of the humanoid robot, and determine the product of the width of the humanoid robot and a preset multiple as a width threshold; Compare the width of each rectangular area with the width threshold respectively; If there is a target rectangular area with a width greater than the width threshold, the rectangular area with the largest width in the target rectangular area is determined as the passable area of ​​the humanoid robot.

4. The method according to any one of claims 1 to 3, characterized in that: The determining of the passing direction angle of the humanoid robot and controlling the humanoid robot to pass through the passable area based on the passing direction angle includes: Determine the coordinates of a first central pixel point of the passable area, and determine the coordinates of a second central pixel point of the original image; Determine a depth value of the first central pixel, where the depth value of the first central pixel is the depth value of the central pixel of the depth image; Calculating a passing direction angle of the humanoid robot based on the coordinates of the first central pixel point, the coordinates of the second central pixel point, and the depth value of the first central pixel point; According to the traveling direction angle and current traveling speed of the humanoid robot, the humanoid robot is controlled to travel in the travelable area, and the step of obtaining the original image of the area to be traveled by the humanoid robot is returned to be executed.

5. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: If the traversable area of ​​the humanoid robot does not exist in the original image, controlling the humanoid robot to rotate on the spot by a preset angle, and controlling the traversing speed of the humanoid robot to be zero; Return to the step of acquiring the original image of the area to be passed by the humanoid robot.

6. The method according to any one of claims 1 to 3, characterized in that: The performing defocus recovery processing on the original image to obtain a defocus-recovered image includes: Inputting the original image into a pre-trained defocus image restoration model to perform defocus restoration processing on the original image through the defocus image restoration model, and outputting the defocus restored image; The defocused image restoration model is obtained by training with a pre-collected defocused image and a clear image corresponding to the defocused image.

7. The method according to any one of claims 1 to 3, characterized in that: The generating a depth image based on the distance from each pixel in the defocused restored image to the monocular camera comprises: Inputting the defocused image into a pre-trained image depth prediction model, so that the image depth prediction model extracts image features of the defocused image, obtains the distance from each pixel in the defocused image to the monocular camera based on the relationship between the image features and the depth distance, and outputs a depth image with the distance as the depth; The image depth distance prediction model is obtained by training with clear images in an image library.

8. A device for generating a traversable area for a humanoid robot, characterized in that: Applied to a humanoid robot including a monocular camera, the device comprises: An original image acquisition module, used for acquiring an original image of the area to be passed by the humanoid robot when it is detected that the humanoid robot is powered on and starts to move, wherein the original image is acquired in real time by the monocular camera; A depth image generation module, used to perform defocus recovery processing on the original image to obtain a defocus-recovered image, and generate a depth image based on the distance from each pixel in the defocus-recovered image to the monocular camera; A region determination module, configured to determine a first region composed of pixels in the depth image whose depth values ​​are greater than a distance threshold, and map the first region to the original image to obtain a second region; the distance threshold is determined based on the current travel speed of the humanoid robot and a preset duration; An impassable area determination module, used to determine an impassable area of ​​the original image, wherein the impassable area is: a rectangular area determined based on a width of the second area and a height of the original image; A traversable area determination module is used to determine a traversable area of ​​the humanoid robot in the original image based on the intraversable area of ​​the original image. The humanoid robot control module is used to determine the passing direction angle of the humanoid robot and control the humanoid robot to pass through the passable area based on the passing direction angle.

9. A humanoid robot, characterized in that: include: A monocular camera, used to collect original images of the area to be passed by the humanoid robot in real time; at least one processor, configured to acquire the original image from the monocular camera when detecting that the humanoid robot is powered on and starts to move; a memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 7.

Images