Embossed robots, their following control methods, and embodied robot systems

Abstract

This application relates to the field of android technology, providing an android robot, its following control method, and an android robot system. The method includes: acquiring images captured by a monocular camera on the android robot and detection data collected by an obstacle detection sensor; if a moving target is detected in the image, determining a following lateral scale for the moving target in the lateral direction of the image; if an obstacle is determined to exist within a preset range of the android robot based on the detection data, determining an obstacle avoidance lateral scale based on the detection data, wherein the obstacle avoidance lateral scale and the following lateral scale are on the same scale; and controlling the android robot to perform obstacle avoidance following based on the following lateral scale and the obstacle avoidance lateral scale. This application enables accurate obstacle avoidance following of a moving target even when sensors cannot obtain accurate depth information of the moving target.

Description

Technical Field

[0001] This application belongs to the field of embodied robot technology, and in particular relates to an embodied robot, its following control method, and an embodied robot system. Background Technology

[0002] In existing technologies, to enable robots to accompany family members or collaborate with humans in performing tasks, the robot is guided to follow a moving target. The conventional method involves using vision, lidar, or other sensors to identify and locate the moving target, determine its relative pose to the robot, then calculate the target's speed and direction using consecutive frames to estimate its short-term trajectory (i.e., predicted position). Finally, the robot uses this predicted position as its navigation target and plans its path based on a pre-built map. The robot then moves towards the predicted position along the planned path, thus achieving human-guided movement.

[0003] However, when sensors cannot obtain accurate depth information of a moving target (for example, accurate depth information can be obtained through a binocular camera), the robot can only detect whether the moving target is to the left or right relative to itself, but cannot obtain accurate position information of the moving target, and therefore cannot accurately avoid obstacles and follow the moving target. Summary of the Invention

[0004] This application provides an embodied robot, its following control method, and an embodied robot system, which can accurately achieve obstacle avoidance and following of a moving target when sensors cannot obtain accurate depth information of the moving target.

[0005] In a first aspect, embodiments of this application provide a following control method for an embodied robot, including: Acquire images captured by a monocular camera and detection data collected by an obstacle detection sensor on the embodied robot; If a moving target is detected in the image, the following horizontal scale of the moving target in the horizontal direction of the image is determined, and the following horizontal scale is used to characterize the offset direction and degree of the moving target relative to the center position of the image; If, based on the detection data, it is determined that there is an obstacle within a preset range of the robot, then an obstacle avoidance lateral scale is determined based on the detection data. The obstacle avoidance lateral scale and the following lateral scale are on the same scale dimension. The obstacle avoidance lateral scale is used to characterize the lateral offset direction and degree required for the robot to avoid the obstacle. Based on the following lateral scale and the obstacle avoidance lateral scale, the robot is controlled to perform obstacle avoidance and following. The obstacle refers to any object other than the moving target.

[0006] In this embodiment, by acquiring images from a monocular camera on the android and detection data from an obstacle detection sensor, and when a moving target is detected in the image, the following lateral scale of the moving target in the lateral direction of the image is determined. If, based on the detection data, an obstacle is determined within a preset range of the android, an obstacle avoidance lateral scale with the same dimensionality as the following lateral scale is determined based on the detection data. This obstacle avoidance lateral scale characterizes the lateral offset direction and degree required for the android to avoid the obstacle. The android can be controlled to perform obstacle avoidance following based on the following and obstacle avoidance lateral scales. In the above obstacle avoidance following control process, accurate depth information and a pre-built map are not required. Therefore, obstacle avoidance following of the moving target can be accurately achieved even when the sensor cannot obtain accurate depth information of the moving target.

[0007] In some embodiments of the first aspect, determining the following lateral scale of the moving target in the lateral direction of the image includes: Determine the current horizontal scale of the moving target in the horizontal direction of the image; Perform target processing on the current horizontal scale to obtain the following horizontal scale; The target processing includes at least one of scale range constraint and time-series smoothing processing.

[0008] In some embodiments of the first aspect, the step of performing target processing on the current horizontal scale to obtain the following horizontal scale includes: When the target processing includes the scale range constraint and the timing smoothing processing, the current horizontal scale is clipped to a preset horizontal scale range to obtain the constrained horizontal scale. The constrained horizontal scale is subjected to time-series smoothing to obtain the following horizontal scale.

[0009] In some embodiments of the first aspect, the timing smoothing process includes: Determine the horizontal scale to be smoothed, wherein the horizontal scale to be smoothed is the current horizontal scale, or is the constrained horizontal scale obtained by constraining the scale range of the current horizontal scale; Subtract the previous smoothed horizontal scale from the current smoothed horizontal scale, multiply by the smoothing coefficient, and then add it to the previous smoothed horizontal scale to obtain the current smoothed horizontal scale. Use the current smooth horizontal scale as the following horizontal scale.

[0010] In some embodiments of the first aspect, the smoothing coefficient is dynamically determined based on at least one of the type of the moving target, the speed of the moving target, and the magnitude of the change between the horizontal scale to be smoothed and the horizontal scale at the previous moment.

[0011] In some embodiments of the first aspect, after determining the following lateral scale of the moving target in the lateral direction of the image, the method further includes: Based on the aforementioned horizontal scale, the speeds of the left and right wheels of the robot are controlled to follow the moving target.

[0012] In some embodiments of the first aspect, determining the obstacle avoidance lateral scale based on the detection data includes: Based on the detection data, the obstacle point data in the robot coordinate system is determined; Based on the distribution of the obstacle points in the lateral direction of the robot coordinate system, a lateral no-entry zone is generated; Within a preset lateral search range of the robot coordinate system, determine the target passable area from other lateral regions besides the lateral no-entry zone; The obstacle avoidance lateral scale is determined based on the lateral coordinates of the midpoint of the target passable area.

[0013] In some embodiments of the first aspect, generating a lateral no-entry zone based on the distribution of the obstacle point data in the lateral direction of the robot coordinate system includes: Based on the lateral coordinates and preset expansion of each obstacle point in the obstacle point data, the lateral restricted area corresponding to each obstacle point is determined. The overlapping lateral restricted areas are merged to obtain the lateral restricted area.

[0014] In some embodiments of the first aspect, determining the target passable area from lateral areas other than the lateral restricted area includes: Multiple candidate passable areas are identified from other lateral areas besides the aforementioned lateral restricted areas; Based on the passage costs corresponding to the multiple candidate passable areas, the target passable area is determined from the multiple candidate passable areas. The passage cost is positively correlated with the distance between the candidate passable area and the lateral centerline of the robot coordinate system, and negatively correlated with the channel width of the candidate passable area.

[0015] In some embodiments of the first aspect, determining the obstacle avoidance lateral scale based on the lateral coordinate value of the midpoint of the target passable area includes: Based on a preset scale conversion coefficient, the lateral coordinate value of the midpoint of the target passable area is converted to obtain the obstacle avoidance lateral scale.

[0016] In some embodiments of the first aspect, controlling the android to perform obstacle avoidance following based on the following lateral scale and the obstacle avoidance lateral scale includes: Based on obstacle avoidance weights, the following lateral scale and the obstacle avoidance lateral scale are weighted and fused to obtain a fused lateral scale. The obstacle avoidance weights are negatively correlated with the obstacle distance, and the obstacle distance refers to the distance between the embodied robot and the obstacle. Based on the fused lateral scale, the speeds of the left and right wheels of the robot are controlled to enable the robot to perform obstacle avoidance and following.

[0017] In some embodiments of the first aspect, during obstacle avoidance and following of the moving target, the method further includes: If the obstacle detection sensor changes from detecting the obstacle to not detecting the obstacle, then for a preset time, the robot continues to be controlled to perform obstacle avoidance and following based on the following lateral scale and the obstacle avoidance lateral scale determined when the obstacle was last detected.

[0018] Secondly, embodiments of this application provide a following control device for an embodied robot, comprising: The data acquisition module is used to acquire images captured by the monocular camera on the embodied robot and detection data collected by the obstacle detection sensor; The follow determination module is used to determine the follow horizontal scale of the moving target in the horizontal direction of the image when a moving target is detected in the image. The follow horizontal scale is used to characterize the offset direction and offset degree of the moving target relative to the center position of the image. The obstacle avoidance determination module is used to determine an obstacle avoidance lateral scale based on the detection data if an obstacle is determined to exist within a preset range of the robot based on the detection data. The obstacle avoidance lateral scale and the following lateral scale are in the same scale dimension. The obstacle avoidance lateral scale is used to characterize the lateral offset direction and degree of lateral offset required for the robot to avoid the obstacle. An obstacle avoidance control module is used to control the android to perform obstacle avoidance following based on the following lateral scale and the obstacle avoidance lateral scale; The obstacle refers to any object other than the moving target.

[0019] Thirdly, embodiments of this application provide a hymenoid robot, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the hymenoid robot performs the method as described in any one of the first aspects above.

[0020] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a computer, implements the method as described in any one of the first aspects above.

[0021] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when run, causes the method as described in any one of the first aspects above to be performed.

[0022] Sixthly, embodiments of this application provide a hymenoid robot system, including a monocular camera and a control device, wherein the monocular camera is connected to the control device; The monocular camera is used to detect moving targets; The control device is used to perform the method as described in any one of the first aspects above.

[0023] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of an application scenario of the embodied robot system provided in the embodiments of this application; Figure 2 This is a flowchart illustrating the following control method for an embodied robot provided in an embodiment of this application; Figure 3 This is another schematic flowchart of the following control method for the embodied robot provided in the embodiments of this application; Figure 4 This is another schematic flowchart of the following control method for the embodied robot provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of the follow-up control device for the embodied robot provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of the embodied robot provided in the embodiments of this application. Detailed Implementation

[0026] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0027] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0028] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0029] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0030] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0031] The term "embedded robot" in this application refers to a robot that possesses a physical entity and achieves intelligent behavior through real-time perception, interaction, and action with its real environment. The physical entity refers to the embedded robot having a real body (such as a robotic arm, mobile chassis, sensors, etc.) and being able to move in the physical world like a human or animal (such as walking, grasping, obstacle avoidance, etc.). It should be understood that embedded robots can be categorized into various types based on their function and application scenarios, including but not limited to companion robots and collaborative robots. It is understood that this application does not limit the form of the various robots listed above; for example, they can be wheeled robots, legged robots, etc.

[0032] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0033] Please see Figure 1 , Figure 1 This illustration shows an application scenario of the embodied robot system provided in this application embodiment. The embodied robot system may include an embodied robot 101, a monocular camera 102, a moving target 103, and a control device 104. The control device 104 can communicate with the embodied robot 101 and the monocular camera 102 via wired or wireless means.

[0034] The number of embodied robots 101 can be one or more (i.e., at least two). The embodied robots 101 can be used to follow one or more pre-set moving targets 103. It should be understood that this application does not limit the specific type of moving target 103, such as a person, a pet, or other moving robots.

[0035] As an example and not a limitation, the embodied robot 101 is a companion robot that can move in companionship scenarios (such as home environments) and follow family members such as children and the elderly, or pets such as dogs and cats.

[0036] The monocular camera 102 can be integrated into or mounted on the embodied robot 101. It is used to acquire real-time images of the environment in which the embodied robot 101 is located. Based on these images, moving targets 103 in the environment can be detected, thereby enabling the embodied robot 101 to accurately follow the moving targets 103 in its environment. It should be understood that this application does not limit the position of the monocular camera 102 on the embodied robot 101. For example, the monocular camera 102 can be positioned anywhere on the nose, head, torso, or arm of the embodied robot 101.

[0037] The control device 104 may include at least one of terminal devices, servers, etc. This application does not limit the specific type of the control device 104, which can be selected according to actual needs.

[0038] Terminal devices can be mobile phones, tablets, wearable devices, in-vehicle devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and other devices.

[0039] The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or any of the following: cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), big data or artificial intelligence platforms, etc. There are no restrictions here.

[0040] The control device 104 can execute the following control method for the embodied robot provided in the embodiments of this application. For specific implementation schemes, please refer to the relevant description of the method embodiments below.

[0041] Please see Figure 2 , Figure 2 This paper illustrates a flowchart of a follow-up control method for an embodied robot provided in an embodiment of this application. This follow-up control method can be applied to, for example... Figure 1 The embodied robot 101 in the embodied robot system shown can also be applied to, for example... Figure 1 The control device 104 in the embodied robot system is shown. By way of example and not limitation, the application of this follow-control method to the embodied robot 101 is used as an example. Figure 2 The process shown is described in detail. This follow-up control method includes the following steps: Step 201: Obtain images captured by the monocular camera on the embodied robot and detection data collected by the obstacle detection sensor.

[0042] The monocular camera captures images of the environment in which the embodied robot is located. For example, it captures images of the environment in front of the embodied robot.

[0043] Detection data refers to raw data collected in real time by obstacle detection sensors to perceive the presence and location of obstacles in the environment in which the embodied robot is located. The data format depends on the specific type of obstacle detection sensor. For example, detection data may include point cloud data, distance, or echo signals.

[0044] In some embodiments of this application, images acquired by a monocular camera and detection data acquired by an obstacle detection sensor can be used in the same control calculation. It is understood that the monocular camera and obstacle detection sensor can acquire data synchronously or asynchronously. In the case of asynchronous acquisition, images and detection data with a time difference less than a preset time difference threshold can be used in the same control calculation; or, images and detection data acquired within the same control cycle can be used in the same control calculation. This avoids using outdated images or detection data for follow-up and obstacle avoidance control, improving the accuracy and stability of the robot's control when following a moving target and avoiding obstacles.

[0045] Step 202: If a moving target is detected in the image, determine the following horizontal scale of the moving target in the horizontal direction of the image. The following horizontal scale is used to characterize the offset direction and degree of the moving target relative to the center position of the image.

[0046] In some embodiments, a moving target may refer to a movable object that the embodied robot needs to follow. By way of example and not limitation, a moving target may include a person, a pet, another robot, a vehicle, or a movable object carrying a preset identifier (such as a color mark, a reflective mark, a pattern mark, etc.). The moving target may be specified in advance by the user or automatically determined by the embodied robot based on image detection results, target category, target confidence level, target size, target location, or historical following status.

[0047] In some embodiments, images can be acquired in real time using a monocular camera. For each frame of the acquired image, a pre-trained target detection model or a preset image processing algorithm (such as background subtraction, optical flow combined with a classifier) ​​can be used to detect moving targets in the image. The detection of moving targets can include the detection of the target bounding box, target center, key point center, or contour region of the moving target.

[0048] In some embodiments, multiple moving targets (e.g., multiple people walking) may exist simultaneously in the image captured by a monocular camera. To avoid tracking confusion, a moving target can be selected as the tracking object from among the multiple moving targets based on a preset selection rule. Optionally, the selection rule can be set according to actual needs. For example, the selection rule may include at least one of the following: Confidence selection rule: Select the moving target with the highest confidence level; Center selection rule: Select the moving target that is closest to the center of the image; Size selection rule: Select the moving target with the largest area in the image; User selection rules: Receive user voice commands or touch selections, and use the user's selected movement target as the follow object; Historical continuation selection rule: If there is already a moving target being followed, then the target will be followed first until the time lost by the target exceeds a preset time threshold before a new target is selected.

[0049] When there are multiple selection rules, the priorities of multiple selection rules can be preset. The rules are judged in descending order of priority. Once the condition is met, the following object is determined and the selection rules with lower priority are no longer considered.

[0050] In this embodiment, the above selection rules enable the embodied robot to stably and accurately select and follow the user's desired moving target in a multi-target environment.

[0051] In some embodiments, following the lateral scale refers to the lateral control quantity used to control the android to follow a moving target. The following lateral scale characterizes the positional offset of the moving target in the lateral direction of the image, including the direction and degree of offset of the moving target relative to the center position of the image. The following lateral scale can be determined based on the current lateral scale of the moving target in the lateral direction of the image. For example, the current lateral scale can be used directly as the following lateral scale, or the following lateral scale can be obtained after applying scale range constraints and / or temporal smoothing to the current lateral scale.

[0052] It should be understood that the following lateral scale is a control position quantity determined based on the image field of view. It can be a pixel offset, a normalized offset, a discrete level value, or a continuous scale value. It does not need to represent the actual lateral distance between the moving target and the embodied robot, nor does it need to represent the actual lateral coordinates of the moving target in the robot coordinate system.

[0053] Monocular cameras typically struggle to directly acquire accurate depth information of a moving target relative to a robot. Therefore, estimating the true lateral distance of the moving target in the robot's coordinate system based on a monocular camera can easily lead to inaccurate or inconsistent lateral distances due to depth errors. This application's embodiments do not require determining the true lateral distance of the moving target relative to the robot based on a monocular camera. Instead, it determines the following lateral scale based on the moving target's positional offset in the lateral direction of the image, and controls the robot's left and right wheel speeds based on this following lateral scale. This reduces the dependence of the following control process on accurate depth information and improves the stability of following control under monocular vision conditions.

[0054] Step 203: If it is determined from the detection data that there is an obstacle within the preset range of the embodied robot, then the obstacle avoidance lateral scale is determined from the detection data. The obstacle avoidance lateral scale and the following lateral scale are under the same scale dimension. The obstacle avoidance lateral scale is used to characterize the lateral offset direction and degree of lateral offset required for the embodied robot to avoid the obstacle.

[0055] Here, obstacles refer to objects other than the moving target.

[0056] Optionally, a preset range can be set based on actual needs or empirical values; it can also be set based on the capabilities of the embodied robot itself and the obstacle detection sensor. For example, if the obstacle detection sensor is a lidar, and the embodied robot is allowed to make sharp turns, a larger preset range can be set, such as: , In this system, a robot coordinate system is established with the robot itself as the origin, the orientation direction is the X-axis, the horizontal direction is the Y-axis, R is the radius of the robot body, and x and y represent the X-axis coordinates and Y-axis coordinates in the robot coordinate system, respectively.

[0057] In some embodiments of this application, the obstacle avoidance lateral scale and the following lateral scale are under the same scale dimension. This can be understood as both the following lateral scale and the obstacle avoidance lateral scale using the same lateral scale expression method to represent the lateral offset direction and degree. For example, the following lateral scale can be determined by the positional offset of the moving target in the lateral direction of the image, and the obstacle avoidance lateral scale can be determined by the detection data collected by the obstacle detection sensor after scale conversion. Although the data sources of the two are different, both can be represented as lateral control quantities within the same scale range, for example, both can be represented as normalized lateral scales within the range of [-1, 1], or both can be represented as a preset number of discrete lateral scales. Thus, the following lateral scale and the obstacle avoidance lateral scale can be directly compared, weighted and fused, or used together to determine the control quantities of the embodied robot, thereby achieving obstacle avoidance while following the moving target. The aforementioned same scale dimension does not require that the two be obtained from the same sensor or the same calculation method, but rather means that after appropriate conversion, they have the same scale meaning and scale range, and both can characterize the lateral offset direction and degree.

[0058] Step 204: Based on the following lateral scale and the obstacle avoidance lateral scale, control the embodied robot to perform obstacle avoidance and following.

[0059] In this embodiment, the following lateral scale is a control quantity calculated based on the lateral offset of the moving target in the image. Its function is to guide the embodied robot to turn in the direction of the moving target. The obstacle avoidance lateral scale is a control quantity calculated based on the detection data collected by the obstacle detection sensor. Its function is to guide the embodied robot to turn away from the obstacle. The obstacle avoidance lateral scale can be determined according to the position of the obstacle in the robot coordinate system, the lateral no-go zone, the passable area, or the passable area of ​​the target. It is used to characterize the lateral direction and degree to which the embodied robot should deflect or move in order to avoid the obstacle. That is to say, the obstacle avoidance lateral scale does not necessarily represent the lateral position of the obstacle itself, but rather the lateral control quantity corresponding to the avoidance direction determined based on the obstacle's position. Therefore, based on the following lateral scale and the obstacle avoidance lateral scale, the embodied robot can be controlled to avoid obstacles while following the moving target.

[0060] In this embodiment, by acquiring obstacle detection data from the obstacle detection sensor in real time during the process of following the moving target, and when it is determined that there is an obstacle within a preset range, an obstacle avoidance lateral scale is generated based on the detection data to characterize the lateral offset direction and degree required for avoidance. Then, the embodied robot is controlled to perform obstacle avoidance following based on the following lateral scale and the obstacle avoidance lateral scale. This enables the embodied robot to autonomously perceive and avoid other objects on the path while continuously following the moving target, realizing the coordinated work of following and obstacle avoidance. This not only ensures the continuity of the following task, but also significantly improves the passage safety and autonomous adaptability of the embodied robot in complex dynamic environments.

[0061] In this embodiment, by acquiring images from a monocular camera on the android and detection data from an obstacle detection sensor, and when a moving target is detected in the image, the following lateral scale of the moving target in the lateral direction of the image is determined. If, based on the detection data, an obstacle is determined within a preset range of the android, an obstacle avoidance lateral scale with the same dimensionality as the following lateral scale is determined based on the detection data. This obstacle avoidance lateral scale characterizes the lateral offset direction and degree required for the android to avoid the obstacle. The android can be controlled to perform obstacle avoidance following based on the following and obstacle avoidance lateral scales. In the above obstacle avoidance following control process, accurate depth information and a pre-built map are not required. Therefore, obstacle avoidance following of the moving target can be accurately achieved even when the sensor cannot obtain accurate depth information of the moving target.

[0062] In some embodiments of this application, the speed of the left and right wheels of the embodied robot is controlled based on following the lateral scale in order to follow the moving target.

[0063] In some embodiments, the projection following the horizontal scale can be converted to angular velocity, and then the angle can be converted into left wheel velocity and right wheel velocity.

[0064] The formula for calculating angular velocity is as follows:

[0065] in, Angular velocity; These are the projection coefficients; To follow the horizontal scale; It is a negative sign; This is the multiplication operator.

[0066] Optionally, the projection factor can be set according to actual needs or empirical values. For example, the projection factor can be any one of 0.10 to 0.15.

[0067] It should be noted that a negative angular velocity indicates that the embodied robot is turning right; a positive angular velocity indicates that the embodied robot is turning left.

[0068] For example, with a projection factor of 0.12, when the moving target is slightly to the right, if (Indicating the offset direction is to the right and the offset degree is 5), then When the moving target is slightly to the left, if (Indicating an offset direction to the left and an offset degree of 5), then .

[0069] The formula for calculating the speed of the left wheel is as follows:

[0070] in, The speed of the left wheel; The linear velocity of the embodied robot; This refers to the distance between the left and right wheels.

[0071] The formula for calculating the speed of the right wheel is as follows:

[0072] in, This represents the speed of the right wheel.

[0073] Optionally, the linear velocity can be set according to actual needs or empirical values.

[0074] It should be understood that when the target is to the right, the angular velocity is negative, the speed of the right wheel decreases, the speed of the left wheel increases, and the robot turns to the right to move towards the target; when the target is to the left, the angular velocity is positive, the speed of the right wheel increases, the speed of the left wheel decreases, and the robot turns to the left to move towards the target.

[0075] In this embodiment, the following lateral scale reflects the direction and degree of the moving target's offset relative to the center of the image. Controlling the left and right wheel speeds of the embodied robot based on this lateral scale allows the robot to adjust its steering direction according to the moving target's offset direction in the image, and adjust the speed difference between the left and right wheels according to the degree of offset. This drives the robot to turn towards the moving target, gradually bringing the target back to the vicinity of the image's center, achieving continuous tracking. Since the above control process mainly relies on the moving target's relative offset in the lateral direction of the image, without needing to obtain the accurate depth or true lateral distance of the moving target relative to the embodied robot, it reduces the impact of monocular camera depth estimation errors on tracking control, minimizing tracking deviations and target loss risks caused by inaccurate or abrupt estimations of the true lateral distance.

[0076] In some embodiments, to avoid excessively high angular velocities in the android, the calculated angular velocities can be cropped to a preset range to obtain constrained angular velocities. Then, the speeds of the left and right wheels are calculated based on these constrained angular velocities. The preset angular velocity range is used to constrain the angular velocity range.

[0077] Optionally, a preset angular velocity range can be set according to actual needs or empirical values. For example, the preset angular velocity range is [-0.6, 0.6], and angular velocities exceeding -0.6 or 0.6 are taken from the extreme values ​​of the preset angular velocity range.

[0078] In some embodiments, during the process of the android following a moving target, the distance between the moving target and the android can be estimated using a monocular camera. It can then be determined whether this distance is less than a preset distance. If the distance is less than the preset distance and an obstacle signal is detected by the obstacle detection sensor, the following is considered successful. If the distance is greater than the preset distance or no obstacle signal is detected by the obstacle detection sensor, the following is considered unsuccessful. Optionally, the preset distance can be set according to actual needs or empirical values; alternatively, it can be calculated based on the depth information error value of the obstacle detection sensor, and must be greater than the depth information error value to avoid mistakenly identifying the moving target as an obstacle and avoiding it, thus losing the moving target. For example, if the depth information error value is 0.3m, then the preset distance can be set to 0.5m.

[0079] In this embodiment, the distance estimated by the monocular camera can be used to roughly determine the distance between the moving target and the android, while the obstacle signal detected by the obstacle detection sensor can be used to characterize the presence of an object within a short distance in front of the android. By combining the distance estimated by the monocular camera and the detection results from the obstacle detection sensor to jointly determine whether the robot has reached the target, premature stopping or continuation due to depth estimation errors when relying solely on the monocular camera for distance estimation can be avoided. Furthermore, relying solely on the obstacle detection sensor can prevent mistaking a non-moving obstacle for a moving target and stopping the robot. Therefore, the reliability of the tracking success determination can be improved, and the risk of the android colliding with or losing the moving target can be reduced.

[0080] It should be understood that this application does not limit the specific type of obstacle detection sensor. For example, the obstacle detection sensor can be at least one of the following: lidar, infrared, ultrasonic, line laser, etc.

[0081] In some embodiments of this application, determining the following horizontal scale of the moving target in the horizontal direction of the image may include steps 301 and 302.

[0082] Step 301: Determine the current horizontal scale of the moving target in the horizontal direction of the image.

[0083] The current horizontal scale is used to characterize the positional offset of the moving target in the horizontal direction of the image acquired by the monocular camera. The positional offset includes the offset direction and the offset degree.

[0084] In some embodiments, the center position of the image can be used as the reference position. The position of the moving target in the horizontal direction of the image can be determined by the target center, target bounding box center, key point center (e.g., the center of the human torso), or contour region center, or other representative target points. Based on this position, the offset direction and degree of the target representative point relative to the reference position are determined and mapped to the current horizontal scale. The current horizontal scale can be a continuous value, such as directly using the pixel difference between the target representative point and the reference position; or it can be a discrete level value, such as dividing the horizontal direction of the image into multiple scale intervals and using the interval where the target representative point is located as the current horizontal scale. The position of the target representative point in the horizontal direction of the image can be understood as the x-coordinate in the image coordinate system of the target representative point. The horizontal position can refer to the original position or coordinates in the image, such as pixel coordinates or normalized coordinates. The horizontal scale can refer to the control quantity or level based on the position mapping in the horizontal direction, used to characterize the direction and degree of offset such as leftward or rightward deviation.

[0085] It should be understood that the current lateral scale is a control position quantity determined based on the image field of view. It can be a pixel offset, a normalized offset, a discrete level value, or a continuous scale value. It reflects the degree of lateral deviation of the moving target in the camera's field of view. It does not need to represent the actual lateral distance between the moving target and the embodied robot, nor does it need to represent the true lateral coordinates of the moving target in the robot's coordinate system.

[0086] Step 302: Perform target processing on the current horizontal scale to obtain the following horizontal scale.

[0087] The target processing includes at least one of scale range constraint and time-series smoothing processing.

[0088] In this embodiment, constraining the scale range of the current lateral scale generates a controllable lateral scale to follow, ensuring that the calculated angular velocity is not too large and preventing the robot from suddenly turning sharply. Temporal smoothing of the current lateral scale generates a smoothly changing lateral scale to follow, thus preventing excessive changes in robot speed (such as sharp turns or jitter) caused by sudden changes in the speed of the moving target, achieving stable following.

[0089] In some embodiments, target processing is performed on the current horizontal scale to obtain a following horizontal scale, including: when the target processing only includes scale range constraints, the constrained horizontal scale is used as the following horizontal scale; if the current horizontal scale is within a preset horizontal scale range, the current horizontal scale can be used as the following horizontal scale.

[0090] In some embodiments, target processing is performed on the current horizontal scale to obtain a following horizontal scale, including: when the target processing only includes time smoothing processing, time smoothing processing is performed on the current horizontal scale to obtain a following horizontal scale.

[0091] In some embodiments of this application, target processing is performed on the current horizontal scale to obtain a following horizontal scale, including: When the target processing includes scale range constraints and time-series smoothing processing, the current horizontal scale is clipped to the preset horizontal scale range to obtain the constrained horizontal scale. The constrained horizontal scale is then subjected to time-series smoothing to obtain the following horizontal scale.

[0092] In this embodiment, if the current horizontal scale is not within the preset horizontal scale range, the current horizontal scale is cropped to the preset horizontal scale range; if the current horizontal scale is within the preset horizontal scale range, there is no need to crop the current horizontal scale, and the current horizontal scale is directly subjected to time-series smoothing processing to obtain the following horizontal scale.

[0093] Optionally, a preset horizontal scale range can be set according to actual needs or empirical values. For example, the preset horizontal scale range is [-10, 10]. If the current horizontal scale is greater than 10, then 10 is taken (i.e., the horizontal scale after constraint is 10); if the current horizontal scale is less than -10, then -10 is taken (i.e., the horizontal scale after constraint is -10); if the current horizontal scale is greater than or equal to -10 and less than or equal to 10, then the original value is retained.

[0094] In this embodiment, by first constraining the current lateral scale range and then performing timing smoothing, the impact of abnormal lateral scale deviations or detected jumps on the smoothing result can be reduced. This ensures that the tracking lateral scale changes smoothly within a preset lateral scale range, thereby reducing sudden changes in left and right wheel speeds and improving following stability. Compared to directly performing timing smoothing on the unconstrained current lateral scale, constraining the scale range first avoids introducing abnormal lateral scale deviations into the smoothing result, thus reducing the continuous impact of abnormal detection values ​​on the control results at subsequent time points.

[0095] In some embodiments of this application, the timing smoothing process described above may include steps 401 to 403.

[0096] Step 401: Determine the horizontal scale to be smoothed. The horizontal scale to be smoothed is the current horizontal scale, or the constrained horizontal scale obtained after constraining the scale range of the current horizontal scale.

[0097] Step 402: Subtract the previous smoothed horizontal scale from the horizontal scale to be smoothed, multiply by the smoothing coefficient, and then add it to the previous smoothed horizontal scale to obtain the current smoothed horizontal scale.

[0098] Step 403: Use the current smooth horizontal scale as the following horizontal scale.

[0099] If the target processing does not include scale range constraints, or if the target processing includes scale range constraints and the current horizontal scale is within the preset horizontal scale range, the horizontal scale to be smoothed is the current horizontal scale. If the target processing includes scale range constraints and the current horizontal scale is not within the preset horizontal scale range, the horizontal scale to be smoothed is the horizontal scale after the constraints are applied.

[0100] Taking a smoothing coefficient of 0.6 as an example, by adding 60% of the horizontal scale to be smoothed and retaining 40% of the smoothed horizontal scale from the previous moment through the above time-series smoothing process, a smoothly changing following horizontal scale can be obtained. This can prevent the robot speed from changing too much due to sudden changes in the speed of the moving target (such as sharp turns or shaking), and achieve stable following.

[0101] In some embodiments of this application, the smoothing coefficient can be dynamically determined based on at least one of the type of the moving target, the speed of the moving target, and the magnitude of the change between the horizontal scale to be smoothed and the horizontal scale at the previous moment.

[0102] The types of moving targets include, but are not limited to, people, pets, robots, vehicles, or movable objects carrying preset identifiers.

[0103] In this embodiment, a correspondence between the aforementioned states (i.e., the type of the moving target, the speed of the moving target, and at least one of the changes in the horizontal scale to be smoothed compared to the previous smoothed horizontal scale) and the smoothing coefficient can be established in advance. After obtaining each state, the smoothing coefficient can be determined from this correspondence. Optionally, the above correspondence can be set based on the product definition and capabilities of the embodied robot. For example, when the embodied robot has a large angular acceleration, high stability, and a fast sensor detection frame rate, the smoothing coefficient can be relatively small; when its capabilities do not support this, the smoothing coefficient should be large to avoid sudden jitter; it is also necessary to consider whether the focus should be on stability or tracking; the above correspondence can be set based on these characteristics.

[0104] In this embodiment, the smoothing coefficient is dynamically determined based on at least one of the following: the type of the moving target, the speed of the moving target, and the magnitude of the change between the horizontal scale to be smoothed and the horizontal scale of the previous smoothing moment. This enables the timing smoothing process to take into account both the stability and timeliness of the following under different types of moving targets, different speeds, and different degrees of motion change, thus achieving more intelligent and robust following control.

[0105] In other embodiments, the smoothing coefficient can be a preset smoothing coefficient, which is a fixed value set in advance. Optionally, the preset smoothing coefficient can be set according to actual needs or empirical values.

[0106] In some embodiments of this application, determining the obstacle avoidance lateral scale based on detection data includes: Based on the detection data, determine the obstacle point data in the robot coordinate system; Based on the distribution of obstacle point data in the lateral direction of the robot coordinate system, a lateral no-entry zone is generated; Within the preset lateral search range of the robot coordinate system, determine the passable area of ​​the target from other lateral regions besides the lateral no-entry zone; The obstacle avoidance lateral scale is determined based on the lateral coordinates of the midpoint of the target passable area.

[0107] In this embodiment, if the detection data collected by the obstacle detection sensor is located in the sensor coordinate system, the detection data can be transformed from the sensor coordinate system to the robot coordinate system based on the installation pose of the obstacle detection sensor relative to the embodied robot to obtain the detection data in the robot coordinate system. If the detection data output by the obstacle detection sensor is already located in the robot coordinate system, it can be directly used as the detection data in the robot coordinate system. Then, the detection data in the robot coordinate system is preprocessed to obtain obstacle point data. The preprocessing may include at least one of filtering (removing invalid points) and clustering (merging spatially adjacent points into the same obstacle). The sensor coordinate system refers to the coordinate system of the obstacle detection sensor.

[0108] A robot coordinate system can refer to a coordinate system established with the embodied robot itself as the reference, where the forward direction is the first coordinate axis, the horizontal direction is the second coordinate axis, and the vertical direction is the third coordinate axis. For example, the first coordinate axis can be the X-axis, the second coordinate axis can be the Y-axis, and the third coordinate axis can be the Z-axis.

[0109] A lateral restricted zone can refer to a lateral area where the android is prohibited from passing. A target passable area can refer to an optimal passable area, and a passable area can refer to a lateral area where the android is allowed to pass, that is, other lateral areas besides the lateral restricted zone.

[0110] Optionally, the preset lateral search range refers to the range used to search for passable areas in the lateral direction of the robot coordinate system. As an example, the forward direction of the robot coordinate system can be the X-axis direction, and the lateral direction can be the Y-axis direction. The preset lateral search range can be expressed as [-Ymax, Ymax], where Ymax can be determined based on at least one of the robot's body width, minimum turning radius, obstacle detection sensor detection range, or obstacle avoidance safety distance. The preset lateral search range can be set according to actual needs or empirical values.

[0111] In some embodiments, the lateral no-entry zones can be sorted in ascending order of their lateral coordinates, and overlapping or adjacent no-entry zones with a distance less than a preset threshold can be merged to obtain merged lateral no-entry zones. Then, within a preset lateral search range, the lateral intervals not covered by the merged lateral no-entry zones are determined as candidate passable areas. If there is only one candidate passable area, it can be determined as the target passable area. If there are multiple candidate passable areas, the passage cost of each candidate passable area can be determined, and the candidate passable area with the lowest passage cost can be determined as the target passable area. The passage cost can be positively correlated with the distance between the candidate passable area and the lateral centerline of the robot coordinate system, and negatively correlated with the channel width of the candidate passable area. If there is no candidate passable area with a channel width greater than or equal to a preset channel width, the candidate passable area with the largest channel width can be determined as the target passable area, or the robot can be controlled to decelerate, stop, or maintain obstacle avoidance and following state.

[0112] The midpoint of the target passable area lies outside the lateral no-entry zone, representing the center of the target passable area in the lateral direction. Determining the obstacle avoidance lateral scale based on the lateral coordinates of this midpoint allows the obstacle avoidance lateral scale to point towards the lateral center of the target passable area, thereby guiding the robot to adjust its movement in a direction that avoids the lateral no-entry zone and reducing the risk of collisions or scrapes with obstacles. Compared to positions near the boundary of the target passable area, the midpoint typically provides a more balanced passage margin in the lateral direction. Therefore, determining the obstacle avoidance lateral scale based on the lateral coordinates of the midpoint enables obstacle avoidance.

[0113] In some embodiments of this application, a lateral no-entry zone is generated based on the distribution of obstacle point data in the lateral direction of the robot coordinate system, including: Based on the lateral coordinates and preset expansion of each obstacle point in the obstacle point data, the lateral restricted area corresponding to each obstacle point is determined. Overlapping lateral restricted areas are merged to obtain lateral restricted areas.

[0114] Optionally, a preset expansion amount can be set according to actual needs or experience.

[0115] In this embodiment, for each obstacle point, the lateral coordinate value of the obstacle point can be added with a preset expansion amount to obtain the end position of the lateral restricted area corresponding to the obstacle point, and the lateral coordinate value of the obstacle point can be subtracted from the preset expansion amount to obtain the start position of the lateral restricted area corresponding to the obstacle point, thus obtaining the lateral restricted area corresponding to the obstacle point. For example, if the lateral coordinate value of an obstacle point is 10 and the preset expansion amount is 2, then the lateral restricted area corresponding to the obstacle point is [8, 12]. As another obstacle point has a lateral coordinate value of 8, then the lateral restricted area corresponding to the obstacle point is [6, 10].

[0116] Since there may be overlapping areas between the lateral restricted areas corresponding to the above obstacle points, the endpoints of each lateral restricted area can be merged to obtain a complete lateral restricted area. Taking the lateral restricted areas [8, 12] and [6, 10] corresponding to the above two obstacle points as an example, the lateral restricted area obtained after merging is [6, 12].

[0117] In this embodiment, a safety margin can be added to the obstacle by pre-setting an expansion amount, thereby ensuring that the robot maintains a sufficient safety gap with the obstacle when passing through. Merging overlapping lateral restricted areas can avoid misjudging different obstacle points of the same obstacle as multiple different obstacles, resulting in incorrect passage judgments, and provides an accurate and reliable basis for determining the restricted area in the subsequent passage.

[0118] In some embodiments of this application, determining a target passable area from lateral areas other than lateral restricted zones includes: Identify multiple candidate passable areas from other lateral areas besides the lateral restricted areas; Based on the passage costs corresponding to multiple candidate passable areas, the target passable area is determined from the multiple candidate passable areas. The passage cost is positively correlated with the distance between the candidate passable area and the lateral centerline of the robot coordinate system, and negatively correlated with the channel width of the candidate passable area.

[0119] Here, the candidate passable area refers to the remaining horizontal area within the preset horizontal search range, after excluding the horizontal prohibited area, which the embodied robot can theoretically pass through. Each candidate passable area can be defined by its left and right boundaries, and has a specific channel width (i.e., the width of the candidate passable area) and position.

[0120] Within the preset horizontal search range, the horizontal restricted area divides the entire preset horizontal search range into multiple candidate passable areas. Specifically, the areas between the starting point of the preset horizontal search range and the left boundary of the first horizontal restricted area (i.e., the horizontal restricted area closest to the starting point of the preset horizontal search range), the areas between two adjacent horizontal restricted areas, and the areas between the right boundary of the last horizontal restricted area (i.e., the horizontal restricted area closest to the ending point of the preset horizontal search range) and the ending point of the preset horizontal search range are extracted as candidate passable areas.

[0121] The passage cost is a quantitative indicator used to evaluate the merits of candidate passable areas. The smaller the value, the more suitable the corresponding candidate passable area is for the embodied robot to pass through.

[0122] In some embodiments, the distance between each candidate passable area and the lateral centerline of the robot coordinate system, and the channel width of each candidate passable area can be determined respectively; the passage cost corresponding to each candidate passable area can be calculated based on the distance and the channel width; and the candidate passable area with the minimum passage cost can be determined as the target passable area.

[0123] The formula for calculating the passage cost is as follows:

[0124] in, The cost of passage; This is the distance between the candidate passable area and the lateral centerline of the robot's coordinate system; The width of the passageway for the candidate passable area; These are preset coefficients.

[0125] Optionally, preset coefficients can be set according to actual needs or experience values.

[0126] The horizontal centerline of the robot coordinate system can be a straight line passing through the origin of the robot coordinate system and coinciding with the robot's facing direction; that is, the line with a Y-axis coordinate of 0 in the robot coordinate system, representing the direction line directly in front of the robot. The smaller the distance between the candidate passable area and the horizontal centerline of the robot coordinate system, the closer the candidate passable area is to the robot, and the less the robot needs to make large turns when avoiding obstacles.

[0127] In this embodiment, by extracting multiple candidate passable areas from the lateral search range (excluding the lateral no-entry zone), and calculating the passage cost based on the distance from each candidate passable area to the robot's lateral centerline and the channel width, the area with the lowest passage cost is selected as the target passable area. This allows for the automatic selection of a comprehensively optimal target passable area from multiple candidate passable areas. For example, areas closer to the robot's front are prioritized to reduce unnecessary turning, while areas with wider channels are prioritized to provide sufficient safety space, thereby achieving a balance between passage efficiency and comfort while ensuring obstacle avoidance safety.

[0128] In some embodiments, if there is a candidate passable area whose width is smaller than the width of the robot itself, the candidate passable area can be filtered out.

[0129] In some embodiments of this application, determining the obstacle avoidance lateral scale based on the lateral coordinate value of the midpoint of the target passable area includes: Based on a preset scale conversion coefficient, the horizontal coordinate value of the midpoint of the target passable area is converted to obtain the obstacle avoidance horizontal scale.

[0130] The preset scale conversion factor can refer to the scaling factor for converting physical distances (e.g., in meters) in the robot coordinate system into horizontal scales.

[0131] In this embodiment, the horizontal coordinate value of the midpoint of the passable area of ​​the target can be multiplied by a preset scale conversion coefficient to obtain the obstacle avoidance horizontal scale.

[0132] Optionally, a preset scale conversion factor can be set according to actual needs or experience values.

[0133] For example, the preset scale conversion factor is 20. The unit of the horizontal coordinate value of the midpoint of the target passable area is m, and the unit of the horizontal scale is scale. The preset scale conversion factor means that 1m equals 20 scales. If the horizontal coordinate value of the midpoint of the target passable area is 0.5m, then the obstacle avoidance horizontal scale obtained after conversion is 10.

[0134] In this embodiment, by using a preset scale conversion coefficient, the lateral coordinate values ​​of points in the passable area of ​​the target are converted into obstacle avoidance lateral scales under the same scale as the following lateral scale. This allows two control quantities that originally came from different sensors (camera and obstacle detection sensor) and were based on different reference systems (camera coordinate system and robot coordinate system) to control the embodied robot to perform obstacle avoidance and following under the same scale (for example, to facilitate subsequent fusion calculations), thus avoiding control anomalies caused by inconsistent scales.

[0135] In some embodiments, the obstacle avoidance lateral scale can be clipped to a preset lateral scale range so as to limit the obstacle avoidance lateral scale and the following lateral scale to the same dimension and to ensure that the subsequent contribution ratio to the passage cost is not distorted.

[0136] In some embodiments of this application, the embodied robot is controlled to perform obstacle avoidance following based on the following lateral scale and the obstacle avoidance lateral scale, including: Based on obstacle avoidance weights, the following lateral scale and obstacle avoidance lateral scale are weighted and fused to obtain the fused lateral scale. The obstacle avoidance weights are negatively correlated with the obstacle distance, which refers to the distance between the embodied robot and the obstacle. Based on the fusion of horizontal scales, the speeds of the left and right wheels of the embodied robot are controlled to enable the embodied robot to perform obstacle avoidance and following.

[0137] Obstacle avoidance weight is a dynamically changing fusion coefficient, typically ranging from [0,1], used to characterize the urgency of obstacle avoidance relative to following at the current moment. When the obstacle distance is large, the obstacle avoidance weight is small (approaching 0), and the following lateral scale dominates, with the embodied robot primarily following in the direction of the moving target; when the obstacle distance is small, the obstacle avoidance weight is large (approaching 1), and the obstacle avoidance lateral scale dominates, with the embodied robot preferentially turning in the avoidance direction.

[0138] The formula for calculating the merging of horizontal scale is as follows:

[0139] in, To integrate the horizontal scale; For obstacle avoidance horizontal scale; For obstacle avoidance weights.

[0140] In this embodiment, by introducing obstacle avoidance weights that are negatively correlated with obstacle distance, the following lateral scale and the obstacle avoidance lateral scale are weighted and fused to obtain a fused lateral scale. The speeds of the left and right wheels are then controlled based on this fused lateral scale, enabling the embodied robot to automatically adjust the priority of following and obstacle avoidance according to the distance to obstacles while following a moving target. For example, when the obstacle is far away, following is prioritized to maintain the tracking of the moving target; when the obstacle approaches, the robot smoothly switches to obstacle avoidance to prevent collisions. The transition between the two is continuous rather than abrupt, thus ensuring the continuity of the following task while achieving safe and reliable autonomous obstacle avoidance.

[0141] It should be noted that the method of controlling the speed of the left and right wheels of the embodied robot based on the fusion of the horizontal scale is the same as the method of controlling the speed of the left and right wheels of the embodied robot based on following the horizontal scale, and will not be repeated here.

[0142] In some embodiments of this application, the method further includes the following steps during obstacle avoidance and following of a moving target: If the obstacle detection sensor changes from detecting an obstacle to not detecting an obstacle, the robot will continue to perform obstacle avoidance and following based on the following lateral scale and the obstacle avoidance lateral scale determined when the obstacle was last detected within a preset time.

[0143] Optionally, a preset time can be set according to actual needs or experience.

[0144] In this embodiment, by continuing to control the robot to avoid obstacles and follow obstacles based on the following lateral scale and the last detected obstacle avoidance lateral scale for a preset time when the obstacle detection sensor changes from detecting an obstacle to not detecting an obstacle, the robot can maintain its obstacle avoidance trend for a period of time after the obstacle is lost from the obstacle detection sensor's field of vision. This allows the robot to continue moving along the original obstacle avoidance direction to completely bypass the side of the obstacle, effectively preventing side scraping caused by the robot turning too early due to the limited field of vision of the obstacle detection sensor. At the same time, since the following lateral scale is still involved in the fusion in real time during this period, the robot can gradually move towards the direction of the moving target while maintaining the obstacle avoidance trend, achieving a smooth transition from obstacle avoidance to following.

[0145] It should be noted that the obstacle detection sensor in this application may only support the detection of obstacles in front. Based on this, obstacle information will gradually be lost during obstacle avoidance, causing the robot to re-follow the moving target, resulting in side collisions with obstacles. Therefore, a freeze window setting is added, which maintains the current state for a preset time after obstacle avoidance is detected, ensuring complete obstacle avoidance.

[0146] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0147] Corresponding to the following control method of the embodied robot described in the above embodiments, Figure 5 A schematic diagram of the following control device for a unibody robot provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0148] Reference Figure 5 The device includes: The data acquisition module 501 is used to acquire images captured by the monocular camera on the embodied robot and detection data collected by the obstacle detection sensor; The follow determination module 502 is used to determine the follow horizontal scale of the moving target in the horizontal direction of the image when a moving target is detected in the image. The follow horizontal scale is used to characterize the offset direction and offset degree of the moving target relative to the center position of the image. The obstacle avoidance determination module 503 is used to determine an obstacle avoidance lateral scale based on the detection data if it is determined that there is an obstacle within a preset range of the robot based on the detection data. The obstacle avoidance lateral scale and the following lateral scale are in the same scale dimension. The obstacle avoidance lateral scale is used to characterize the lateral offset direction and degree of lateral offset required for the robot to avoid the obstacle. The obstacle avoidance control module 504 is used to control the android to perform obstacle avoidance following based on the following lateral scale and the obstacle avoidance lateral scale; The obstacle refers to any object other than the moving target.

[0149] In some embodiments, the above-described apparatus further includes: The speed control module is used to control the speed of the left and right wheels of the robot based on the following horizontal scale, so as to follow the moving target.

[0150] In some embodiments, the follow-determination module 502 is specifically used for: Determine the current horizontal scale of the moving target in the horizontal direction of the image; Perform target processing on the current horizontal scale to obtain the following horizontal scale; The target processing includes at least one of scale range constraint and time-series smoothing processing.

[0151] In some embodiments, the follow-determination module 502 is specifically used for: When the target processing includes the scale range constraint and the timing smoothing processing, the current horizontal scale is clipped to a preset horizontal scale range to obtain the constrained horizontal scale. The constrained horizontal scale is subjected to time-series smoothing to obtain the following horizontal scale.

[0152] In some embodiments, the follow-determination module 502 is specifically used for: Determine the horizontal scale to be smoothed, wherein the horizontal scale to be smoothed is the current horizontal scale, or is the constrained horizontal scale obtained by constraining the scale range of the current horizontal scale; Subtract the previous smoothed horizontal scale from the current smoothed horizontal scale, multiply by the smoothing coefficient, and then add it to the previous smoothed horizontal scale to obtain the current smoothed horizontal scale. Use the current smooth horizontal scale as the following horizontal scale.

[0153] In some embodiments, the smoothing coefficient is dynamically determined based on at least one of the type of the moving target, the speed of the moving target, and the magnitude of the change between the horizontal scale to be smoothed and the horizontal scale at the previous moment.

[0154] In some embodiments, the obstacle avoidance determination module 503 is specifically used for: Based on the detection data, the obstacle point data in the robot coordinate system is determined; Based on the distribution of the obstacle points in the lateral direction of the robot coordinate system, a lateral no-entry zone is generated; Within a preset lateral search range of the robot coordinate system, determine the target passable area from other lateral regions besides the lateral no-entry zone; The obstacle avoidance lateral scale is determined based on the lateral coordinates of the midpoint of the target passable area.

[0155] In some embodiments, the obstacle avoidance determination module 503 is specifically used for: Based on the lateral coordinates and preset expansion of each obstacle point in the obstacle point data, the lateral restricted area corresponding to each obstacle point is determined. The overlapping lateral restricted areas are merged to obtain the lateral restricted area.

[0156] In some embodiments, the obstacle avoidance determination module 503 is specifically used for: Multiple candidate passable areas are identified from other lateral areas besides the aforementioned lateral restricted areas; Based on the passage costs corresponding to the multiple candidate passable areas, the target passable area is determined from the multiple candidate passable areas. The passage cost is positively correlated with the distance between the candidate passable area and the lateral centerline of the robot coordinate system, and negatively correlated with the channel width of the candidate passable area.

[0157] In some embodiments, the obstacle avoidance determination module 503 is specifically used for: Based on a preset scale conversion coefficient, the lateral coordinate value of the midpoint of the target passable area is converted to obtain the obstacle avoidance lateral scale.

[0158] In some embodiments, the obstacle avoidance determination module 503 is specifically used for: Based on obstacle avoidance weights, the following lateral scale and the obstacle avoidance lateral scale are weighted and fused to obtain a fused lateral scale. The obstacle avoidance weights are negatively correlated with the obstacle distance, and the obstacle distance refers to the distance between the embodied robot and the obstacle. Based on the fused lateral scale, the speeds of the left and right wheels of the robot are controlled to enable the robot to perform obstacle avoidance and following.

[0159] In some embodiments, during obstacle avoidance and following of the moving target, the device further includes: The follow control module is used to control the robot to perform obstacle avoidance and follow, based on the follow lateral scale and the obstacle avoidance lateral scale determined when the obstacle was last detected, if the obstacle detection sensor changes from detecting the obstacle to not detecting the obstacle.

[0160] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0161] Figure 6 This is a schematic diagram of the structure of the embodied robot provided in an embodiment of this application. Figure 6 As shown, the embodied robot of this embodiment includes: at least one processor 60 ( Figure 6 (Only one is shown in the diagram), memory 61, and computer program 62 stored in said memory 61 and executable on said at least one processor 60, wherein when said processor 60 executes said computer program 62, it causes the embodied robot to perform the steps in any of the above method embodiments.

[0162] The embodied robot may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will understand that... Figure 6 This is merely an example of an embodied robot and does not constitute a limitation on embodied robots. It may include more or fewer parts than shown in the illustration, or a combination of certain parts, or different parts, such as input / output devices, network access devices, etc.

[0163] The processor 60 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0164] In some embodiments, the memory 61 may be an internal storage unit of the avatar robot, such as a hard drive or memory. In other embodiments, the memory 61 may be an external storage device of the avatar robot, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, the memory 61 may include both internal and external storage units of the avatar robot. The memory 61 is used to store operating systems, applications, bootloaders, data, and other programs, such as the program code of computer programs. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0165] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0166] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to the device / embodied robot, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, such as a USB flash drive, a portable hard drive, a magnetic disk, or an optical disk.

[0167] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0168] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0169] In the embodiments provided in this application, it should be understood that the disclosed devices / embodied robots and methods can be implemented in other ways. For example, the device / embodied robot embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0170] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0171] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for controlling the following of an embodied robot, characterized in that, include: Acquire images captured by a monocular camera and detection data collected by an obstacle detection sensor on the embodied robot; If a moving target is detected in the image, the following horizontal scale of the moving target in the horizontal direction of the image is determined, and the following horizontal scale is used to characterize the offset direction and degree of the moving target relative to the center position of the image; If, based on the detection data, it is determined that there is an obstacle within a preset range of the robot, then an obstacle avoidance lateral scale is determined based on the detection data. The obstacle avoidance lateral scale and the following lateral scale are on the same scale dimension. The obstacle avoidance lateral scale is used to characterize the lateral offset direction and degree required for the robot to avoid the obstacle. Based on the following lateral scale and the obstacle avoidance lateral scale, the robot is controlled to perform obstacle avoidance and following. The obstacle refers to any object other than the moving target.

2. The method according to claim 1, characterized in that, Determining the following horizontal scale of the moving target in the horizontal direction of the image includes: Determine the current horizontal scale of the moving target in the horizontal direction of the image; Perform target processing on the current horizontal scale to obtain the following horizontal scale; The target processing includes at least one of scale range constraint and time-series smoothing processing.

3. The method according to claim 2, characterized in that, The step of performing target processing on the current horizontal scale to obtain the following horizontal scale includes: When the target processing includes the scale range constraint and the timing smoothing processing, the current horizontal scale is clipped to a preset horizontal scale range to obtain the constrained horizontal scale. The constrained horizontal scale is subjected to time-series smoothing to obtain the following horizontal scale.

4. The method according to claim 2, characterized in that, The timing smoothing process includes: Determine the horizontal scale to be smoothed, wherein the horizontal scale to be smoothed is the current horizontal scale, or is the constrained horizontal scale obtained by constraining the scale range of the current horizontal scale; Subtract the previous smoothed horizontal scale from the current smoothed horizontal scale, multiply by the smoothing coefficient, and then add it to the previous smoothed horizontal scale to obtain the current smoothed horizontal scale. Use the current smooth horizontal scale as the following horizontal scale.

5. The method according to claim 4, characterized in that, The smoothing coefficient is dynamically determined based on at least one of the following: the type of the moving target, the speed of the moving target, and the magnitude of the change between the horizontal scale to be smoothed and the horizontal scale at the previous moment.

6. The method according to any one of claims 1 to 5, characterized in that, After determining the following horizontal scale of the moving target in the horizontal direction of the image, the method further includes: Based on the aforementioned horizontal scale, the speeds of the left and right wheels of the robot are controlled to follow the moving target.

7. The method according to any one of claims 1 to 5, characterized in that, Determining the obstacle avoidance lateral scale based on the detection data includes: Based on the detection data, the obstacle point data in the robot coordinate system is determined; Based on the distribution of the obstacle points in the lateral direction of the robot coordinate system, a lateral no-entry zone is generated; Within a preset lateral search range of the robot coordinate system, determine the target passable area from other lateral regions besides the lateral no-entry zone; The obstacle avoidance lateral scale is determined based on the lateral coordinates of the midpoint of the target passable area.

8. The method according to claim 7, characterized in that, The generation of a lateral no-entry zone based on the distribution of obstacle point data in the lateral direction of the robot coordinate system includes: Based on the lateral coordinates and preset expansion of each obstacle point in the obstacle point data, the lateral restricted area corresponding to each obstacle point is determined. The overlapping lateral restricted areas are merged to obtain the lateral restricted area.

9. The method according to claim 7, characterized in that, Determining the target passable area from other lateral areas besides the lateral restricted area includes: Multiple candidate passable areas are identified from other lateral areas besides the aforementioned lateral restricted areas; Based on the passage costs corresponding to the multiple candidate passable areas, the target passable area is determined from the multiple candidate passable areas. The passage cost is positively correlated with the distance between the candidate passable area and the lateral centerline of the robot coordinate system, and negatively correlated with the channel width of the candidate passable area.

10. The method according to claim 7, characterized in that, Determining the obstacle avoidance lateral scale based on the lateral coordinates of the midpoint of the target passable area includes: Based on a preset scale conversion coefficient, the lateral coordinate value of the midpoint of the target passable area is converted to obtain the obstacle avoidance lateral scale.

11. The method according to any one of claims 1 to 5, characterized in that, The control of the android to perform obstacle avoidance and following based on the following lateral scale and the obstacle avoidance lateral scale includes: Based on obstacle avoidance weights, the following lateral scale and the obstacle avoidance lateral scale are weighted and fused to obtain a fused lateral scale. The obstacle avoidance weights are negatively correlated with the obstacle distance, and the obstacle distance refers to the distance between the embodied robot and the obstacle. Based on the fused lateral scale, the speeds of the left and right wheels of the robot are controlled to enable the robot to perform obstacle avoidance and following.

12. The method according to any one of claims 1 to 5, characterized in that, During the obstacle avoidance and following of the moving target, the method further includes: If the obstacle detection sensor changes from detecting the obstacle to not detecting the obstacle, then for a preset time, the robot continues to be controlled to perform obstacle avoidance and following based on the following lateral scale and the obstacle avoidance lateral scale determined when the obstacle was last detected.

13. A hymenoid robot, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the android to perform the method as described in any one of claims 1 to 12.

14. A hymenoidae robot system, characterized in that, It includes a monocular camera and a control device, wherein the monocular camera is connected to the control device; The monocular camera is used to detect moving targets; The control device is used to perform the method as described in any one of claims 1 to 12.

Images