Obstacle avoidance and precise positioning method for intelligent robots based on face tracking

Abstract

The present application discloses an intelligent robot obstacle avoidance and precise positioning method based on face tracking. First, it proposes an obstacle avoidance method based on face tracking for an intelligent robot that improves the BUG algorithm, and proposes a model that supports small-angle obstacle avoidance. It finds the accurate switching time between face tracking behavior and BUG algorithm obstacle avoidance behavior, and summarizes the robot's obstacle avoidance method in a complex environment with multiple obstacles. Second, it proposes a method for precise visual positioning of an intelligent robot based on face recognition in response to the shortcoming that the face tracking algorithm of the intelligent robot is not accurate enough in locating the face. The image coordinate position of the face is identified by image processing, and a robot monocular visual positioning model is constructed for visual positioning in combination with the robot's hardware parameters and the person's height, which obtains a more accurate distance measurement. The method is practical and efficient.

Description

Technical Field

[0001] The present application relates to a robot obstacle avoidance and positioning method based on face tracking, and in particular to an intelligent robot obstacle avoidance and precise positioning method based on face tracking, belonging to the technical field of robot obstacle avoidance and positioning. Background Art

[0002] In the work of service robots, the target tracking function provides a way for human-computer interaction. Target tracking is based on robot vision equipment and has a wide range of applications, including visual surveillance, visual retrieval, human-computer interaction, and facial animation. The target of the tracking process is a specific person or a specific object. When this person or object moves, the robot detects the direction of the target and locates the target. Face tracking technology is divided into two processes: one is to detect the face and make the camera capture the face at all times as the face moves, which is called face following technology. The second is to locate the face to be followed and move in real time in the direction of the face. The target tracking technology is divided into: target person detection, camera tracking of the target person, and dynamic tracking of the target person according to the robot's motion state. Face following technology has important application value in robot human-computer interaction, intelligent monitoring, image face acquisition, etc.

[0003] The problem that needs to be considered in the target tracking process is that obstacles need to be taken into account in complex environments, and path planning is required for obstacle environments. A typical path planning problem is that there are obstacles in the surrounding environment, and the robot needs to plan a feasible route from the starting point to the target point based on some evaluation indicators such as the shortest walking path length, the least walking time, etc., through the surrounding environment information obtained by the sensor.

[0004] Robot following also requires more accurate positioning of the target. The purpose of following is to interact with a specific person being followed, so accurate visual positioning is also an important issue in target person following.

[0005] The NAO robot has a monocular camera that supports 1280*960 pixel high-definition images and an ultrasonic sensor that supports 0.25 to 2.55m ranging. Based on these hardware conditions, this application proposes a method to complete face tracking behavior by combining local path planning and visual positioning, providing the face movement orientation to the robot as multi-dimensional information, and combining the sonar of the intelligent robot to detect surrounding obstacles, so that the intelligent robot can autonomously avoid obstacles when tracking the face, and by capturing the face in the image and combining the person's height, it can locate the person, thereby performing a more accurate face tracking process. This method is of great research and development value and practical value for enriching the functions of human-computer interaction of intelligent robots and realizing services such as owner following and intelligent companionship.

[0006] The problems that need to be solved by the obstacle avoidance and positioning method of the existing face tracking robot and the key technical difficulties of this application include:

[0007] (1) In the process of the robot's target tracking, the robot is required to be able to detect obstacles in the surrounding environment and avoid them; be able to accurately visually locate the target, and be able to control the robot to walk in front of the target and interact with the target. However, in the actual application of the robot in the prior art, the real-time performance of the robot in target tracking and obstacle avoidance is not good, and the robot's final positioning distance for the target person is not accurate enough, which affects the effect of the robot's target interaction. The prior art lacks a method for improving bug obstacle avoidance of intelligent robots based on face tracking, and cannot enable the robot to switch between face tracking and obstacle avoidance in real time. It lacks a monocular visual positioning model based on face recognition, and cannot accurately locate the face after the person stops moving, and cannot achieve more real-time and higher-precision human-computer interaction.

[0008] (2) The existing technology lacks a good obstacle avoidance algorithm. The ultrasonic sensor of the intelligent robot can only detect obstacles within the range of 0.25 to 2.55 m in length with a 60° cone angle at the center of the intelligent chest. This is a very narrow ultrasonic detection range and does not support obstacle information feedback at every small angle. When the intelligent robot encounters an obstacle, it can only get the distance of the nearest obstacle detected by the robot's left and right sonars. This existing condition makes it difficult for the intelligent robot to apply some algorithms, such as the TangentBUG algorithm and the CautionsBUG algorithm. These algorithms require ultrasonic detection in a large range to achieve a local optimal path detection. In a complex environment with multiple obstacles, it is difficult for the intelligent robot to determine whether the obstacle is detected by ultrasonic pseudo-obstacles (obstacles detected by the robot but do not affect the robot's walking) and whether the obstacle is multiple or a whole. The visual structure of the intelligent robot is two vertically distributed cameras, which are located at the mouth and forehead of the intelligent robot respectively. The horizontal detection range of the two cameras is the same. Therefore, if a multi-information fusion obstacle avoidance strategy is adopted and the corner edge of obstacles is determined by combining visual images, it will affect the real-time switching of the camera for face following and obstacle avoidance. In an unknown environment, in the local path planning when the target is not moving, the robot perceives external information and analyzes a local shortest path so that the robot can reach the destination faster. However, in the face following process, the face is a dynamic target, and this local shortest path determination is not valid in some cases. In order to obtain the local shortest path, the robot often needs to combine visual images or more accurate large-angle sonar to feedback obstacle information at each angle. According to the hardware conditions of the intelligent robot, it cannot be supported.

[0009] (3) The existing technology lacks considerations for accurate visual positioning. The visual positioning part of intelligent face tracking is based on the size of the face. When tracking, it is necessary to set a critical value facewidth for the size of the face in the visual image. As the distance to the person gradually decreases, the robot stops when the size of the face in the visual image reaches this critical value. The problem with this method is poor positioning accuracy. The size of real faces varies from person to person, and the change in perspective of the robot after getting closer to the person will also affect the pixel changes of the face in the visual image. The sound source positioning is easily affected by the external environment, especially when the indoor reverberation is severe, the positioning effect is not good. In the process of human-computer interaction, the surrounding environment of service robots is often complex, and the service that needs to be provided requires high accuracy. Therefore, the visual positioning of face tracking needs to have higher accuracy than the simple method of setting the face width to measure the distance. Summary of the invention

[0010] This application aims at the problems existing in the target following behavior of service robots, takes intelligent robots as robot models, and proposes a method for service robots to deal with the face following process. First, in view of the process in which the intelligent robot performs face following behavior in an unknown environment, performs obstacle avoidance behavior when encountering obstacles, and returns to the face following behavior after leaving the obstacles, an obstacle avoidance method of an intelligent robot based on face following and an improved BUG algorithm is proposed. A model supporting small-angle obstacle avoidance is proposed for the intelligent robot, which is a small-angle sonar detection robot. The accurate switching time between face following behavior and BUG algorithm obstacle avoidance behavior is found, and the robot's obstacle avoidance method in a complex environment with multiple obstacles is summarized. The algorithm's own parameters are adjusted and tested, and good results are obtained. Second, in view of the shortcomings of the intelligent robot's face following algorithm in locating faces, a method for precise visual positioning of intelligent robots based on face recognition is proposed. According to the image coordinate position of the face identified by image processing, and combined with the robot hardware parameters and the height of the person, a model of the robot's monocular visual positioning is constructed for visual positioning. In the experimental test, it was compared with the face positioning algorithm in the face tracking module of the intelligent robot, and a more accurate distance measurement was obtained. The method is practical and efficient.

[0011] In order to achieve the above technical effects, the technical solutions adopted in this application are as follows:

[0012] The intelligent robot obstacle avoidance and precise positioning method based on face tracking is: first, based on the process of the robot following faces in an unknown environment, avoiding obstacles when encountering obstacles, and returning to face following behavior after leaving the obstacles, an obstacle avoidance method of the intelligent robot based on face tracking and improved BUG algorithm is constructed, a model that supports small-angle obstacle avoidance is proposed for the small-angle sonar detection robot, the accurate switching time between face following behavior and BUG algorithm obstacle avoidance behavior is found, and the robot's obstacle avoidance methods in a complex environment with multiple obstacles are summarized and processed, and the algorithm's own parameters are adjusted and tested; second, a method for precise visual positioning of an intelligent robot based on face recognition is constructed, the image coordinate position of the face is identified according to image processing, and a robot monocular visual positioning model is constructed for visual positioning in combination with the robot's hardware parameters and the person's height;

[0013] The target-following behavior of the intelligent robot is divided into two stages:

[0014] Phase 1: During the roughly positioned face tracking process, when encountering an obstacle, the intelligent robot performs obstacle avoidance behavior using the improved BUG algorithm: the robot moves by detecting the face and following the movement of the face. The movement process is implemented through the ALFaceTracker method of the robot's ALTrackerProxy module. The face that appears in the image is detected and followed, and the face is located according to the set face mean width faceWidth. After obtaining this position, the robot is controlled to move in front of the person. The robot builds a spatial model and uses its own displacement to measure the person's height. When the robot encounters an obstacle blocking the robot's progress while performing face tracking, it switches from face tracking behavior to improved BUG algorithm obstacle avoidance behavior. When the robot determines that there are no obstacles on the route in the direction of the face, it switches from obstacle avoidance behavior to face tracking behavior until the target stops moving. When the robot's rough positioning is completed and there are no obstacles within the sonar detection range, Phase 1 ends.

[0015] In the second stage, after completing the rough positioning of the face following behavior, the robot accurately locates the face and walks to a position at a specified distance from the person: the robot uses the position of the face in the image coordinate system, the height of the person obtained in the previous stage, and the calibrated camera parameters as calculation parameters, builds a geometric model for calculation, obtains a more accurate position of the face in the world coordinate system, and walks to a position 10 cm in front of the person.

[0016] Preferably, the framework of the BUG algorithm is improved: factors considered when the BUG algorithm is improved to optimize the detour walking strategy include: the humanoid walking mode of the intelligent robot, the way the sonar detects obstacles, and the dynamic walking direction of the target, and how to determine the starting and ending points of obstacle detection and the specific detour strategy are improved;

[0017] The robot's circumnavigation process is divided into two modes: straight-line walking and turning. The turning behavior is further divided into two modes: circumnavigation turning and departure turning from the perspective of turning properties.

[0018] 1) Detour and turn: When the intelligent robot encounters an obstacle in the obstacle avoidance detection area, it needs to turn around the obstacle;

[0019] 2) Leaving and turning: After the intelligent robot determines that it has reached the departure point, it has left the influence range of the obstacle and switches back to the face following mode to perform the turning process required for face following;

[0020] By turning and walking in a straight line, the overall walking route map is presented in the form of multiple broken lines, and the walking behavior of each broken line can be determined to be safe;

[0021] The robot triggers obstacle avoidance behavior in three possible situations during face following motion, as follows:

[0022] a) An obstacle appears in the obstacle avoidance detection area on the left side of the sonar;

[0023] b) An obstacle appears in the obstacle avoidance detection area on the right side of the sonar;

[0024] c) Obstacles appear in both the left and right obstacle avoidance detection areas of the sonar;

[0025] According to the triggering conditions of three different situations, the following two-stage obstacle avoidance strategy is implemented to deal with the obstacle avoidance situation accompanied by face tracking in an environment with multiple obstacles: 1) turning around obstacles, and 2) turning around the edge of obstacles.

[0026] Preferably, an improved algorithm for detour and steering when encountering obstacles: when the intelligent robot detects an obstacle in the obstacle avoidance detection area in the direction of travel, it stops following the human face and determines whether the distance between the robot and the obstacle is greater than or equal to the steering distance. When the robot is farther away from the obstacle than the steering distance, the difference between the distance measured by the forward sonar and the steering distance is calculated; when the robot is farther away from the obstacle than the steering distance, the difference between the backward steering distance and the distance measured by the sonar is calculated to ensure a safe steering distance ahead. The real-time position of the intelligent robot after forward or backward correction is then marked as the obstacle point. The real-time location of the point and the robot's forward direction are recorded through the motion.getlocation method. At this time, the location of the obstacle is determined, and different detour and steering strategies are selected according to the different locations of the sonar obstacles on both sides.

[0027] At this time, different detour strategies are selected according to the direction of the obstacle encountered by the robot. When obstacles appear on both sides of the robot, the obstacle avoidance detection areas on the left and right sides of the robot detect obstacles. At this time, it is no longer possible to quickly detour the obstacles by simply avoiding obstacles on one side. The movement direction in the same direction as the target to be followed, that is, the movement direction of the face, is selected as the obstacle avoidance direction to reduce the overall time cost.

[0028] When the robot encounters obstacles on both sides, it turns according to the obstacle avoidance turning strategy on both sides, and the obstacle avoidance problem on both sides will be transformed into obstacle avoidance on one side. When the robot encounters obstacles on one side and turns to avoid them, the sonar on the other side still needs to detect whether there are new obstacles at any time. If a new obstacle is detected in the other obstacle-free sonar, obstacles will appear in the obstacle avoidance detection areas on both sides, and the problem will be transformed into obstacle avoidance behavior on both sides, and the face direction will be preferred for obstacle avoidance.

[0029] When an obstacle appears on one side of the intelligent robot, the robot chooses the direction away from the obstacle as the detour direction, and each time it turns at an angle of β, it obtains the sonar parameters after each turn. When the obstacle is a convex obstacle, after each turn, the robot uses the obtained sonar parameters to correct the position of the robot and the obstacle by moving forward or backward to ensure that a turning distance is met in front, so as to ensure the safe turning of the intelligent robot.

[0030] When the robot detects an obstacle only on one side of the sonar, the sonar detection on the other side of the robot indicates that there is no obstacle in the obstacle avoidance detection area on the other side. The robot quickly avoids the obstacle by turning at a small angle, adopting a steering strategy in the direction of avoiding the obstacle, while ignoring the direction of movement of the target.

[0031] The steering angle is determined on one side of the steering direction. At this time, the obstacle position is determined by the steering angle. If the steering angle is too large, the robot is facing a multi-obstacle or maze-like obstacle environment. When the cumulative steering angle is greater than 360°, it indicates that there are obstacles all around the robot, and the face tracking target cannot be reached.

[0032] Preferably, obstacle avoidance strategy in the case of multiple obstacles: When the robot avoids obstacles in a multi-obstacle environment, different solutions are adopted for the following different situations:

[0033] 1- When the robot encounters an obstacle and turns around, it encounters a new obstacle. When the robot turns, in a multi-obstacle environment, if there is no obstacle and the sonar on one side or both sides detects a new obstacle, the robot will continue to turn in the direction selected by the previous obstacle end until:

[0034] 1) If the turning angle exceeds 360 degrees, the robot will not be able to reach the target;

[0035] 2) Turn to the point where obstacles appear on one side of the robot's sonar and there are no obstacles on the other side.

[0036] Perform a unilateral detour again;

[0037] 2-When the robot is bypassing an obstacle at the edge, a new obstacle appears. When bypassing an obstacle on the left side, a new obstacle appears on the left or right side of the robot's sonar. In this case, the obstacles may be multiple obstacles or convex polygonal obstacles. At this time, another unilateral turn will be performed to bypass the obstacle. When the robot is bypassing the edge of the obstacle, when a new obstacle appears on the other end without an obstacle, the robot bypasses a section close to the obstacle. After the obstacle appears on the other end, the distance between the two obstacles is less than the width of the robot.

[0038] Preferably, the architecture of precise positioning: when the face tracking module performs rough positioning in the first stage, the estimated value of the face size threshold faceWidth is fixedly set to 0.1m, which is smaller than the real value of the universal face. In this way, the robot will be between 30 and 40cm away from the person. Then, the face is accurately visually positioned, and the robot moves forward to the precise position Xcm in front of the target person for human-computer interaction, so as to make up for the defects of rough positioning;

[0039] The precise positioning method is to use the monocular vision positioning architecture and face recognition image processing method after the rough positioning in the previous stage. The architecture is as follows:

[0040] Through the relationship between the robot's camera, the robot's image coordinate system, and the real-world coordinate system, a geometric model is constructed to obtain the known parameters that can be used in the model to solve the position of the target from the camera and obtain the precise position of the target from the intelligent robot.

[0041] The method of obtaining pixel coordinates from the robot image coordinate system is to take a picture through the robot's local camera, transmit it remotely to a computer for image processing, detect the face in the image through the Harr to Like feature and return the coordinates;

[0042] After obtaining the precise position of the target person, the robot is controlled to move to a distance X cm in front of the target person.

[0043] Preferably, the image coordinate system, camera coordinate system and world coordinate system of the intelligent robot are:

[0044] 1) Image coordinate system: Take the upper left corner of the image as the origin and pixel (px) as the unit to establish the coordinate system uv. The horizontal and vertical coordinates of the pixel are the column and row numbers in the image array respectively.

[0045] Assume that the entire image size is an intelligent robot camera image of 20*20 pixels. In the uv coordinate system with the upper left corner of the image as the origin, the coordinates of the origin u are (0, 0), and the coordinates of the image center point O are (10, 10);

[0046] If it is converted into a coordinate system with O as the origin relative to the center of the image, the conversion relationship is shown in Formula 1. At this time, assuming that (u0, v0) is the coordinate of O1 in the uv coordinate system, dx, dy is the physical size of a pixel in the x and y axis directions, then the physical coordinates of any pixel P (x, y) in the O|xy coordinate system are:

[0047]

[0048] The coordinate system of the image thus obtained is O 1xy , where Q1 is the coordinate of the center point of the face, P1 is the coordinate of Q1 on the y-axis vector, and U (u, v) represents the coordinate system of the image pixels;

[0049] 2) Camera coordinate system: Point O is the origin of the camera coordinate system, where X c Axis and Y c The axes are represented as the two dimensions of the camera coordinate system, which are parallel to the x-axis and y-axis of the image coordinate system, where x c Axis and world coordinate system X w Axis parallel, Y c Axis and world coordinate system Y w The angle between the Z and Z axes is the vertical rotation angle of the robot head. c It coincides with the z-axis of the image coordinate system and is perpendicular to the two planes;

[0050] The focal length of the camera is f OO1, and the horizontal focal length of the intelligent robot is f x With vertical focal length f y The same, then we can deduce:

[0051]

[0052] 3) World coordinate system: The robot's world coordinate system O w X w Y w Z w The coordinate system is established at the bottom of the robot's feet. Point O is the position of the robot's head camera. 001 is the focal length of the camera, which is f, and the angle with the Y axis is α. The imaging position of any point P in space on the image is approximated by the pinhole model. M1 is the internal parameter of the camera, which is completely determined by f. x 、f y, u0, and v0, which are only related to the internal structure of the camera. M2 is the external parameter of the camera, which is completely determined by the orientation of the camera relative to the world coordinate system. R is a 3*3 orthogonal unit matrix. t is a three-dimensional translation vector, and the vector O=(0, 0, 0).

[0053] Preferably, the model based on the monocular vision of the intelligent robot: the robot stands vertically in its world coordinate system 0 w Right above the point, 0 is the position of its camera. The intelligent robot faces the negative half axis direction of the Y axis, with its head raised. At this time, a human face appears obliquely to the right of the robot and is captured by the camera. The pixel imaging coordinates appearing on the camera are Q1(u1, v1). NQ' represents the person. The height of NQ' at the center of the face is H. The height of the robot camera is 00. w is h, the angle of the head when it is raised is the angle between the optical axis OL and the Y axis, which is α, obtained by the robot's head motor parameters. The focal length of the camera is obtained by calibrating the camera, and Q' is the center point of the face obtained after the image processing target is recognized;

[0054] Assuming NO w The extended line of the line segment and the extended line of point Q'0 intersect at point Q(X w , Y w ), triangle NQQ' and 0 w Similar to Q0, the coordinates of point Q1 (the center of the face in the image coordinate system) are (x, y), which are converted from the coordinates of Q1 in the uv coordinate system (u1, v1) by formula 1. γ represents ∠OO1P1, and β is the angle between the extension line of OP1 and the Y axis;

[0055] Find O w N, that is, the distance from the robot to the human foot: get the angle relationship between a, β, and γ. When the mapping Q1(x, y) of the midpoint Q'(x, y) of the top of the forehead of the human face appears in the third and fourth quadrants of the image coordinates, the angle relationship at this time is:

[0056] β=α+γ Formula 3

[0057] When the mapping Q(x, y) of the midpoint Q'(x, y) of the top of the forehead of the face appears in the first and second quadrants of the image coordinates, the angle relationship at this time is:

[0058] β=α-γ Formula 4

[0059] First find the horizontal coordinate X of point P through the constructed triangle w Length:

[0060]

[0061] y is calculated by formula 1, and the horizontal coordinate of point P is obtained as formula 7, f x =f / d x , f y =f / d y , represent the equivalent focal length expressed in pixels in the X-axis and Y-axis directions respectively;

[0062]

[0063] The coordinates of point Q on the horizontal plane are (X w , Y w ), where Y w =PQ, PQ is calculated according to the similarity theorem to obtain formula 8, OP and OP1 are calculated according to formula 9 and formula 10, and the ordinate Y of point Q is calculated according to formula 8 w As shown in formula 12:

[0064]

[0065] P1Q1=(u-u0)dx Formula 11

[0066]

[0067] Get Q(X w , Y w ) and then, through the two constructed triangles NQQ' and 0 w Q0 is similar, find O w The length of N, according to the triangle similarity theorem, is:

[0068]

[0069] Get the distance between the intelligent robot and the human foot.

[0070] Preferably, the method for obtaining the height of a person is to obtain the height of a person by combining the information extracted by the robot through vision and the information obtained by the outside world through movement and walking:

[0071] The height of a person is measured by the robot. When the robot and the person are in a straight line, the face is in the center of the visual image after two face trackings. At this time, the robot will obtain the information of the head angle respectively. The robot performs face tracking activities. The height of the robot is h. When the robot sees the face, the current angle of the robot's head is recorded as ∠2. At this time, the robot moves forward a distance S and captures the face again. The angle of the robot's head is ∠1. The height of the person H is expressed as:

[0072]

[0073] The behavior of obtaining the height of a person here is obtained during the process of the robot's face tracking and obstacle avoidance. This process is repeated many times during the obstacle avoidance and face tracking process, and the height of the person is obtained multiple times, and finally the average value is taken to participate in the final visual positioning operation.

[0074] Preferably, the specific implementation of face recognition is:

[0075] 1) Detect whether a face appears: Use the FaceDetected subscription event in the robot's ALFaceDetection;

[0076] 2) Remotely obtain facial images: After detecting a face, the robot stops turning its head up and down, and keeps its head at the position where the face appears in the robot's visual image. It uses the getImageRemote method in ALVideoDevice to capture a latest image from the camera's video source and convert it into an image that conforms to the set format to obtain images of different resolutions and color standards. Finally, this group of images is loaded into an array container containing a binary image and the image's width, height, and color space information in the form of ALValue, and transmitted to the remote computer through the network;

[0077] 3) Computer local image processing: The acquired image is processed on the remote computer using the image tool of Opencv2.1.0, and the scientific computing numpy is used to reorganize the array rows and columns of the acquired image, and finally a 480*640 array format image is obtained.

[0078] Preferably, the visual positioning model and algorithm of the intelligent robot based on face recognition: by obtaining the height of the person and the image of the face, a more accurate distance between the person and the intelligent robot is calculated;

[0079] The validity of the rough face positioning result is ensured by setting the estimated value of faceWidth to a critical value smaller than the standard face size. The second stage of this application: precise visual positioning of intelligent robots based on face recognition starts from the end of rough face positioning. The target person is not moving. The following are the specific processes and judgment criteria:

[0080] 1) When the intelligent robot performs face tracking, each time it switches from obstacle avoidance to face tracking, it roughly locates the face and walks to the target person;

[0081] a. If the target person is not moving, the robot will stop about 30 to 40 cm in front of the target person; at this stage, the robot will perform precise visual positioning;

[0082] b. If the target person continues to move, the robot will continue to follow the person's face and avoid obstacles;

[0083] 2) Adjust the direction of the robot's body to be consistent with the head so that the robot can recognize the face acquired by the remote image;

[0084] 3) After the recognition is completed, the height of the person obtained by the previous robot movement is combined with the visual positioning model to calculate the direction and distance of the face;

[0085] 4) If there are no obstacles detected within the sonar detection range, as the robot moves, the sonar detection distance is shortened to ensure that people are not identified as obstacles by the robot;

[0086] 5) If an obstacle appears within the sonar detection range, the robot will continue to follow the face and avoid the obstacle;

[0087] 6) Finally, the vehicle moves to a position about X cm in front of the target person, and the accurate face positioning process is completed, preparing for the next face tracking and obstacle avoidance behavior.

[0088] Compared with the prior art, the innovations and advantages of this application are:

[0089] (1) This application aims at the problems existing in the target following behavior of service robots, takes an intelligent robot as a robot model, and proposes a method for a service robot to deal with the face following process. First, in view of the process in which the intelligent robot performs face following behavior in an unknown environment, performs obstacle avoidance behavior when encountering obstacles, and returns to the face following behavior after leaving the obstacles, an obstacle avoidance method of an intelligent robot based on face following and an improved BUG algorithm is proposed. A model that supports small-angle obstacle avoidance is proposed for the intelligent robot, which is a small-angle sonar detection robot. The accurate switching time between face following behavior and BUG algorithm obstacle avoidance behavior is found, and the robot's obstacle avoidance method in a complex environment with multiple obstacles is summarized. The algorithm's own parameters are adjusted and tested, and good results are obtained. Second, in view of the shortcomings of the intelligent robot's face following algorithm in not being accurate enough in locating faces, a method for accurate visual positioning of an intelligent robot based on face recognition is proposed. According to the image coordinate position of the face identified by image processing, and combined with the robot's hardware parameters and the height of the person, a robot monocular visual positioning model is constructed for visual positioning. In the experimental test, it was compared with the face positioning algorithm in the face tracking module of the intelligent robot, and a more accurate distance measurement was obtained. The method is practical and efficient.

[0090] (2) The present application creatively proposes an obstacle avoidance method based on an improved BUG algorithm for face tracking. The original BUG algorithm is a reactive algorithm that cannot obtain the local shortest path, but when applied to face tracking and dynamic target obstacle avoidance, this algorithm has good real-time performance. Based on the sonar of the intelligent robot, the present application specifically defines the walking path, collision area, minimum detection range and other related concepts required for the intelligent robot to walk, so that the intelligent robot can more accurately switch between face tracking behavior and obstacle avoidance behavior; the minimum detection range of the intelligent robot is used to ensure that the intelligent robot can pass through narrow paths; at the same time, a new obstacle avoidance strategy is derived by analyzing the environment with multiple obstacles.

[0091] (3) This application creatively proposes a method of visual positioning based on face recognition. The robot autonomously measures the height of the target and remotely obtains the visual image of the face through the intelligent robot, extracts the face of the intelligent robot in the visual image, and obtains the coordinates of the top of the head. A spatial model of the robot image coordinate system and the world coordinate system is established, and an equation is constructed based on the coordinates of the face in the visual image, the height of the person, and the internal parameters of the camera, so as to calculate the more accurate distance between the intelligent robot and the person. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Figure 1 It is an obstacle avoidance flow chart based on the improved BUG algorithm for intelligent robot face following.

[0093] Figure 2 This is a schematic diagram of the robot's detour when the sonars on both sides detect obstacles.

[0094] Figure 3 This is a schematic diagram of the robot's detour when the sonar on the right side detects an obstacle.

[0095] Figure 4 This is a schematic diagram of the robot turning around on the left and encountering a new obstacle.

[0096] Figure 5 This is a schematic diagram of the structure of the two cameras on the head of the intelligent robot.

[0097] Figure 6 It is a schematic diagram of the structure of the camera coordinate system of the intelligent robot.

[0098] Figure 7 This is a positioning model diagram of the robot's monocular vision space point for obstacle avoidance.

[0099] Figure 8 This is a graph showing the relationship between the number of times the robot turns a corner and the time it takes.

[0100] Fig. 9 This is a comparison chart of the experimental results of the obstacle avoidance algorithm based on face recognition and other algorithms.

[0101] Fig.10 This is a comparison chart of experimental results of precise positioning of intelligent robots based on face recognition. DETAILED DESCRIPTION

[0102] The following, in conjunction with the accompanying drawings, further describes the technical solution of the intelligent robot obstacle avoidance and precise positioning method based on face tracking provided by the present application, so that technical personnel in the field can better understand the present application and implement it.

[0103] From early industrial robots to today's intelligent robots that are used in all areas of social application, robots have moved from novels and movies to people's lives. Service robots are a hot topic in the development of the robotics industry today, and target tracking of service robots is a core issue and difficulty that service robots need to study. In the process of target tracking, the robot is required to be able to detect obstacles in the surrounding environment and avoid them; to be able to accurately visually locate the target, and to control the robot to walk in front of the target and interact with it. However, in actual robot applications, the real-time performance of the robot in target tracking and obstacle avoidance is not good, and the robot's final positioning distance for the target person is not accurate enough, which affects the effect of the robot's target interaction.

[0104] In response to the above problems, this application proposes a method for improving bug obstacle avoidance of intelligent robots based on face tracking. This method proposes an improved bug algorithm obstacle avoidance method in the face tracking scenario, which enables the robot to switch between face tracking and obstacle avoidance in real time, and proposes a monocular vision positioning model based on face recognition, which can accurately locate the face when the person stops moving, thus achieving more real-time and higher-precision human-computer interaction.

[0105] Experiments on intelligent robots have shown that the face tracking method proposed in this application is effective. Compared with the positioning methods of the other three local path planning algorithms and the face tracking module built into the intelligent robot, the method proposed in this application enhances the real-time performance of the robot's obstacle avoidance and improves the accuracy of face positioning.

[0106] 1. Intelligent robot obstacle avoidance method based on face tracking

[0107] 1. Overview of the intelligent robot face tracking module

[0108] Intelligent robot face tracking is achieved by calling the ALTracker module, which allows the robot to follow different targets, such as a red ball, a face, and a landmark. The following motion is achieved in different ways, such as the robot head following the object, turning in place to follow the object, and dynamically following the target. This module coordinates with the robot's motion module through the robot's visual detection module to ensure that the target is always at the center of the robot's camera coordinates.

[0109] In the rough positioning stage, a specific person in ALTracker is used as the target, and FaceDetected in Event is used as the trigger event. When one or more faces appear in the robot's visual image, the event is triggered and a set of facial feature values ​​is returned.

[0110] By saving the characteristic face of a specific person, a specific face is selected for tracking. If the target is lost, the face that appears is re-detected to see if it is the target person's face.

[0111] After storing the feature tuple of a specific face, the FaceTracker() method in ALTracker is used to dynamically track the face.

[0112] The robot adjusts the parameters of the head motor to make the face appear in the center of the robot's visual image based on the position of the face in the visual image, obtains the angular position of the face and the estimated value of the actual width of the face through faceWidth, calculates the actual distance between the face and the robot, and cooperates with the ALMotion module to enable the robot to complete the face tracking behavior.

[0113] The intelligent robot visual tracking module is used to complete the rough positioning. The advantage is that the face tracking is completed by calling the robot's local module, which has good real-time performance. It is much faster than the method that needs to detect the face in the image on the remote computer and return the value to the robot. The disadvantage is that since faceWidth is an estimated value set for the width of the real face, the positioning result is not accurate, and the face needs to be accurately positioned visually later.

[0114] (II) Improved BUG algorithm based on intelligent robot face tracking

[0115] According to the sonar parameters and body shape of the intelligent robot, an improved BUG algorithm is proposed for avoiding obstacles in the surrounding environment that affect the intelligent robot's following path during the target following process.

[0116] 1. Improve the framework of BUG algorithm

[0117] According to the traditional BUG algorithm, the robot's obstacle avoidance behavior in an unknown environment is divided into two modes, namely, walking in a straight line close to the target when there are no obstacles and walking around to avoid obstacles when obstacles are detected. The bypassing behavior is a relatively complex way of walking. The factors considered when improving the BUG algorithm to optimize the bypassing walking strategy include: the humanoid walking method of the intelligent robot, the way the sonar detects obstacles, and the dynamic walking direction of following the target. Improvements are made to how to determine the starting and ending points of obstacle detection and the specific strategies for bypassing.

[0118] The robot's circumnavigation process is divided into two modes: straight-line walking and turning. The turning behavior is further divided into two modes: circumnavigation turning and departure turning from the perspective of turning properties.

[0119] 1) Detour and turn: When the intelligent robot encounters an obstacle in the obstacle avoidance detection area, it needs to turn around the obstacle;

[0120] 2) Leaving and turning: After the intelligent robot determines that it has reached the departure point, it has left the influence range of the obstacle and switches back to the face following mode to perform the turning process required for face following;

[0121] Since the intelligent robot is a humanoid robot, its sonar angle cannot achieve 360° all-round detection, so it is easy to collide with the blind spot in the collision area when walking diagonally or turning while walking. By turning and walking in a straight line, the overall walking route map is presented in the form of multiple broken lines, and the walking behavior of each broken line can be determined to be safe.

[0122] The robot triggers obstacle avoidance behavior in three possible situations during face following motion, as follows:

[0123] a) An obstacle appears in the obstacle avoidance detection area on the left side of the sonar;

[0124] b) An obstacle appears in the obstacle avoidance detection area on the right side of the sonar;

[0125] c) Obstacles appear in both the left and right obstacle avoidance detection areas of the sonar;

[0126] According to the trigger conditions of three different situations, the following two-stage obstacle avoidance strategy is carried out to deal with the obstacle avoidance accompanied by face tracking in an environment with multiple obstacles: 1) Steering behavior when encountering obstacles, 2) Steering behavior when avoiding obstacles. The specific process is as follows: Figure 1 shown.

[0127] This classification of behavior-based obstacle avoidance algorithms can better solve the problem of obstacle avoidance in complex environments, whether there are multiple obstacles or maze terrain, by detecting the direction of obstacles based on sonar.

[0128] 2. Improved algorithm for steering around obstacles

[0129] When the intelligent robot detects an obstacle in the obstacle avoidance detection area in the direction of travel, it stops following the human face and determines whether the distance between the robot and the obstacle is greater than or equal to the turning distance. When the robot is farther away from the obstacle than the turning distance, the difference between the distance measured by the forward sonar and the turning distance is used; when the robot is farther away from the obstacle than the turning distance, the difference between the backward turning distance and the distance measured by the sonar is used to ensure that there is a safe turning distance ahead. Then, the real-time location of the intelligent robot after forward or backward correction is marked as the obstacle point, and the real-time location of the point and the robot's forward direction are recorded through the motion.getlocation method. At this time, the location of the obstacle is determined, and different detour strategies are selected according to the different locations of the sonar obstacles on both sides.

[0130] At this time, different detour strategies are selected according to the direction of the obstacle encountered by the robot. When obstacles appear on both sides of the robot, the process is as follows: Figure 2 , the obstacle avoidance detection areas on both the left and right sides of the robot detect obstacles. At this time, it is no longer possible to quickly bypass the obstacles by simply avoiding obstacles on one side.

[0131] Considering that the target being followed is a dynamic target and may continue to move in one direction, if the obstacles on the left and right sides are avoided first, the obstacle on the side closer to the target may be in the opposite direction of the target's movement, and it will eventually take a longer distance to catch up with the target, thus increasing the time required to start the course. Therefore, this application selects the same direction of movement as the target being followed, that is, the direction of face movement, as the obstacle avoidance direction to reduce the overall time cost.

[0132] When the robot encounters obstacles on both sides, it will turn according to the strategy for bypassing obstacles on both sides, and the problem of bypassing obstacles on both sides will be transformed into bypassing obstacles on one side. When the robot encounters obstacles on one side and bypasses them, the sonar on the other side must still detect whether there are new obstacles at any time. If a new obstacle is detected in the other obstacle-free sonar, obstacles will appear in the obstacle avoidance detection areas on both sides, and the problem will be transformed into bypassing obstacles on both sides, and the face direction will be preferred for obstacle avoidance.

[0133] When an obstacle appears on one side of the intelligent robot, the process is as follows Figure 3 The robot chooses the direction away from the unilateral obstacle as the detour direction, and each time it turns at an angle of β, it obtains the sonar parameters after each turn. When the obstacle is a convex obstacle, after each turn, the robot uses the obtained sonar parameters to correct the position of the robot and the obstacle by moving forward or backward to ensure that a turning distance is met in front, so as to ensure the safe turning of the intelligent robot.

[0134] When only one side of the robot's sonar detects an obstacle, the sonar detection on the other side of the robot indicates that there is no obstacle in the obstacle avoidance detection area on the other side. The robot quickly avoids the obstacle by turning at a relatively small angle, adopting a steering strategy in the direction of avoiding the obstacle, while ignoring the movement direction of the target.

[0135] Regardless of whether it is a unilateral obstacle-circumventing steering behavior or a circumventing steering behavior with obstacles on both sides, they all have the same point in that in the process of selecting the steering direction, it is ensured that there are no obstacles in the obstacle avoidance detection areas on both sides before the end. Therefore, the steering angle is determined on one side of the steering direction, and the obstacle position is determined by the steering angle. If the steering angle is too large, the robot is facing a multi-obstacle or maze-like obstacle environment. When the cumulative steering angle is greater than 360°, it indicates that there are obstacles all around the robot, and the face tracking target cannot be reached.

[0136] 3. Behavior of bypassing obstacle edges

[0137] During the unilateral detour, the robot walks in segments, and the length of each segment is equal to the detection distance. When walking, the sonar data on both sides are first collected to ensure: 1) During the detour, it always walks on a predictable path without obstacles in front; 2) Always detect whether there are new unknown obstacles in front.

[0138] Each walking section is planned based on the data on one side of the obstacle: in the process of straight-line walking, the y-axis is corrected on the basis of ensuring that the walking direction remains unchanged to maintain a controllable distance from the obstacle. This distance is the distance between the obstacle avoidance detection area and the buffer area detected by the sonar; when the intelligent detour reaches the edge of the obstacle, the sonar acquisition parameters will exceed the buffer area, and the detour is determined to be over.

[0139] 4. Select the robot's disengagement point

[0140] Since the intelligent robot needs to perform simultaneous obstacle avoidance and face tracking during face tracking, the breakaway point defined by the traditional BUG algorithm needs to be redefined specifically. The purpose is to ensure that the robot can accurately find the breakaway point and leave in time when the destination changes at any time.

[0141] Therefore, when analyzing the problem specifically, we set the situation to a one-sided obstacle environment. During the process of intelligent one-sided detour, we face two problems:

[0142] First, when the unilateral detour ends, the sonar data will have a nonlinear mutation, which means the unilateral detour ends. At this time, the robot head performs face following activities to obtain the relative position of the robot, and performs a face-finding corner search. After the corner, if there are no obstacles, the point is defined as the departure point. Perform face following activities and prepare for the next obstacle avoidance;

[0143] Second, during the unilateral detour, the robot's head always follows the movement of the human face. When the direction of the robot's head is consistent with the unilateral direction of the obstacle, it means that the obstacle still needs to be detoured. If they are inconsistent, it means that the detour can be stopped and the robot's body can be turned to face the human face. This point is defined as the departure point, and the next face target tracking can be carried out, and whether there is a new obstacle can be detected at all times.

[0144] When the above two situations are triggered, the point is defined as the departure point, and the departure turn is performed to align with the target face and switch the obstacle avoidance behavior to the face tracking behavior.

[0145] 5. Leave the turn and switch to face tracking mode

[0146] After finding the escape point, the robot turns away. After multiple turns, the robot may lose the face. At this time, the steering angle needs to be obtained. The steering angle value of the detour steering behavior is added to the steering angle value after encountering multiple obstacles in the detour obstacle behavior to obtain the steering angle of the robot during the obstacle avoidance process. After modulating this value by 360, the net value of the robot's steering angle is obtained. After the obstacle avoidance is completed, the robot needs to turn again and look for the face.

[0147] When the net value of the robot's turning angle is above 180°, the robot loses the direction of the face. At this time, the robot is commanded to record the turning angle, turn back to the corresponding angle after the detour, search for the face again, and then turn away according to the direction of the face.

[0148] 3. Obstacle avoidance strategy in the case of multiple obstacles

[0149] When the robot avoids obstacles, in a multi-obstacle environment, different solutions are adopted for the following different situations:

[0150] 1. When the robot encounters an obstacle and turns around, it encounters a new obstacle, such as Figure 4 When the robot turns, in a multi-obstacle environment, if there is no obstacle and the sonar on one side or both sides detects a new obstacle, the robot will continue to turn in the direction selected by the previous obstacle end until:

[0151] 1) If the turning angle exceeds 360 degrees, the robot will not be able to reach the target;

[0152] 2) Turn to the point where obstacles appear on one side of the robot's sonar and there are no obstacles on the other side.

[0153] Perform a unilateral detour again;

[0154] 2. When the robot is bypassing an obstacle at the edge, a new obstacle appears. When bypassing an obstacle on the left side, a new obstacle appears on the left or right side of the robot's sonar. In this case, the obstacle may be multiple obstacles or convex polygonal obstacles. At this time, another unilateral turn and bypassing obstacle behavior will be performed. When the robot is bypassing the edge of the obstacle, when a new obstacle appears on the other side without an obstacle, because the robot bypasses a section close to the obstacle, after the obstacle appears on the other side, the distance between the two obstacles is less than the width of the robot.

[0155] 2. Precise positioning of intelligent robots based on face recognition

[0156] (I) Necessity of achieving precise positioning

[0157] Because the faceWidth parameter set in the robot face tracking module is an estimated value when measuring the distance between the face and the robot, a rough estimate of the width of the face is made, resulting in unsatisfactory positioning accuracy. However, when faceWidth is set to the same size as the real face, the experimental results are not very ideal.

[0158] The effect of the intelligent robot's face tracking module finally reaching the target position is not related to the real face size. The final positioning stop distance is within 25 to 30 cm in front of the person. This is because as the robot and the person get closer, due to the angle, the robot's face tracking module's recognition effect on the face will drop significantly, and the positioning cannot be guaranteed to be effective and accurate. To obtain such test results, it is also necessary to measure the real value of the face of the tracking target, and then set the face parameters each time the tracking is performed.

[0159] When the faceWidth parameter is fixed at 0.1m, as the size of the real face increases, the distance between the robot and the human after the robot stops will also increase. This positive proportional relationship shows that the robot's face tracking module can always recognize the face in the image during the final positioning and stopping process, ensuring the effectiveness of the positioning result. However, the final stopping distance between the robot and the human is obviously not accurate enough and cannot meet the distance for human-machine interaction.

[0160] (II) Precision Positioning Architecture

[0161] The problem faced in the second stage is that after the robot completes the rough positioning in the first stage, it will stop at a rough position of 30cm to 40cm away from the target person. Obviously, this cannot meet the final human-computer interaction distance requirements for face tracking.

[0162] Therefore, in the first stage of rough positioning of the face tracking module, in order to ensure that the face will not be lost due to the close distance between the human and the machine, and stop the face tracking activity, and to ensure the effectiveness of positioning, the estimated value of the threshold faceWidth of the face size is fixed to 0.1m, which is slightly smaller than the real value of the universal face. In this way, the robot will be between 30 and 40cm away from the human. Then, the face is accurately positioned visually, and the robot moves forward to the precise position Xcm (which can be set by parameters) in front of the target person for human-machine interaction to make up for the defects of rough positioning.

[0163] The precise positioning method proposed in this application is an image processing method that uses a monocular vision positioning architecture and face recognition after the rough positioning in the previous stage. The architecture is as follows:

[0164] Through the relationship between the robot's camera, the robot's image coordinate system, and the real-world coordinate system, a geometric model is constructed, and the known parameters that can be used in the model are obtained to solve the position of the target from the camera and obtain the precise position (distance and direction) of the target person from the intelligent robot.

[0165] The method of obtaining pixel coordinates from the robot image coordinate system is to take a picture through the robot's local camera, transmit it remotely to a computer for image processing, detect the face in the image through the Harr to Like feature and return the coordinates.

[0166] After obtaining the precise position of the target person, the robot is controlled to move to a distance X cm in front of the target person.

[0167] (III) Monocular vision positioning model framework

[0168] The intelligent robot has two cameras on its head. Figure 5 As shown. One is located at the forehead of the robot, 63.64 mm from the bottom of the robot head. One is at the mouth of the robot, 17.74 mm from the bottom of the robot head. The monocular camera used in this application is the camera on the forehead, which is the 0 point marked in the figure.

[0169] The wide angle of the camera on the forehead of the intelligent robot forms a two-dimensional image composed of the horizontal and vertical axes of the image coordinates. The wide angle of the camera on the head of the intelligent robot is 47.64° in the vertical direction, plus the angle that the head of the intelligent robot can rotate vertically. The intelligent robot can observe targets within a range of 53.3° below the horizontal line and 90° above the horizontal line, a total of 143.3° in the vertical direction. The horizontal wide angle of the intelligent robot is 60.97°. The robot can locate targets within 30 degrees to the left and right of the robot's visual center.

[0170] 1. Image coordinate system, camera coordinate system, and world coordinate system of intelligent robots

[0171] 1) Image coordinate system

[0172] With the upper left corner of the image as the origin and pixels (px) as the unit, a coordinate system uv is established. The horizontal and vertical coordinates of a pixel are the column and row numbers of the pixel in the image array, respectively.

[0173] Assume that the entire image size is an intelligent robot camera image of 20*20 pixels. In the uv coordinate system with the upper left corner of the image as the origin, the coordinates of the origin u are (0, 0), and the coordinates of the image center point O are (10, 10);

[0174] If it is converted into a coordinate system with O as the origin relative to the center of the image, the conversion relationship is shown in Formula 1. At this time, assuming that (u0, v0) is the coordinate of O1 in the uv coordinate system, dx, dy is the physical size of a pixel in the x and y axis directions, then the physical coordinates of any pixel P (x, y) in the O|xy coordinate system are:

[0175]

[0176] The coordinate system of the image thus obtained is O 1xy , where Q1 is the coordinate of the center point of the face, P1 is the coordinate of Q1 on the y-axis vector, and U (u, v) represents the coordinate system of the image pixels.

[0177] 2) Camera coordinate system

[0178] The camera coordinate system is Figure 6 As shown, point O is the origin of the camera coordinate system, where X c Axis and Y c The axes are represented as the two dimensions of the camera coordinate system, which are parallel to the x-axis and y-axis of the image coordinate system, where x c Axis and world coordinate system X w Axis parallel, Y c Axis and world coordinate system Y w The angle between the Z and Z axes is the vertical rotation angle of the robot head. c It coincides with the z-axis of the image coordinate system and is perpendicular to the two planes.

[0179] The focal length of the camera is f OO1, and the horizontal focal length of the intelligent robot is f x With vertical focal length f y The same, then we can deduce:

[0180]

[0181] 3) World Coordinate System

[0182] The robot's world coordinate system O w Xw Y w Z w The coordinate system is established at the bottom of the robot's feet. Point O is the position of the robot's head camera. 001 is the focal length of the camera, which is f, and the angle with the Y axis is α. The imaging position of any point P in space on the image is approximated by the pinhole model. M1 is the internal parameter of the camera, which is completely determined by f. x 、f y , u0, v0, which is only related to the internal structure of the camera. M2 is the external parameter of the camera, which is completely determined by the orientation of the camera relative to the world coordinate system. R is a 3*3 orthogonal unit matrix, t is a three-dimensional translation vector, and vector O = (0, 0, 0).

[0183] 2. Model based on monocular vision of intelligent robots

[0184] The positioning model of monocular vision space points is established as follows Figure 7 The robot stands vertically in its world coordinate system 0 w Right above the point, 0 is the position of its camera. The intelligent robot faces the negative half axis direction of the Y axis, with its head raised. At this time, a human face appears obliquely to the right of the robot and is captured by the camera. The pixel imaging coordinates appearing on the camera are Q1(u1, v1). NQ' represents the person. The height of NQ' at the center of the face is H. The height of the robot camera is 00. w is h, the angle of view when the head is raised is the angle between the optical axis OL and the Y axis is α, which is obtained by the head motor parameters of the robot. By calibrating the camera, the focal length of the camera is f, and Q' is the center point of the face obtained after identifying the image processing target.

[0185] Assuming NO w The extended line of the line segment and the extended line of point Q'0 intersect at point Q(X w , Y w ), triangle NQQ' and 0 w Similar to Q0, the coordinates of point Q1 (the center of the face in the image coordinate system) are (x, y), which are converted from the coordinates of Q1 in the uv coordinate system (u1, v1) through formula 1. γ represents ∠OO1P1, and β is the angle between the extension line of OP1 and the Y axis.

[0186] Find O w The distance N, that is, the distance from the robot to the human foot: Figure 7 The angle relationship between a, β, and γ can be basically obtained. When the mapping Q1(x, y) of the midpoint Q'(x, y) of the top of the forehead of the face appears in the third and fourth quadrants of the image coordinates, the angle relationship at this time is:

[0187] β=α+γ Formula 3

[0188] When the mapping Q(x, y) of the midpoint Q'(x, y) of the top of the forehead of the face appears in the first and second quadrants of the image coordinates, the angle relationship at this time is:

[0189] β=α-γ Formula 4

[0190] First find the horizontal coordinate X of point P through the constructed triangle w Length:

[0191]

[0192] y is calculated by formula 1, and the horizontal coordinate of point P is obtained as formula 7, f x =f / d x , f y =f / d y , represent the equivalent focal length expressed in pixels in the X-axis and Y-axis directions respectively;

[0193]

[0194] Figure 7 The coordinates of point Q on the mid-horizontal plane are (X w , Y w ), where Y w =PQ, PQ is calculated according to the similarity theorem to obtain formula 8, OP and OP1 are calculated according to formula 9 and formula 10, and the ordinate Y of point Q is calculated according to formula 8 w As shown in formula 12:

[0195]

[0196]

[0197] P1Q1=(u-u0)dx Formula 11

[0198]

[0199] Get Q(X w , Y w ) and then, through the two constructed triangles NQQ' and 0 w Q0 is similar, find O w The length of N, according to the triangle similarity theorem, is:

[0200]

[0201] Get the distance from the intelligent robot to the human foot.

[0202] 3. How to obtain a person's height

[0203] The above model needs to obtain the height of the person as a parameter for the robot's visual positioning. The height of the person is obtained by combining the robot's visual information extraction with the outside world's movement and walking.

[0204] When the robot approaches the person's height in a straight line, the face is at the center of the visual image during two face trackings. At this time, the robot will obtain the information of the head angle.

[0205] The robot performs face tracking. The height of the robot is h. When it sees a face, it records the current angle of the robot's head as ∠2. At this time, the robot moves forward a distance S and captures the face again. The angle of the robot's head is ∠1. The height of the person H is expressed as:

[0206]

[0207] The behavior of obtaining the height of a person here is obtained during the process of the robot's face tracking and obstacle avoidance. This process is repeated many times during the obstacle avoidance and face tracking process. In order to improve the accuracy of calculating the height of a person, the height of a person is obtained multiple times, and the average value is finally taken to participate in the final visual positioning operation.

[0208] (IV) Specific implementation of face recognition

[0209] 1) Detect whether a face appears: Use the FaceDetected subscription event in the robot's ALFaceDetection;

[0210] 2) Remotely obtain facial images: After detecting a face, the robot stops turning its head up and down, and keeps its head at the position where the face appears in the robot's visual image. It uses the getImageRemote method in ALVideoDevice to capture a latest image from the camera's video source and convert it into an image that conforms to the set format to obtain images of different resolutions and color standards. Finally, this group of images is loaded into an array container containing a binary image and the image's width, height, and color space information in the form of ALValue, and transmitted to the remote computer through the network;

[0211] 3) Computer local image processing: The acquired image is processed on the remote computer using the image tool of Opencv2.1.0, and the scientific computing numpy is used to reorganize the array rows and columns of the acquired image, and finally a 480*640 array format image is obtained.

[0212] 5. Accurate face positioning method

[0213] The entire visual positioning process is combined with the robot's obstacle avoidance algorithm to participate in the robot's face tracking and obstacle avoidance process. In the final stage of face tracking, the robot needs to calculate the distance between the robot and the person, and accurately walk in front of the person to conduct human-computer interaction. Therefore, a visual positioning model and algorithm for intelligent robots based on face recognition is proposed. By obtaining the height and face image of the person, a more accurate distance between the person and the intelligent robot is calculated.

[0214] The validity of the rough face positioning result is ensured by setting the estimated value of faceWidth to a critical value smaller than the standard face size. The second stage of this application: precise visual positioning of intelligent robots based on face recognition starts from the end of rough face positioning, and the target person is no longer moving. The following are the specific processes and judgment criteria.

[0215] 1) When the intelligent robot performs face tracking, each time it switches from obstacle avoidance to face tracking, it roughly locates the face and walks to the target person;

[0216] a. If the target person is not moving, the robot will stop about 30 to 40 cm in front of the target person; at this time, the first stage ends and the robot performs precise visual positioning behavior;

[0217] b. If the target person continues to move, the robot will continue to follow the person's face and avoid obstacles;

[0218] 2) Adjust the direction of the robot's body to be consistent with the head so that the robot can recognize the face acquired by the remote image;

[0219] 3) After the recognition is completed, the height of the person obtained by the previous robot movement is combined with the visual positioning model to calculate the direction and distance of the face;

[0220] 4) If there are no obstacles detected within the sonar detection range, as the robot moves, the sonar detection distance is shortened to ensure that people are not identified as obstacles by the robot;

[0221] 5) If an obstacle appears within the sonar detection range, the robot will continue to follow the face and avoid the obstacle;

[0222] 6) Finally, the vehicle moves to a position about X cm in front of the target person, and the accurate face positioning process is completed, preparing for the next face tracking and obstacle avoidance behavior.

[0223] 3. Experimental Results and Analysis

[0224] The experimental test is divided into two processes.

[0225] Process 1: The robot follows and avoids faces in an obstacle environment. The two parameters of sonar detection distance: sonarDist and each turning angle: alph are adjusted to analyze the impact of each parameter setting on the experimental results. The evaluation indicators of the experiment are the length of the journey and the time spent. The results of the application of the three most popular local path planning algorithms on intelligent robots are compared, and the advantages and disadvantages of the bug obstacle avoidance algorithm proposed in this application in the face following scenario are analyzed.

[0226] The second process: After the robot has finished following the face, the person stops moving, accurately visually locates the face, and walks to a specified distance from the person. The error of the robot's walking distance s used to measure the person's height is measured, and the degree of error in the measurement of the person's height caused by the error in the walking distance s is analyzed, which in turn causes the degree of error in the measurement of the face positioning. The positioning effect is compared with the face tracking module that comes with the intelligent robot, and the advantages and disadvantages of this application are analyzed.

[0227] (I) Experimental results and analysis of obstacle avoidance algorithm based on face recognition

[0228] The experimental test of the obstacle avoidance algorithm based on face recognition simulated a multi-obstacle environment in the laboratory through a robot entity. The intelligent robot followed the person and avoided multiple random obstacles during the walking process. Among them, the critical values ​​faced by the intelligent robot in obstacle avoidance include the robot's sonar detection distance, the buffer distance during the robot's detour, and the angle of each turn of the robot during the turning process. Different values ​​were substituted respectively, and the results obtained were divided into the number of collisions with obstacles during the obstacle avoidance process, the total path length of the robot's walking, and the total time spent on walking, to evaluate the entire obstacle avoidance process.

[0229] In the experimental scene and the robot's motion trajectory, the entire scene is an indoor environment with a length of 4 meters and a width of 2 meters. Three obstacles are placed in the environment. The obstacles are rectangular cardboard obstacles with a width of 1.2 meters, a length of 0.3 meters, and a height of 0.5 meters. The interval between the two obstacles is 0.75 meters.

[0230] Here we optimize the parameters in the obstacle avoidance process. When alpha is 45 degrees and the sonar detection distance is 0.35: Figure 8 The data analysis shows that the number of turns is positively correlated with the time spent walking. Since the robot turns in place when turning, the turning process is not counted in the robot's odometer, and the robot's walking path has no direct relationship with each turning angle. Therefore, the relationship between each turning angle and the time spent is analyzed here. While ensuring that the buffer distance remains unchanged, the alpha turning angles are adjusted to 15 degrees, 30 degrees, and 45 degrees respectively, and the time spent is compared.

[0231] The influence of the number of turns per turn on the time spent: It can be concluded from the experimental data that the angle of each turn is inversely proportional to the average number of turns. The average number of turns changes significantly when the turning angle changes from 15 degrees to 30 degrees, while the average number of turns does not decrease much from 30 degrees to 45 degrees and then to 60 degrees. The reason for this phenomenon may be that in most cases, the turning angle problem faced by the robot is less than 90 degrees. Each turning angle of 15 degrees requires 4 to 6 turns, while each turning angle of 30 degrees or 45 degrees only requires 2 to 3 turns. When the turning angle is 60 degrees each time, although the number of turns is reduced to a certain extent, the average time spent and the average walking distance will increase to a certain extent. This is because in the process of robot obstacle avoidance, the turning angle of more than 90 degrees rarely occurs, and when the turning angle of more than 45 degrees occurs more than twice, the robot will incur a greater distance cost and time cost due to over-steering. The average time spent is the least when the turning angle is 45 degrees each time. The average walking distance is also considerable. The number of turns per time does not have much effect on the average walking distance, and the difference between the two maximum values ​​is 10 cm. This is because there is no change in the robot's displacement when turning a corner. The small number of turns per time makes the robot's walking path relatively short when bypassing obstacles, but there is no significant difference.

[0232] Increasing SonarDist can reduce the average time spent, but the cost is an increase in the average path length and the distance between obstacles. In addition to adjusting its own parameters, the experiment also compared with three other dynamic path planning algorithms. Fig. 9 The obstacle avoidance results of the intelligent robot in the experimental scene using the artificial potential field method, TangentBUG algorithm, TangentBUG algorithm with memory direction, and improved BUG algorithm for face tracking are listed respectively. From the comparison of experimental results, it can be seen that the artificial potential field method takes a long time, and the improved BUG algorithm for face tracking takes a shorter average time. The real-time advantage of this method is quite obvious.

[0233] (II) Experimental results and analysis of precise positioning of intelligent robots based on face recognition

[0234] Since the walking position s of the robot is involved in the calculation of the human height when locating the height, the error in the calculation of the human height is involved in the calculation of the final positioning of the monocular vision model, resulting in a certain deviation in the final stop position of the intelligent robot. Here, the precise visual positioning of the robot is analyzed, and the relationship between the robot's walking distance s, the actual walking distance S, the measured human height h, the human's actual height H and the visual positioning calculation results and errors is analyzed.

[0235] The measurement results are as follows Fig.10As shown, the actual height of the test person is 172cm, the visual positioning distance is 1m, s is the set walking distance, each group of walking distance tests is performed 10 times, s actual average distance is the actual walking distance of the robot each time, the average height value is the calculated height value of the person, and the visual positioning calculation value is the calculated value of the distance between the robot and the person when the visual positioning is 1m.

[0236] From the above measurement data, we can see that when the robot walks, the walking distance is larger than the actual distance due to inertia and the robot's footsteps slipping. Because the s distance is too high, plus the angle error of the robot's head motor, the average height measurement is smaller than the actual height. The accuracy of the calculated value of the robot's visual positioning is between 7% and 8%, and the errors are all negative. However, since the errors between the robot's walking value and the set value are all positive, the final walking effect is better than the measured value. By using the monocular vision positioning model, a more accurate positioning result for the face positioning of the intelligent robot was obtained, and the experiment verified that this method is effective.

Claims

1. An intelligent robot obstacle avoidance and precise positioning method based on face tracking, characterized in that: First, based on the process of the robot following faces in an unknown environment, avoiding obstacles when encountering obstacles, and returning to face following behavior after leaving the obstacles, an obstacle avoidance method of the intelligent robot based on face following and the improved BUG algorithm is constructed, and an improved algorithm for obstacle detour and steering that supports small-angle obstacle avoidance is proposed for the small-angle sonar detection robot. The accurate switching time between face following behavior and BUG algorithm obstacle avoidance behavior is found, and the robot's obstacle avoidance methods in complex environments with multiple obstacles are summarized and processed, and the parameters of the obstacle avoidance strategy under multiple obstacles are adjusted and tested; second, a method for precise visual positioning of intelligent robots based on face recognition is constructed, the image coordinate position of the face is identified according to image processing, and a model based on the monocular vision of the intelligent robot is constructed for visual positioning in combination with the robot's hardware parameters and the height of the person; The target-following behavior of the intelligent robot is divided into two stages: Phase 1: During the roughly positioned face tracking process, when encountering an obstacle, the intelligent robot performs obstacle avoidance behavior using the improved BUG algorithm: the robot moves by detecting the face and following the movement of the face. The movement process is implemented through the ALFaceTracker method of the robot's ALTrackerProxy module. The face that appears in the image is detected and followed, and the face is located according to the set face mean width faceWidth. After obtaining this position, the robot is controlled to move in front of the person. The robot builds a spatial model and uses its own displacement to measure the person's height. When the robot encounters an obstacle blocking the robot's progress while performing face tracking, it switches from face tracking behavior to improved BUG algorithm obstacle avoidance behavior. When the robot determines that there are no obstacles on the route in the direction of the face, it switches from obstacle avoidance behavior to face tracking behavior until the target stops moving. When the robot's rough positioning is completed and there are no obstacles within the sonar detection range, Phase 1 ends. Phase 2: After completing the roughly positioned face following behavior, the robot accurately positions the face and walks to a position at a specified distance from the person: the robot uses the position of the face in the image coordinate system, the height of the person obtained by using the method of obtaining the height of the person, and the camera coordinate system parameters obtained by calibration as operation parameters, and uses the specific implementation of face recognition to build a geometric model for operation. Based on the visual positioning model and algorithm of the intelligent robot based on face recognition, the robot obtains a more accurate orientation of the face in the world coordinate system and walks to a position at a specified distance in front of the person. How to obtain a person's height: The robot uses visual information extraction and the outside world uses movement and walking to obtain the person's height: The height of a person is measured by the robot. When the robot and the person are in a straight line, the face is in the center of the visual image after two face trackings. At this time, the robot will obtain the information of the head angle respectively. The robot performs face tracking activities. The height of the robot is h. When the robot sees the face, the current angle of the robot's head is recorded as ∠2. At this time, the robot moves forward a distance S and captures the face again. The angle of the robot's head is ∠1. The height of the person H is expressed as: The behavior of obtaining the height of a person here is obtained during the process of the robot tracking the face and avoiding obstacles. This process is repeated many times during the obstacle avoidance and face tracking process, and the height of the person is obtained many times, and finally the average value is taken to participate in the final visual positioning calculation; Visual positioning model and algorithm of intelligent robot based on face recognition: by obtaining the height and face image of a person, a more accurate distance between the person and the intelligent robot can be calculated; The validity of the rough face positioning result is ensured by setting the estimated value of faceWidth to a critical value smaller than the standard face size. The second stage of this application: precise visual positioning of intelligent robots based on face recognition starts from the end of rough face positioning. The target person is not moving. The following are the specific processes and judgment criteria: 1) When the intelligent robot performs face tracking, each time it switches from obstacle avoidance to face tracking, it roughly locates the face and walks to the target person; a. If the target person is not moving, the robot will stop 30 to 40 cm in front of the target person; at this stage, the robot will perform precise visual positioning; b. If the target person continues to move, the robot will continue to follow the person's face and avoid obstacles; 2) Adjust the direction of the robot's body to be consistent with the head so that the robot can recognize the face acquired by the remote image; 3) After the recognition is completed, the height of the person obtained by the previous robot movement is combined with the visual positioning model to calculate the direction and distance of the face; 4) If there are no obstacles detected within the sonar detection range, as the robot moves, the sonar detection distance is shortened to ensure that people are not identified as obstacles by the robot; 5) If an obstacle appears within the sonar detection range, the robot will continue to follow the face and avoid the obstacle; 6) Finally, the system moves to a position at a specified distance in front of the target person, and the accurate face positioning process is completed, preparing for the next face tracking and obstacle avoidance behavior.

2. According to claim 1, the intelligent robot obstacle avoidance and precise positioning method based on face tracking is characterized in that: Improved BUG algorithm framework: The factors considered when improving the BUG algorithm to optimize the detour walking strategy include: the human-like walking method of the intelligent robot, the way the sonar detects obstacles, and the dynamic walking direction of the target. Improvements are made to how to determine the starting and ending points of obstacle detection and the specific detour strategy; The robot's circumnavigation process is divided into two modes: straight-line walking and turning. The turning behavior is further divided into two modes: circumnavigation turning and departure turning from the perspective of turning properties. 1) Detour and turn: When the intelligent robot encounters an obstacle in the obstacle avoidance detection area, it needs to turn around the obstacle; 2) Leaving and turning: After the intelligent robot determines that it has reached the departure point, it has left the influence range of the obstacle and switches back to the face following mode to perform the turning process required for face following; By turning and walking in a straight line, the overall walking route map is presented in the form of multiple broken lines, and the walking behavior of each broken line can be determined to be safe; The robot triggers obstacle avoidance behavior in three possible situations during face following motion, as follows: a) An obstacle appears in the obstacle avoidance detection area on the left side of the sonar; b) An obstacle appears in the obstacle avoidance detection area on the right side of the sonar; c) Obstacles appear in both the left and right obstacle avoidance detection areas of the sonar; According to the triggering conditions of three different situations, the following two-stage obstacle avoidance strategy is implemented to deal with the obstacle avoidance situation accompanied by face tracking in an environment with multiple obstacles: 1) turning around obstacles, and 2) turning around the edge of obstacles.

3. The intelligent robot obstacle avoidance and precise positioning method based on face tracking according to claim 1 is characterized in that: Improved algorithm for detour and steering when encountering obstacles: When the intelligent robot detects an obstacle in the obstacle avoidance detection area in the direction of travel, it stops following the human face and determines whether the distance between the robot and the obstacle is greater than or equal to the steering distance. When the robot is farther away from the obstacle than the steering distance, the difference between the distance measured by the forward sonar and the steering distance is calculated; when the robot is farther away from the obstacle than the steering distance, the difference between the backward steering distance and the distance measured by the sonar is calculated to ensure a safe steering distance ahead. The real-time location of the intelligent robot after forward or backward correction is then marked as the obstacle point. The real-time location of the point and the robot's forward direction are recorded through the motion.getlocation method. At this time, the location of the obstacle is determined, and different detour and steering strategies are selected according to the different locations of the sonar obstacles on both sides. At this time, different detour strategies are selected according to the direction of the obstacle encountered by the robot. When obstacles appear on both sides of the robot, the obstacle avoidance detection areas on the left and right sides of the robot detect obstacles. At this time, it is no longer possible to quickly detour the obstacles by simply avoiding obstacles on one side. The movement direction in the same direction as the target to be followed, that is, the movement direction of the face, is selected as the obstacle avoidance direction to reduce the overall time cost. When the robot encounters obstacles on both sides, it turns according to the obstacle avoidance turning strategy on both sides, and the obstacle avoidance problem on both sides will be transformed into obstacle avoidance on one side. When the robot encounters obstacles on one side and turns to avoid them, the sonar on the other side still needs to detect whether there are new obstacles at any time. If a new obstacle is detected in the other obstacle-free sonar, obstacles will appear in the obstacle avoidance detection areas on both sides, and the problem will be transformed into obstacle avoidance behavior on both sides, and the face direction will be preferred for obstacle avoidance. When an obstacle appears on one side of the intelligent robot, the robot chooses the direction away from the obstacle as the detour direction, and each time it turns at an angle of β, it obtains the sonar parameters after each turn. When the obstacle is a convex obstacle, after each turn, the robot uses the obtained sonar parameters to correct the position of the robot and the obstacle by moving forward or backward to ensure that a turning distance is met in front, so as to ensure the safe turning of the intelligent robot. When the robot detects an obstacle only on one side of the sonar, the sonar detection on the other side of the robot indicates that there is no obstacle in the obstacle avoidance detection area on the other side. The robot quickly avoids the obstacle by turning at a small angle, adopting a steering strategy in the direction of avoiding the obstacle, while ignoring the direction of movement of the target. The steering angle is determined on one side of the steering direction. At this time, the obstacle position is determined by the steering angle. If the steering angle is too large, the robot is facing a multi-obstacle or maze-like obstacle environment. When the cumulative steering angle is greater than 360°, it indicates that there are obstacles all around the robot, and the face tracking target cannot be reached.

4. The intelligent robot obstacle avoidance and precise positioning method based on face tracking according to claim 1, characterized in that: Obstacle avoidance strategy in the case of multiple obstacles: When the robot avoids obstacles, in a multi-obstacle environment, different solutions are adopted for the following different situations: 1- When the robot encounters an obstacle and turns around, it encounters a new obstacle. When the robot turns, in a multi-obstacle environment, if there is no obstacle and the sonar on one side or both sides detects a new obstacle, the robot will continue to turn in the direction selected by the previous obstacle end until: 1) If the turning angle exceeds 360 degrees, the robot will not be able to reach the target; 2) Turn to the point where obstacles appear on one side of the robot's sonar and there are no obstacles on the other side. Perform a unilateral detour again; 2-When the robot is bypassing an obstacle at the edge, a new obstacle appears. When bypassing an obstacle on the left side, a new obstacle appears on the left or right side of the robot's sonar. In this case, the obstacles may be multiple obstacles or convex polygonal obstacles. At this time, another unilateral turn will be performed to bypass the obstacle. When the robot is bypassing the edge of the obstacle, when a new obstacle appears on the other end without an obstacle, the robot bypasses a section close to the obstacle. After the obstacle appears on the other end, the distance between the two obstacles is less than the width of the robot.

5. The intelligent robot obstacle avoidance and precise positioning method based on face tracking according to claim 1, characterized in that: Precise positioning architecture: In the first stage, when the face tracking module roughly locates the human face, the estimated value of the face width threshold is fixed to 0.1m, which is smaller than the true value of the universal face. In this way, the robot will be between 30 and 40cm away from the human. Then, the face is accurately visually located, and the robot moves forward to a position at a specified distance in front of the target person for human-machine interaction, which is used to make up for the defects of rough positioning. The precise positioning method is to use the monocular vision positioning architecture and face recognition image processing method after the rough positioning in the previous stage. The architecture is as follows: Through the relationship between the camera coordinate system, the image coordinate system, and the world coordinate system, a geometric model is constructed to obtain the known parameters that can be used in the model to solve the position of the target from the camera and obtain the precise position of the target from the intelligent robot. The method of obtaining pixel coordinates from the robot image coordinate system is to take a picture through the robot's local camera, transmit it remotely to a computer for image processing, detect the face in the image through the Harr to Like feature and return the coordinates; After obtaining the precise position of the target person, the robot is controlled to move to a position at a specified distance in front of the target person.

6. The intelligent robot obstacle avoidance and precise positioning method based on face tracking according to claim 1, characterized in that: The image coordinate system, camera coordinate system, and world coordinate system of the intelligent robot: 1) Image coordinate system: Take the upper left corner of the image as the origin and pixel (px) as the unit to establish the coordinate system uv. The horizontal and vertical coordinates of the pixel are the column and row numbers in the image array respectively. Assume that the entire image size is an intelligent robot camera image of 20*20 pixels. In the uv coordinate system with the upper left corner of the image as the origin, the coordinates of the origin u are (0, 0), and the coordinates of the image center point O are (10, 10); If it is converted into a coordinate system with O as the origin relative to the center of the image, the conversion relationship is shown in Formula 1. At this time, assuming that (u0, v0) is the coordinate of O1 in the uv coordinate system, dx, dy is the physical size of a pixel in the x and y axis directions, then the physical coordinates of any pixel P (x, y) in the O|xy coordinate system are: The coordinate system of the image thus obtained is O 1xy , where Q1 is the coordinate of the center point of the face, P1 is the coordinate of Q1 on the y-axis vector, and U (u, v) represents the coordinate system of the image pixels; 2) Camera coordinate system: Point O is the origin of the camera coordinate system, where X c Axis and Y c The axes are represented as the two dimensions of the camera coordinate system, which are parallel to the x-axis and y-axis of the image coordinate system, where x c Axis and world coordinate system X w Axis parallel, Y c Axis and world coordinate system Y w The angle between the Z and Z axes is the vertical rotation angle of the robot head. c It coincides with the z-axis of the image coordinate system and is perpendicular to the two planes; The focal length of the camera is f OO1, and the horizontal focal length of the intelligent robot is f x With vertical focal length f y The same, then we can deduce: 3) World coordinate system: The robot's world coordinate system O w X w Y w Z w The coordinate system is established at the bottom of the robot's feet. Point O is the position of the robot's head camera. 001 is the focal length of the camera, which is f, and the angle with the Y axis is α. The imaging position of any point P in space on the image is approximated by the pinhole model. M1 is the internal parameter of the camera, which is completely determined by f. x 、f y , u0, and v0, which are only related to the internal structure of the camera. M2 is the external parameter of the camera, which is completely determined by the orientation of the camera relative to the world coordinate system. R is a 3*3 orthogonal unit matrix. t is a three-dimensional translation vector, and the vector O=(0, 0, 0).

7. The intelligent robot obstacle avoidance and precise positioning method based on face tracking according to claim 1, characterized in that: Model based on monocular vision of intelligent robot: the robot stands vertically in its world coordinate system 0 w Right above the point, 0 is the position of its camera. The intelligent robot faces the negative half axis direction of the Y axis, with its head raised. At this time, a human face appears obliquely to the right of the robot and is captured by the camera. The pixel imaging coordinates appearing on the camera are Q1(u1, v1). NQ' represents the person. The height of NQ' at the center of the face is H. The height of the robot camera is 00. w is h, the angle of the head when it is raised is the angle between the optical axis OL and the Y axis, which is α, obtained by the robot's head motor parameters. The focal length of the camera is obtained by calibrating the camera, and Q' is the center point of the face obtained after the image processing target is recognized; Assuming NO w The extended line of the line segment and the extended line of point Q'0 intersect at point Q(X w , Y w ), triangle NQQ' and 0 w Similar to Q0, the coordinates of the face center point Q1 in the image coordinate system are (x, y), which are converted from the coordinates of Q1 in the uv coordinate system (u1, v1) through formula 1. γ represents ∠OO1P1, and β is the angle between the extension line of OP1 and the Y axis; Find O w N, that is, the distance from the robot to the human foot: get the angle relationship between a, β, and γ. When the mapping Q1(x, y) of the midpoint Q'(x, y) of the top of the forehead of the human face appears in the third and fourth quadrants of the image coordinates, the angle relationship at this time is: β=α+γ Formula 3 When the mapping Q(x, y) of the midpoint Q'(x, y) of the top of the forehead of the face appears in the first and second quadrants of the image coordinates, the angle relationship at this time is: β=α-γ Formula 4 First find the horizontal coordinate X of point P through the constructed triangle w Length: y is calculated by formula 1, and the horizontal coordinate of point P is obtained as formula 7, f x =f / d x , f y =f / d y , represent the equivalent focal length expressed in pixels in the X-axis and Y-axis directions respectively; The coordinates of point Q on the horizontal plane are (X w , Y w ), where Y w =PQ, PQ is calculated according to the similarity theorem to obtain formula 8, OP and OP1 are calculated according to formula 9 and formula 10, and the ordinate Y of point Q is calculated according to formula 8 w As shown in formula 12: P1Q1=(u-u0)dx Formula 11 Get Q(X w , Y w ) and then, through the two constructed triangles NQQ' and 0 w Q0 is similar, find O w The length of N, according to the triangle similarity theorem, is: Get the distance between the intelligent robot and the human foot.

8. The intelligent robot obstacle avoidance and precise positioning method based on face tracking according to claim 1, characterized in that: Specific implementation of face recognition: 1) Detect whether a face appears: Use the FaceDetected subscription event in the robot's ALFaceDetection; 2) Remotely obtain face images: After detecting a face, the robot stops turning its head up and down, and keeps its head at the position where the face appears in the robot's visual image. It uses the getImageRemote method in ALVideoDevice to capture a latest picture from the camera's video source and convert it into an image that conforms to the set format to obtain images of different resolutions and color standards. Finally, this group of images is loaded into an array container containing a binary image and the image's width, height, and color space information in the form of ALValue, and transmitted to the remote computer through the network; 3) Computer local image processing: The acquired image is processed on the remote computer using the image tool of Opencv2.1.0, and the scientific computing numpy is used to reorganize the array rows and columns of the acquired image, and finally a 480*640 array format image is obtained.

Images