A focusing tracking method for robotic bionic eyeball

Abstract

A focusing tracking method for a robot bionic eyeball is based on a robot platform, a robot bionic eyeball rotatingly arranged in the robot platform, an eyeball camera image acquisition module arranged in the bionic eyeball and a processor control module, the processor control module is used for controlling the robot platform, edge point position data sets of left eyes, right eyes and a mouth are obtained through image acquisition, corresponding target points of the left eyes, the right eyes and the mouth are accurately calculated, whether the face target moves is judged according to different target point coordinates and distances from the image field center, focusing and tracking are carried out by using the looking strategy and the tracking strategy respectively, and the robot bionic eyeball tracks the face target position with a natural scanning amplitude. The application can greatly improve the focusing tracking efficiency of the robot bionic eyeball, almost realizes real-time tracking, thereby avoiding the delay and lag phenomenon, and the eye contact of the bionic robot is more natural.

Description

Technical Field

[0001] This invention relates to the technical field of bionic robot control algorithms, specifically to a focusing tracking method for a robot's bionic eyeball. Background Technology

[0002] With the rapid development of artificial intelligence and robotics, bionic robots are playing an increasingly important role in real life. These robots can not only mimic human appearances but also communicate naturally with people through rich facial expressions and body language. They have broad application prospects in fields such as elderly care, companionship, digital twins, and the metaverse.

[0003] In terms of elderly care and companionship, bionic facial expression robots can provide emotional support and daily care for the elderly, alleviating the pressure brought by an aging society. For example, bionic facial expression robots can converse with the elderly, recognize emotional changes, and provide appropriate feedback, enhancing their quality of life. In the fields of digital twins and metaverse, bionic facial expression robots can serve as a bridge between the physical and virtual worlds, providing users with immersive interactive experiences. Furthermore, through realistic expressions and movements, bionic facial expression robots can help users communicate and collaborate in virtual environments, expanding the boundaries of human interaction.

[0004] However, existing eye-tracking technologies typically employ image comparison analysis before and after acquisition, or marker position comparison analysis, to achieve target tracking. For example, patent document CN114787755A discloses an eye-tracking system and its usage method. The eye-tracking system includes: an eye marker, a marker information acquisition module, and a processing module. The usage method includes: acquiring an eye marker worn by a user; acquiring the position information of the user's eye marker; and analyzing the acquired eye marker position information to determine the direction of the user's gaze. While the above solution, using marker position comparison analysis, can meet the tracking and recognition requirements, its application in human-human interaction with bionic robots results in low tracking and focusing efficiency, leading to delays and stuttering, and unnatural eye contact. Furthermore, after tracking the target, there is no specific focusing and gaze strategy for human-computer interaction, resulting in a poor human-computer interaction experience and reducing the effectiveness of bionic robots in social scenarios. Summary of the Invention

[0005] To address the shortcomings of existing bionic robot control algorithms, this invention proposes a focusing and tracking method for bionic robot eyes that can significantly improve the focusing efficiency of bionic eye tracking, achieving near real-time tracking and avoiding delays or stuttering. This method also enables more natural eye contact in bionic robots and incorporates a focusing and gaze strategy that interacts with human eyes.

[0006] The specific technical solution is as follows:

[0007] A focusing and tracking method for a robot's bionic eyeball, based on a robot platform, a robot bionic eyeball rotating within the robot platform, an eyeball camera image acquisition module within the bionic eyeball, and a processor control module, wherein the processor control module controls the robot platform, includes the following steps:

[0008] Step 1: Use the robot's bionic eyeball to collect image data in its initial posture;

[0009] Step 2: Use Media Pipe's Holistic model (multimodal human perception model) to identify key facial features in the acquired image data, and obtain data sets of edge plotting points for the left eye, right eye, and mouth respectively;

[0010] Step 3: Calculate the center point coordinates of the left eye, right eye and mouth respectively based on different edge plotting point position data groups, and set them as the corresponding target points of the left eye, right eye and mouth. Randomly select a point within the face range on the symmetrical center line of the center point of the left eye and right eye as the inter-eye target point, and continuously collect image data to cyclically update the position coordinates of key parts of the face.

[0011] Step 4: Determine whether the face target has moved based on the distance between the position coordinates of the key facial features and the center of the field of view of the image data acquisition.

[0012] Step 5: If the target face is determined to be stationary, focus is achieved using a gaze strategy, causing the robot's bionic eyes to move naturally between the left eye, right eye, and the interocular target point; if a dialogue occurs, the robot's bionic eyes will move naturally between the mouth, left eye, right eye, and the interocular target point.

[0013] Step 6: If the movement of the face target is determined, a tracking strategy is adopted to control the robot's bionic eyeball and / or other joints of the robot platform to rotate according to the movement distance, so that the robot's bionic eyeball tracks the face target position with a natural scanning amplitude.

[0014] More preferably, the gaze strategy in Step 5 is as follows:

[0015] Step 5.1: In the initial state, the robot's bionic eye focuses on the target point between the eyes;

[0016] Step 5.2: After a random time interval of 2 to 5 seconds, the robot's bionic eye randomly selects the target point of the left or right eye for focusing;

[0017] Step 5.3: Perform 1 to 3 switching of target focus between the left and right eyes, with the number of switching times in each round being random and the switching period being a random time of 0.5 to 2 seconds;

[0018] Step 5.4: Then, the robot's bionic eyeball returns to focusing on the interocular target position;

[0019] Step 5.5: Repeat Step 5.2 to Step 5.4.

[0020] More preferably, dialogue can be generated by the mouth of the facial target or by other auditory modules receiving dialogue information, thereby determining whether a dialogue should be generated.

[0021] Step 5.6: If a dialogue is determined to occur, the robot's bionic eyeballs will rotate and focus on the target point of the mouth for a random period of 1 to 3 seconds, and maintain this focus for a random period of 0.5 to 2 seconds, before returning to focus on the target point between the eyes.

[0022] Step 5.7: Then, repeat Step 5.2 to Step 5.4 and Step 5.6 until it is determined that no new dialogue has been generated.

[0023] More preferably, in Step 6, the tracking strategy includes the following response strategies based on the size of its movement range:

[0024] Step 6.1: If it is determined that any target point is offset outward from the center of the field of view by no more than 20% of the field of view, small-range tracking is adopted, which only requires eye movement to achieve focus shift and tracking;

[0025] Step 6.2: If any target point is determined to have shifted outward from the center of the field of view by more than 20% of the field of view, large-scale tracking is adopted. This requires the robot's bionic eyeball and the robot platform's neck joint to rotate together to complete the transfer and tracking of the field of view center. During this process, the rotation of the robot platform's neck joint will lag behind the rotation of the robot's bionic eyeball by 0.1 seconds. Finally, after the field of view center focuses and locks onto the key facial target point, the robot's bionic eyeball is adjusted to maintain a neutral position, and the robot platform's neck joint maintains the angle of tracking and focusing on the target point.

[0026] More preferably, the tracking strategy in Step 6 also includes the following response strategies:

[0027] Step 6.3: If it is determined that any target point has deviated from the dynamic field of view, quickly switch the target point and use over-range tracking. Rotate the robot's body pose through the body joints such as the waist joint or leg joint of the robot platform to further expand the dynamic field of view.

[0028] Step 6.4: If the target point is tracked within the extended dynamic field of view, return to Step 6.1 to complete the focus shift and tracking at the center of the field of view;

[0029] Step 6.5: If the target point still cannot be tracked within the extended dynamic field of view, then perform a blinking action, which simultaneously returns the robot platform and the robot's bionic eyeball to their initial posture.

[0030] More preferably, in Step 3, the specific algorithm for calculating the center point coordinates of the left eye, right eye, and mouth based on different edge plotting point position data sets is as follows:

[0031] Suppose that the obtained edge plotting point position data set is

[0032] The general equation for constructing a circle using these edge points as centers is: (xa) 2 +(yb) 2 =r 2 ;

[0033] Then, the objective function is obtained:

[0034] We Take the partial derivatives with respect to a, b, and r, and set these partial derivatives to 0 to find the extreme points:

[0035]

[0036]

[0037] By solving the system of equations with partial derivatives equal to 0, we can obtain the values ​​of a, b, and r, and thus determine the center of the circle. The coordinates of the center point of the left eye, right eye, or mouth are the obtained center of the circle (a, b).

[0038] More preferably, the method for determining whether the face target has moved in Step 4 is as follows:

[0039] Let the target point of the left eye be (x l ,y l The target point for the right eye is (x). r ,y r The target point in the eye is (x) c ,y c The target point of the mouth is (x) m ,y m The center point of the image is (x). o ,y o The length, width, or diagonal length of the image is H;

[0040] From the Euclidean distance formula The distance between points i and j can be calculated, and from this, the distances between the left eye, right eye, mouth, and the target points between the eyes and the center point of the picture can be calculated.

[0041] If the bionic eyeballs of the robot select to fixate on and focus on the left eye, right eye, mouth, or the target points between the eyes, then the corresponding or is assigned to D;

[0042] When D < H / 5, only the bionic eyeballs of the robot rotate to achieve focus transfer and tracking, and the movement amplitude of the eye servo per frame is D / 2H;

[0043] When D > H / 5, the movement amplitude of the bionic eyeball servo of the robot per frame is D / 2H, while the other joint parts of the robot platform move with a 0.1 - second delay relative to the bionic eyeballs of the robot, and the movement amplitude of the other joint parts of the robot platform is D / 2H;

[0044] When, by adjusting the angles of the bionic eyeballs of the robot and the other joint parts of the robot platform, or is less than a certain threshold value M, it is thus determined that the focus tracking is completed.

[0045] More preferably, the threshold value M is H / 20.

[0046] More preferably, in the robot platform, two bionic eyeballs of the robot are rotationally provided, and each bionic eyeball of the robot is controlled by two servos to rotate in the horizontal and vertical directions respectively. The mechanical limit angles of rotation in the horizontal and vertical directions of each bionic eyeball of the robot are both ±25° - 35°, the lowest rotation speed is 35° / 0.2s, and the distance between the two bionic eyeballs is 62 - 64mm.

[0047] More preferably, the two bionic eyeballs of the robot rotate synchronously in the horizontal and vertical directions, and each bionic eyeball of the robot is internally provided with a camera.

[0048] The beneficial effects of the present invention are as follows: By analyzing the position coordinates of the key parts of the human face and the distance between the center of the visual field of the image data collection, the moving distance and direction of the tracking focus are obtained in real - time, which can greatly improve the tracking focus efficiency of the bionic eyeballs of the robot, achieve real - time tracking, thus avoiding the phenomenon of delay and lag, and adjusting its tracking rotation speed according to the offset distance, making the eye contact of the bionic robot more natural; and it has a focus fixation strategy for eye interaction with people, randomly and freely switching in the triangular area of eye gaze, conforming to official business and social gaze etiquette, and creating a good interaction atmosphere. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1This is a flowchart of the entire invention.

[0050] Figure 2 This is a schematic diagram of the hardware and software collaboration architecture of the present invention. Detailed Implementation

[0051] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0052] like Figure 1 and Figure 2 The diagram illustrates a focusing and tracking method for a robot's bionic eyeball. It comprises a robot platform, a rotating bionic eyeball within the robot platform, an eyeball camera image acquisition module embedded in the bionic eyeball, and a processor control module. The processor control module is an STM32 processor control module, typically located within the robot platform, but can also be a remote processor used to control the robot platform. Two rotating bionic eyeballs are positioned within the robot platform. Each bionic eyeball can be independently controlled by two servos to rotate horizontally and vertically, but synchronized rotation in both directions is preferable. Each bionic eyeball has a built-in camera, and these cameras work together to calculate the rotation. The mechanical limits for horizontal and vertical rotation of each bionic eyeball are ±25° to 35°, with a minimum rotation speed of 35° / 0.2s. The distance between the two bionic eyeballs is 62-64mm. The specific method described is based on the publication number CN118721236. A describes a robot eye simulation drive component with the best related robot bionic eye structure, which has higher rotation speed and degree of freedom, higher flexibility and more natural bionic eye, and better performance.

[0053] Its focused tracking method includes the following steps:

[0054] Step 1: Use a flexible rotating robotic bionic eyeball to collect image data in the initial posture; the flexible rotating robotic bionic eyeball can ensure the vividness of the mechanical structure of eye interaction, so as not to cause the dull eye gaze due to insufficient execution efficiency caused by the lack of mechanical structure; the initial posture is generally an upright position with eyes looking straight ahead, other postures are also possible, but this is more in line with the natural conventional form of the human body.

[0055] Step 2: Use Media Pipe's Holistic model (Media Pipe is a data stream processing machine learning application development framework developed and open-sourced by Google, and the Holistic model is a multimodal human perception model that can simultaneously detect and analyze human facial features, hand movements, and body posture) to identify key facial features in the collected image data and obtain data sets of edge plotting points for the left eye, right eye, and mouth respectively.

[0056] Step 3: Calculate the center point coordinates of the left eye, right eye, and mouth based on different edge point data sets, and set them as the corresponding target points for the left eye, right eye, and mouth. Randomly select a point within the face range on the symmetrical center line between the center points of the left and right eyes as the inter-eye target point, i.e., on a line of the nose area, preferably not exceeding the nose area, which better matches the triangular area of ​​eye gaze (the area formed by the left and right eyes and the inter-eye target point). Continuously collect image data and cyclically update the position coordinates of key facial features. That is, different edge point data sets are obtained based on different image data collected each time, thereby realizing the cyclical update of the position coordinates of key facial features, completing uninterrupted real-time recognition, and continuously executing the subsequent focusing and tracking strategies.

[0057] The specific algorithm for calculating the center point coordinates of the left eye, right eye, and mouth based on different edge plotting point location data sets is as follows:

[0058] Suppose that the obtained edge plotting point position data set is

[0059] The general equation for constructing a circle using these edge points as centers is: (xa) 2 +(yb) 2 =r 2 ;

[0060] Then, the objective function is obtained:

[0061] We Take the partial derivatives with respect to a, b, and r, and set these partial derivatives to 0 to find the extreme points:

[0062]

[0063] By solving the system of equations with partial derivatives equal to 0, we can obtain the values ​​of a, b, and r, and thus determine the center of the circle. The coordinates of the center point of the left eye, right eye, or mouth are the obtained center of the circle (a, b).

[0064] Step 4: Determine whether the face target has moved based on the position coordinates of the key facial features and the distance between the center of the field of view of the image data acquisition.

[0065] The specific methods for determining whether the face target moves and the corresponding tracking or focusing rotation methods are as follows:

[0066] Let the target point of the left eye be (x l , y l ), the target point of the right eye be (x r , y r ), the target point between the eyes be (x c , y c ), the target point of the mouth be (x m , y m ), the center of the field of view / the center point of the picture be (x o , y o ), and the length, width or diagonal length of the picture be H;

[0067] From the Euclidean distance formula the distance between two points i and j can be calculated, and from this, the distances between the left eye, right eye, mouth, and the target point between the eyes and the center point of the picture can be calculated

[0068] If the robot bionic eyeball selects to fixate on the left eye, right eye, mouth or the target point between the eyes, then the corresponding or is assigned to D;

[0069] When D < H / 5, only the robot bionic eyeball rotates to achieve focus transfer and tracking, and the movement amplitude of the eye servo per frame is D / 2H;

[0070] When D > H / 5, the movement amplitude of the robot bionic eyeball servo per frame is D / 2H, while the other joint parts of the robot platform move 0.1 seconds later relative to the robot bionic eyeball, and the movement amplitude of the other joint parts of the robot platform is D / 2H;

[0071] When, by adjusting the angles of the robot bionic eyeball and the other joint parts of the robot platform, or is less than a certain threshold M, it is determined that the focusing and tracking are completed. The threshold M is determined according to the clarity of the images that can be captured by the camera built into the robot bionic eyeball. Because there is uncertainty in the granularity of the image pixel points, setting the threshold M can avoid jitter when focusing on the target. Currently, the threshold M is generally optimal at H / 20.

[0072] Step 5: If the face target is determined to be stationary, focus is applied using a gaze strategy, causing the robot's bionic eyes to naturally rotate and gaze between the left eye, right eye, and interocular target point; if dialogue occurs (the judgment can be based on dialogue actions produced by the face target's mouth, or dialogue effects produced by other auditory modules), then the robot's bionic eyes will naturally rotate and gaze between the mouth, left eye, right eye, and interocular target point.

[0073] The above-mentioned eye contact strategy is as follows:

[0074] Step 5.1: In the initial state, the robot's bionic eye focuses on the target point between the eyes;

[0075] Step 5.2: After a random time interval of 2 to 5 seconds, the robot's bionic eye randomly selects the target point of the left or right eye for focusing;

[0076] Step 5.3: Perform 1 to 3 switching of target focus between the left and right eyes, with the number of switching times in each round being random and the switching period being a random time of 0.5 to 2 seconds;

[0077] Step 5.4: Then, the robot's bionic eyeball returns to focusing on the interocular target position;

[0078] Step 5.5: Repeat Step 5.2 to Step 5.4.

[0079] The system determines whether to initiate a dialogue by generating dialogue actions through the mouth of the facial target or by receiving dialogue information through other auditory modules.

[0080] Step 5.6: If a dialogue is determined to occur, the robot's bionic eyeballs will rotate and focus on the target point of the mouth for a random period of 1 to 3 seconds, and maintain this focus for a random period of 0.5 to 2 seconds, before returning to focus on the target point between the eyes.

[0081] Step 5.7: Then, repeat Step 5.2 to Step 5.4 and Step 5.6 until it is determined that no new dialogue has been generated.

[0082] Step 6: If the movement of the face target is determined, a tracking strategy is adopted to control the robot's bionic eyeball and / or other joints of the robot platform to rotate according to the movement distance, so that the robot's bionic eyeball tracks the face target position with a natural scanning amplitude.

[0083] The above tracking strategies, based on the size of its movement range, include the following response strategies:

[0084] Step 6.1: If it is determined that any target point is offset outward from the center of the field of view by no more than 20% of the field of view, small-range tracking is adopted. Focus shift and tracking can be achieved simply by rotating the eyeball. This method is used in combination with the gaze strategy to achieve a more natural posture.

[0085] Step 6.2: If any target point is determined to have shifted outward from the center of the field of view by more than 20% of the field of view, large-scale tracking is adopted. This requires the robot's bionic eyeball and the robot platform's neck joint to rotate together to complete the transfer and tracking of the field of view center. During this process, the rotation of the robot platform's neck joint will lag behind the rotation of the robot's bionic eyeball by 0.1 seconds. After the field of view center focuses and locks onto the target point of the key part of the face, the robot's bionic eyeball is adjusted to maintain a neutral position, that is, the position of the robot's bionic eyeball is adjusted to look straight ahead, and the robot platform's neck joint maintains the angle of tracking and focusing on the target point.

[0086] Step 6.3: If it is determined that any target point has deviated from the dynamic field of view (i.e., the face target has been lost within the image data acquisition range of the robot's bionic eye), then quickly switch the target point and use over-range tracking. The robot's body pose is rotated through the body joints such as the waist joint or leg joint of the robot platform, thereby further expanding the dynamic field of view.

[0087] Step 6.4: If the target point is tracked within the extended dynamic field of view, return to Step 6.1 to complete the focus shift and tracking at the center of the field of view; at the same time, the robot platform can also remind the face target to return to the robot platform's facing position, i.e., the initial posture position, through voice prompts.

[0088] Step 6.5: If the target point still cannot be tracked within the extended dynamic field of view, a blinking action is performed, simultaneously returning the robot platform and the robot's bionic eyeball to their initial posture, thus enabling the next facial image recognition and acquisition. Theoretically, processing the same image data in real time would be very fast, so returning to the cyclically updated image data after completing Step 5 or Step 6 would not cause any lag. However, if the software control system's computing power is too weak, pre-cyclically updated image data can also be used.

[0089] The above-mentioned scheme analyzes the position coordinates of key facial features and the distance from the center of the field of view of image data acquisition to obtain the real-time tracking and focusing distance and direction. This greatly improves the tracking and focusing efficiency of the robot's bionic eyeball, enabling real-time tracking and avoiding delays and stuttering. It also adjusts the tracking rotation speed according to the offset distance, making the bionic robot's eye communication more natural. Furthermore, it has a focusing gaze strategy that interacts with human eyes, randomly and freely switching between the triangular area of ​​eye gaze (the area composed of the left and right eyes and the target point between the eyes), which conforms to official and social gaze etiquette and creates a good interactive atmosphere.

[0090] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims.

Claims

1. A focusing and tracking method for a robot's bionic eyeball, based on a robot platform, a robot bionic eyeball rotating within the robot platform, an eyeball camera image acquisition module within the bionic eyeball, and a processor control module, wherein the processor control module controls the robot platform, characterized in that, Includes the following steps: Step 1: Use the robot's bionic eyeball to collect image data in its initial posture; Step 2: Use Media Pipe's Holistic model to identify key facial features in the acquired image data and obtain data sets of edge plotting points for the left eye, right eye, and mouth. Step 3: Calculate the center point coordinates of the left eye, right eye and mouth respectively based on different edge plotting point position data groups, and set them as the corresponding target points of the left eye, right eye and mouth. Randomly select a point within the face range on the symmetrical center line of the center point of the left eye and right eye as the inter-eye target point, and continuously collect image data to cyclically update the position coordinates of key parts of the face. Step 4: Determine whether the face target has moved based on the distance between the position coordinates of the key facial features and the center of the field of view of the image data acquisition. Step 5: If the target face is determined to be stationary, focus is achieved using a gaze strategy, causing the robot's bionic eyes to move naturally between the left eye, right eye, and the interocular target point; if a dialogue occurs, the robot's bionic eyes will move naturally between the mouth, left eye, right eye, and the interocular target point. Step 6: If the movement of the face target is determined, a tracking strategy is adopted to control the robot's bionic eyeball and / or other joints of the robot platform to rotate according to the movement distance, so that the robot's bionic eyeball tracks the face target position with a natural scanning amplitude.

2. The focusing and tracking method for a robot's bionic eyeball according to claim 1, characterized in that, In Step 5, the eye contact strategy specifically includes: Step 5.1: In the initial state, the robot's bionic eye focuses on the target point between the eyes; Step 5.2: After a random time interval of 2 to 5 seconds, the robot's bionic eye randomly selects the target point of the left or right eye for focusing; Step 5.3: Perform 1 to 3 switching of target focus between the left and right eyes, with the number of switching times in each round being random and the switching period being a random time of 0.5 to 2 seconds; Step 5.4: Then, the robot's bionic eyeball returns to focusing on the interocular target position; Step 5.5: Repeat Step 5.2 to Step 5.

4.

3. The focusing and tracking method for a robot's bionic eyeball according to claim 2, characterized in that, The system determines whether to initiate a dialogue by generating dialogue actions through the mouth of the facial target or by receiving dialogue information through other auditory modules. Step 5.6: If a dialogue is determined to occur, the robot's bionic eyeballs will rotate and focus on the target point of the mouth for a random period of 1 to 3 seconds, and maintain this focus for a random period of 0.5 to 2 seconds, before returning to focus on the target point between the eyes. Step 5.7: Then, repeat Step 5.2 to Step 5.4 and Step 5.6 until it is determined that no new dialogue has been generated.

4. The focusing tracking method for a robot's bionic eyeball according to any one of claims 1-3, characterized in that, In Step 6, the tracking strategy, based on the size of its movement range, includes the following response strategies: Step 6.1: If it is determined that any target point is offset outward from the center of the field of view by no more than 20% of the field of view, small-range tracking is adopted, which only requires eye movement to achieve focus shift and tracking; Step 6.2: If it is determined that any target point deviates from the center of the field of view by more than 20% of the field of view, large-range tracking is adopted, and the robot bionic eyeball and the neck joint of the robot platform need to rotate jointly to complete the transfer and tracking of the center of the field of view; during this process, the rotation of the neck joint of the robot platform will be delayed by 0.1 seconds compared to the rotation of the robot bionic eyeball. After the center of the field of view finally focuses and locks on the key part target of the human face, adjust the robot bionic eyeball to maintain a neutral position, and the neck joint of the robot platform maintains the angle of tracking and focusing on the target point.

5. The focusing tracking method for a robot's bionic eyeball according to claim 4, characterized in that, In Step 6, the tracking strategy also includes the following coping strategies: Step 6.3: If it is determined that any target point has deviated from the dynamic field of view, quickly switch the target point and adopt out-of-range tracking. Rotate the body pose of the robot through body joints such as the waist joint or leg joint of the robot platform to further expand the dynamic field of view. Step 6.4: If the target point is tracked within the expanded dynamic field of view, return to execute Step 6.1 to complete the focus transfer and tracking of the center of the field of view; Step 6.5: If the target point still cannot be tracked within the expanded dynamic field of view, perform a blinking action, and at the same time, make the robot platform and the robot bionic eyeball return to the initial pose.

6. The focusing tracking method for a robot's bionic eyeball according to any one of claims 1-3 or 5, characterized in that, In Step 3, the specific algorithms for calculating the center point position coordinates of the left eye, right eye, and mouth respectively based on different edge point position data groups are as follows: Suppose that the obtained edge plotting point position data set is The general equation for constructing a circle with these edge points as its centers is: (xa) 2 +(yb) 2 =r 2 ; Then, the objective function is obtained: We Take the partial derivatives with respect to a, b, and r, and set these partial derivatives to 0 to find the extreme points: By solving the system of equations where the above partial derivatives are equal to 0, we can obtain the values of a, b, and r, and then determine the center of the circle. Thus, the center point position coordinates of the left eye, right eye, or mouth are the obtained center of the circle (a, b).

7. The focusing tracking method for a robot's bionic eyeball according to claim 6, characterized in that... , In Step 4, the specific method for determining whether the human face target moves is as follows: Let the target point of the left eye be (x l ,y l The target point for the right eye is (x). r ,y r The target point in the eye is (x) c ,y c The target point of the mouth is (x) m ,y m The center point of the image is (x). o ,y o The length, width, or diagonal length of the image is H; From the Euclidean distance formula The distance between points i and j can be calculated, and from this, the distances between the left eye, right eye, mouth, and the interocular target point and the center point of the image can be calculated. If the robot's bionic eye selects to focus on the left eye, right eye, mouth, or interocular target, then the corresponding... or Assign the value to D; When D < H / 5, only the robot bionic eyeball rotates to achieve focus transfer and tracking, and the movement amplitude of the eye servo per frame is D / 2H; When D > H / 5, the movement amplitude of the robot bionic eyeball servo per frame is D / 2H, while the other joint parts of the robot platform move 0.1 seconds later relative to the robot bionic eyeball, and the movement amplitude of the other joint parts of the robot platform is D / 2H; By adjusting the angles of the robot's bionic eyeballs and other joints on the robot platform, thus... or When the value is less than a certain threshold M, the focusing tracking is considered complete.

8. The focusing tracking method for a robot's bionic eyeball according to claim 7, characterized in that... , The threshold M is H / 20.

9. The focusing tracking method for a robotic bionic eyeball according to any one of claims 1-3, 5, 7 or 8, characterized in that... , There are two robot bionic eyeballs rotatably arranged in the robot platform. The mechanical limit angles of each robot bionic eyeball in the horizontal and vertical directions are both ±25° - 35°, the lowest rotation speed is 35° / 0.2s, and the distance between the two robot bionic eyeballs is 62 - 64mm.

10. The focusing tracking method for a robot's bionic eyeball according to claim 9, characterized in that... , The two robot bionic eyeballs rotate synchronously in the horizontal and vertical directions, and each robot bionic eyeball is equipped with a camera inside.

Images