Robot holder control method and robot

Abstract

The invention discloses a robot holder control method and a robot, and the method comprises the steps: determining a first relative position relation between a to-be-detected target and a robot holder based on the image collection parameters of the robot holder under the condition that the robot holder is located at a first position; determining a target coordinate of the to-be-detected target based on the first relative position relation; under the condition that the robot holder is located at the second position, control parameters of the robot holder are determined based on the pose information of the robot holder and the target coordinates of the to-be-detected target; and based on the control parameters, controlling the robot holder to collect an image of the to-be-detected target.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and more specifically to a robot pan-tilt control method and a robot. Background Art

[0002] With the development of intelligent technology, robots can perform more and more tasks. For example, robots can replace inspectors to perform inspections. When performing inspection tasks, robots generally use pan-tilt cameras to shoot the inspected targets.

[0003] Before the robot inspects, it needs to determine a deployment point and the corresponding pan-tilt information of the robot pan-tilt. After the inspection begins, the robot reaches the preset position through the navigation algorithm and controls the pan-tilt according to the preset pan-tilt information. However, due to the error in the robot's navigation and positioning, the robot cannot accurately reach the preset position. If the pan-tilt information is used, the pan-tilt cannot accurately capture the detected target, reducing the shooting accuracy. Summary of the invention

[0004] In view of this, this application provides the following technical solutions:

[0005] A robot pan-tilt control method, comprising:

[0006] When the robot gimbal is in a first position, determining a first relative position relationship between the target to be detected and the robot gimbal based on image acquisition parameters of the robot gimbal;

[0007] Based on the first relative position relationship, determining the target coordinates of the target to be detected;

[0008] When the robot gimbal is in the second position, determining control parameters of the robot gimbal based on the posture information of the robot gimbal and the target coordinates of the target to be detected;

[0009] Based on the control parameters, the robot gimbal is controlled to capture an image of the target to be detected.

[0010] Optionally, when the robot gimbal is in the first position, determining a first relative position relationship between the target to be detected and the robot gimbal based on an image acquisition parameter of the robot gimbal includes:

[0011] When the robot gimbal is in a first position, determining a first relative distance between the target to be detected and the robot gimbal based on an image acquisition parameter of the robot gimbal;

[0012] A first relative position relationship between the target to be detected and the robot gimbal is determined according to the first relative distance.

[0013] Optionally, determining a first relative distance between the target to be detected and the robot gimbal based on an image acquisition parameter of the robot gimbal includes:

[0014] Processing the image acquisition parameters of the robot gimbal based on the target processing model to obtain a first relative distance between the target to be detected and the robot gimbal, wherein the image acquisition parameters include a focus parameter and a zoom parameter;

[0015] The target processing model is obtained by training a target data set, wherein the target data set includes a plurality of data samples, and each data sample includes relevant information between the current image acquisition parameters of the robot gimbal and the distance of the acquired target.

[0016] Optionally, when the robot gimbal is in the first position, determining a first relative distance between the target to be detected and the robot gimbal based on an image acquisition parameter of the robot gimbal comprises:

[0017] In response to the robot gimbal detecting that the target to be detected is in the target area of ​​the acquisition field of view when the robot gimbal is at the first position, obtaining current image acquisition parameters of the robot gimbal when the focus processing is completed;

[0018] Based on the current image acquisition parameters, a first relative distance between the target to be detected and the robot gimbal is determined.

[0019] Optionally, determining a first relative position relationship between the target to be detected and the robot gimbal according to the first relative distance includes:

[0020] Determine a first direction parameter of the target to be detected relative to the robot gimbal according to the first relative distance;

[0021] Obtaining a second direction parameter and a third direction parameter corresponding to the target area, wherein the first direction parameter, the second direction parameter and the third direction parameter correspond to different directions respectively;

[0022] The first relative position relationship is determined according to the first direction parameter, the second direction parameter and the third direction parameter.

[0023] Optionally, the first relative position relationship is a first coordinate of the target to be detected in a gimbal optical center coordinate system corresponding to the robot gimbal; and determining the target coordinate of the target to be detected based on the first relative position relationship includes:

[0024] Determine a second coordinate of the target to be detected in the robot coordinate system according to the relative posture of the robot gimbal with respect to the robot body and the first coordinate in the first position;

[0025] The target coordinates are determined according to the robot posture of the robot body in the scene coordinate system and the second coordinates in the case of the first position.

[0026] Optionally, when the robot gimbal is in the second position, determining the control parameters of the robot gimbal based on the posture information of the robot gimbal and the target coordinates of the target to be detected includes:

[0027] Determine the fourth coordinate of the target to be detected in the optical center coordinate of the gimbal according to the third coordinate of the robot body in the scene coordinate system, the target coordinate of the target to be detected, and the corresponding relationship between the scene coordinate system and the gimbal optical center coordinate system in the case of the second position;

[0028] According to the fourth coordinate, control parameters of the robot gimbal are determined.

[0029] Optionally, determining the control parameters of the robot gimbal according to the fourth coordinate includes:

[0030] Determine a rotation angle parameter of the robot gimbal based on the coordinate sub-parameters of each coordinate dimension corresponding to the fourth coordinate, wherein the rotation angle parameter includes at least a horizontal rotation angle and a vertical rotation angle;

[0031] Based on the rotation angle parameter, a control parameter is determined.

[0032] Optionally, it also includes:

[0033] When the gimbal type feature of the robot gimbal is updated, the target processing model is adjusted based on the gimbal type feature, so that the image acquisition parameters of the robot gimbal are processed based on the adjusted target processing model to obtain a first relative distance between the target to be detected and the robot gimbal.

[0034] A robot, comprising:

[0035] Robot body;

[0036] A controller and a pan-tilt platform, wherein the pan-tilt platform is provided with an image acquisition module, and the image acquisition module is used to acquire an image of a target to be detected;

[0037] The controller is used to:

[0038] When the robot gimbal is in a first position, determining a first relative position relationship between the target to be detected and the robot gimbal based on image acquisition parameters of the robot gimbal;

[0039] Based on the first relative position relationship, determining the target coordinates of the target to be detected;

[0040] When the robot gimbal is in the second position, determining control parameters of the robot gimbal based on the posture information of the robot gimbal and the target coordinates of the target to be detected;

[0041] Based on the control parameters, the robot gimbal is controlled to capture an image of the target to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0043] Figure 1 A schematic diagram of a flow chart of a robot pan-tilt control method provided in an embodiment of the present application;

[0044] Figure 2 A schematic diagram of a scene of a robot gimbal and a target to be detected provided in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of a scene of a robot pan / tilt platform applied to inspection tasks provided in an embodiment of the present application;

[0046] Figure 4 A schematic diagram of the structural composition of a robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] The terms "first" and "second" in this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may include steps or units that are not listed.

[0049] The embodiment of the present application provides a robot pan-tilt control method, through which the control parameters of the robot pan-tilt at a corresponding position are determined, and the robot pan-tilt is controlled based on the control parameters to collect images of the target to be detected, thereby improving the image collection accuracy of the robot pan-tilt. Figure 1 , is a flow chart of a robot gimbal control method provided in an embodiment of the present application, the method may include the following steps:

[0050] S101. When the robot gimbal is in a first position, determine a first relative position relationship between the target to be detected and the robot gimbal based on image acquisition parameters of the robot gimbal.

[0051] A robot gimbal is a supporting device for installing and fixing movable parts, and is usually used to carry equipment such as cameras, sensors or transmitting devices, so that they can flexibly adjust their angles and positions in space. In an embodiment of the present application, the robot gimbal includes an image acquisition module, which is used to acquire images of corresponding targets. Further, the image acquisition module can be a camera, such as a monocular camera, a binocular camera or a depth camera. When the robot gimbal is in the first position, it can be the initial position where the robot gimbal performs the corresponding image acquisition task, or it can be a designated position where the robot gimbal is placed. For example, the robot gimbal is used in an inspection scenario, and the first position can be the starting position for the robot gimbal to perform the inspection task.

[0052] The image acquisition parameters of the robot gimbal characterize the image acquisition performance parameters of the gimbal's image acquisition module (such as a gimbal camera). Taking the gimbal camera as an example, the image acquisition parameters may include the rotation parameters of the gimbal camera, the zoom capability parameters, i.e., the variable range of the lens focal length, etc. When the robot gimbal is in the first position, it can acquire an image of the target to be detected based on its current image acquisition parameters, and then determine the first relative position relationship between the target to be detected and the robot gimbal. Therefore, the position of the target to be detected can be determined in subsequent processing based on the first relative position relationship.

[0053] S102: Determine the target coordinates of the target to be detected based on the first relative position relationship.

[0054] After determining the first relative position relationship between the target to be detected and the robot gimbal, the coordinates of the target to be detected in the current scene can be calculated based on the current posture of the robot, the rotation angle parameters of the gimbal, etc., and recorded as the target coordinates. This can accurately obtain the spatial position information of the target to be detected, making it easier to collect images of the target to be detected later.

[0055] S103: When the robot gimbal is in the second position, determine the control parameters of the robot gimbal based on the posture information of the robot gimbal and the target coordinates of the target to be detected.

[0056] S104: Based on the control parameters, control the robot gimbal to collect images of the target to be detected.

[0057] When the robot gimbal moves to the second position, the control parameters of the robot gimbal can be determined according to the corresponding posture information when the robot gimbal is in the second position and the target coordinates of the target to be detected obtained by previous calculation. That is, if the relevant image acquisition parameters of the robot gimbal are adjusted based on the control parameters, the robot gimbal can acquire a high-precision image of the target to be detected at the current second position.

[0058] In the embodiment of the present application, there is no need to predetermine the control parameters of the robot gimbal at each position point, which can avoid the problem of poor image acquisition accuracy caused by the deviation between the position of the robot at the corresponding position point and the position in the pre-point collection stage due to the robot movement error. In the embodiment of the present application, it is only necessary to determine the first relative position relationship between the target to be detected and the robot gimbal when the robot gimbal is in the first position, and there is no need to record additional information in the point collection stage (i.e., the stage of pre-controlling the robot gimbal to move along a specified path), thereby improving the application deployment efficiency of the robot gimbal. In the embodiment of the present application, the target coordinates of the target to be detected are determined based on the correlation relationship between the image acquisition parameters and the first relative position relationship between the target to be detected and the robot gimbal, without the need for a complex data processing model, nor the need to frequently perform feature matching operations between the actual features of the target to be detected and the shooting features, making the actual operation simpler and improving the application efficiency of the robot gimbal.

[0059] The following describes the relevant contents of the robot gimbal control method in the embodiment of the present application in combination with specific application scenarios.

[0060] In the embodiment of the present application, the first relative position relationship between the target to be detected and the robot gimbal can be determined based on the relative distance between the two. That is, in one embodiment, when the robot gimbal is in the first position, the first relative position relationship between the target to be detected and the robot gimbal is determined based on the image acquisition parameters of the robot gimbal, including:

[0061] When the robot pan-tilt platform is in the first position, a first relative distance between the target to be detected and the robot pan-tilt platform is determined based on the image acquisition parameters of the robot pan-tilt platform; and a first relative position relationship between the target to be detected and the robot pan-tilt platform is determined according to the first relative distance.

[0062] See also Figure 2 , a schematic diagram of a scene of a robot gimbal and a target to be detected provided in an embodiment of the present application. The first relative distance between the target to be detected and the robot gimbal can be determined according to the image acquisition parameters of the robot gimbal, the actual features of the target to be detected, and the image features of the target to be detected in the image taken by the robot gimbal. Then, according to the relative direction, angle and other information of the target to be detected and the robot gimbal in the same coordinate system and the first relative distance, the first relative position relationship between the two is determined. For example, the image acquisition parameters include the focal length of the gimbal camera, and the first relative distance between the target to be detected and the robot gimbal can be calculated according to the actual width or height of the target to be detected and the imaging width or height of the target to be detected in the image taken by the gimbal camera. For another example, the image acquisition parameters include the initial focal length and zoom factor of the gimbal camera, and the first relative distance can be determined according to the initial focal length, zoom factor, the actual width of the target to be detected and the imaging width of the target to be detected.

[0063] Furthermore, in order to quickly determine the relative distance between the target to be detected and the robot gimbal, the relative distance can be obtained through the mapping relationship between the image acquisition parameters and the relative distance and the current image acquisition parameters. For example, the mapping relationship can be calculated through test data or existing recorded data. In order to improve data processing efficiency and accuracy, the first relative distance between the target to be detected and the robot gimbal can be determined through a neural network model. Specifically, the image acquisition parameters of the robot gimbal are processed based on the target processing model to obtain the first relative distance between the target to be detected and the robot gimbal.

[0064] Among them, the image acquisition parameters include focus parameters and zoom parameters. The target processing model is obtained by training the target data set, which includes multiple data samples, and each data sample includes the relevant information between the current image acquisition parameters of the robot gimbal and the distance of the acquired target. Specifically, an initial neural network model can be constructed, and the initial neural network model can be trained with the target data set until the loss function of the neural network model converges. The neural network model at the end of the training is used as the target processing model, and the loss function characterizes the difference between the predicted distance predicted by the neural network model and the distance marked in the data sample.

[0065] For example, the image acquisition parameters corresponding to the robot gimbal focusing on target objects at different working distances at different magnifications can be collected to form a data set, where zoom represents the zoom parameter of the robot gimbal; focus represents the focus parameter of the robot gimbal, which can include the focal length parameter, and the focal length determines the viewing angle and magnification of the camera. The longer the focal length, the narrower the viewing angle and the greater the magnification; the shorter the focal length, the wider the viewing angle and the smaller the magnification; wd represents the working distance, that is, the distance between the robot gimbal and the target object to be photographed. At this time, the data set refers to the data set formed by zoom, focus, and wd, and then a neural network with a fully connected layer is constructed, with the input being the data values ​​corresponding to zoom and focus, and the output being wd. The neural network is trained using the above data set to obtain the target processing model, i.e., the basic neural network.

[0066] Furthermore, in order to improve the application efficiency and scope of application of the target processing model, the target processing model can also be fine-tuned in the embodiment of the present application to adapt to the current application scenario. Specifically, when the type feature of the robot gimbal is updated, the target processing model is adjusted based on the gimbal type feature, so that the image acquisition parameters of the robot gimbal are processed based on the adjusted target processing model to obtain the first relative distance between the target to be detected and the robot gimbal. For example, the target processing model is used as the basic neural network, and then for different gimbal individuals of the same model, only a small amount of data needs to be collected as a fine-tuning data set according to the type features corresponding to the gimbal individuals, and then the target processing model is fine-tuned using the fine-tuning data set to obtain the adjusted target processing model, so as to be applied to the current image acquisition scenario. This can overcome the individual differences of the gimbals and facilitate the rapid acquisition of the relative distance between the robot gimbal and the target.

[0067] In one implementation of the embodiments of the present application, when the robot gimbal is in a first position, a first relative distance between the target to be detected and the robot gimbal is determined based on the image acquisition parameters of the robot gimbal, including: in response to the robot gimbal detecting that the target to be detected is in a target area of ​​the acquisition field of view when the robot gimbal is in the first position, after completing the focusing process, obtaining the current image acquisition parameters of the robot gimbal; based on the current image acquisition parameters, determining the first relative distance between the target to be detected and the robot gimbal.

[0068] In this embodiment, when the robot gimbal is in the first position, it is detected that the target to be detected is in the target area of ​​the acquisition field of view, indicating that the current posture of the robot gimbal can capture the target to be detected. In an optional embodiment, the target area is the central area of ​​the acquisition field of view. In order to enable it to clearly obtain the image of the target to be detected. Focus processing can be performed first. Focus (FOCUS) is to adjust the distance between the lens and the sensor so that the target forms a clear image on the image sensor. If it is not focused, the image may be blurred, resulting in loss of details. After the focus is completed, the focal length (FOCUS value) and zoom factor (ZOOM value) of the camera have been adjusted to the optimal state. The image acquisition parameters (such as field of view angle, imaging size, etc.) obtained at this time can more accurately reflect the actual shooting conditions. In this way, the first relative distance between the robot gimbal and the target to be detected can be determined more accurately when the gimbal captures a clearer image.

[0069] After obtaining the first relative distance between the target to be detected and the robot gimbal, the direction parameter between the two can also be determined, thereby determining the first relative position relationship between the two. In one embodiment, determining the first relative position relationship between the target to be detected and the robot gimbal according to the first relative distance includes:

[0070] Determine a first direction parameter of the target to be detected relative to the robot gimbal according to the first relative distance; obtain a second direction parameter and a third direction parameter corresponding to the target area; and determine a first relative position relationship according to the first direction parameter, the second direction parameter, and the third direction parameter. The first relative position relationship includes the coordinates of the target to be detected in the gimbal optical center coordinate system corresponding to the robot gimbal.

[0071] Among them, the first direction parameter, the second direction parameter and the third direction parameter correspond to different directions respectively. The first direction parameter is the coordinate value of the target to be detected in the z-axis direction in the optical center coordinate system of the robot gimbal corresponding to the gimbal. Since the target to be detected is located in the central area of ​​the acquisition field of view, as long as it is located in the central area, its x-axis direction coordinate and y-axis direction coordinate in the optical center coordinate system of the gimbal are fixed. Therefore, the second direction parameter can be the coordinate value of the x-axis direction corresponding to the preset central area of ​​the acquisition field of view, and the third direction parameter can be the coordinate value of the y-axis direction corresponding to the preset central area of ​​the acquisition field of view.

[0072] Through the above implementation, as long as the robot gimbal posture is adjusted so that the target to be detected is located in the central area of ​​the acquisition field of view, only the first relative distance can be calculated, and then the first relative position relationship can be directly obtained by using the preset second direction parameters and third direction parameters.

[0073] In an embodiment of the present application, a method for determining the target coordinates of a target to be detected is also provided, in which the first relative position relationship is the first coordinate of the target to be detected in the gimbal optical center coordinate system corresponding to the robot gimbal, wherein the target coordinates of the target to be detected are determined based on the first relative position relationship, including: determining the second coordinate of the target to be detected in the robot coordinate system according to the relative posture and the first coordinate of the robot gimbal relative to the robot body in the first position; determining the target coordinates according to the robot posture and the second coordinate of the robot body in the scene coordinate system in the first position.

[0074] In the embodiment of the present application, the scene coordinate system is a map coordinate system, which is a fixed world coordinate system used to represent the global reference frame of the entire environment. The scene coordinate system is fixed and does not change with the movement of the robot or the gimbal. The robot coordinate system usually takes the center of mass or base of the robot as the origin, and characterizes the local coordinates of the robot body. This coordinate system moves with the movement of the robot and is used to describe the position and posture of the robot itself. The gimbal optical center coordinate system takes the optical center of the gimbal camera as the circle point, and is usually defined as the Z axis pointing to the optical axis direction of the camera, and the X axis and Y axis pointing to the horizontal and vertical directions of the camera respectively. This coordinate system is used to describe the position of the target detected by the gimbal camera in the camera's field of view.

[0075] Specifically, when the robot gimbal is in the first position, the coordinates of the robot in the scene coordinates can be obtained as p (x0, y0, z0). The current image acquisition parameters of the gimbal camera are obtained, including the rotation angle parameters and focal length and zoom parameters of the gimbal camera. For example, the image acquisition parameters include the horizontal rotation parameter pan0, the vertical rotation parameter tilt0, the zoom parameter zoom0, the focal length parameter focus0, etc. At this time, zoom0 and focus0 can be used as the input of the target processing model to obtain the working distance wd between the target to be detected and the robot gimbal (specifically, it can be the optical center of the robot gimbal camera), that is, the first relative distance wd is obtained.

[0076] According to the horizontal rotation parameter pan0 of the gimbal camera, the vertical rotation parameter tilt0 and the first relative distance wd, the first coordinate of the target to be detected in the gimbal optical center coordinate system can be obtained. At this time, the first relative position relationship can be expressed by the first coordinate, such as expressing the first coordinate as p (xcd, ycd, zcd). Then, according to the relative posture of the robot gimbal relative to the robot body and the first coordinate at the first position, the second coordinate of the target to be detected in the robot coordinate system is determined. According to the configuration data in the robot URDF model, the relative posture of the robot gimbal relative to the robot body can be obtained, and the conversion relationship between the robot coordinate system and the gimbal optical center coordinate system can be further determined, such as represented by MTR0. According to the first coordinate of the target to be detected in the gimbal optical center coordinate system and the conversion relationship between the robot coordinate system and the gimbal coordinate system corresponding to the relative posture, the second coordinate of the target to be detected in the robot coordinate system is determined, such as expressed as p (xrd, yrd, zrd). Among them, the robot URDF model is used to describe the structure, joints, degrees of freedom and other information of the robot machine part. The description information of the robot body and the physical properties of the relevant parts of the robot (such as inertia matrix, shape, etc.) can be determined based on the robot URDF model. Based on this information, the position and rotation information of the gimbal installed on the robot body can be determined, thereby determining the relative position and posture of the robot gimbal relative to the robot body.

[0077] After obtaining the second coordinate of the target to be detected in the robot coordinate system, the target coordinate is determined according to the robot posture and the second coordinate of the robot body in the scene coordinate system. Among them, the robot posture of the robot body in the scene coordinate system mainly includes the position coordinates of the robot body in the scene coordinate system, that is, p (x0, y0, z0), and the posture of the robot body in the scene coordinate system. The target coordinates of the target to be detected are the coordinates of the target to be detected in the scene coordinate system, such as p (xd, yd, zd). In this way, the coordinates of the target to be detected in the scene coordinate system can be accurately obtained, so that the control parameters of the robot gimbal can be adjusted according to the coordinates to accurately capture the image of the target to be detected.

[0078] Correspondingly, in one implementation of the embodiment of the present application, when the robot gimbal is in the second position, the process of determining the control parameters of the robot gimbal based on the posture information of the robot gimbal and the target coordinates of the target to be detected includes:

[0079] According to the third coordinate of the robot body in the scene coordinate system in the second position, the target coordinates of the target to be detected, and the correspondence between the scene coordinate system and the gimbal optical center coordinate system, the fourth coordinate of the target to be detected in the gimbal optical center coordinates is determined; according to the fourth coordinate, the control parameters of the robot gimbal are determined.

[0080] Specifically, when the robot performs a corresponding task, such as when the robot performs an inspection task, when the robot moves to the second position, the coordinates of the robot body actually reached in the scene coordinate system are used as the third coordinates, such as p(x1, y1, z1). According to the relative position of the robot gimbal relative to the robot body in the second position, the corresponding relationship between the scene coordinate system and the gimbal optical center coordinate system can be determined, and then the fourth coordinates of the target to be detected in the gimbal optical center coordinates are calculated, such as p(xcd1, ycd1, zcd1). Then, according to the fourth coordinates, the control parameters of the robot gimbal can be determined, such as the control parameters of the rotation angle, so that the shooting angle of the robot gimbal can be adjusted based on the control parameters, and a more accurate image can be obtained.

[0081] In one implementation of the embodiment of the present application, the process of determining the control parameters of the robot gimbal according to the fourth coordinate includes:

[0082] Based on the coordinate sub-parameters of each coordinate dimension corresponding to the fourth coordinate, the rotation angle parameters of the robot gimbal are determined; based on the rotation angle parameters, the control parameters are determined. Among them, the rotation angle parameters include at least the horizontal rotation angle and the vertical rotation angle. Specifically, the coordinate dimensions may include x, y, and z dimensions, that is, according to the various coordinate sub-parameters of the fourth coordinate, the rotation angle parameters of the gimbal that can accurately and clearly capture the current target to be detected are determined. Then, according to the current rotation angle of the gimbal, the control parameters can be determined, that is, based on the control parameters, the current rotation angle of the gimbal is controlled and adjusted to achieve the rotation angle parameters that match the fourth coordinate.

[0083] For example, the fourth coordinate of the target to be detected in the optical center coordinates of the gimbal is represented by p(xcd1, ycd1, zcd1). Then the corresponding vertical rotation parameter tilt1 and horizontal rotation parameter pan1 are calculated:

[0084] ;

[0085] pan1=arctan(xcd1 / ycd1).

[0086] Then, according to the above rotation parameters, the control parameters of adjusting the horizontal rotation parameter pan0 and the vertical rotation parameter tilt0 of the current robot gimbal are determined so that the robot gimbal is adjusted to the vertical rotation parameter tilt1 and the horizontal rotation parameter pan1, and then the gimbal is controlled to shoot the target to be detected to obtain the captured image.

[0087] See also Figure 3, is a scene schematic diagram of a robot gimbal applied to inspection tasks provided in an embodiment of the present application. When the robot gimbal is in the first position, the coordinates of the robot body in the scene coordinate system are represented by p(x0, y0, z0). Then, the relative distance between the robot gimbal and the target to be detected is calculated according to the current image acquisition parameters, as represented by wd, and the target coordinates of the target to be detected are calculated, represented by p(xd, yd, zd). When the robot gimbal moves to the second position, the coordinates of the robot body in the scene coordinate system are represented by p(x1, y1, z1), and then the target coordinates of the target to be detected are converted to its coordinates in the optical center coordinate system of the gimbal, represented by p(xcd1, ycd1, zcd1), and the control parameters for controlling the rotation parameters of the gimbal are calculated based on the coordinates. Correspondingly, each position point of the robot gimbal during the execution of the task can determine the control parameters at the current position point according to the relative distance obtained at the first position and the target coordinates of the target to be detected in the above-mentioned processing method. Since the target to be detected is generally far away from the robot gimbal, the relative distance between the two is less affected by errors. Therefore, in the embodiment of the present application, a mapping relationship between image acquisition parameters and relative distance can be established, so as to determine the coordinates of the target to be detected in the corresponding coordinate system according to the relative distance, and determine the control parameters for the gimbal, thereby achieving the purpose of efficiently and accurately controlling the gimbal to capture accurate images.

[0088] It can be seen that in the embodiment of the present application, the establishment of a complex and uncertain mathematical model inside a real pan-tilt camera is avoided, and the error between the model and reality is eliminated through the generalization of the target processing model corresponding to the neural network. And it is only necessary to calculate and calculate the relative position relationship between the target to be detected and the robot pan-tilt according to the current image acquisition parameters in the point acquisition stage, and at the same time calculate and record the target coordinates of the target to be detected. There is no need to control the robot to pre-execute the corresponding task route in the point acquisition stage, and record the image acquisition parameters corresponding to each task point, etc., which provides the efficiency of actual deployment application. The control method is simple in the task execution stage of the robot pan-tilt, and it only needs to control the rotation parameters of the pan-tilt, which improves the task execution efficiency of the robot pan-tilt.

[0089] In the embodiment of the present application, a robot is provided. Figure 4 , the robot may include the following components:

[0090] The robot body 401, the controller 402 and the pan-tilt platform 403, wherein the pan-tilt platform is provided with an image acquisition module 4031, and the image acquisition module 4031 is used to acquire the image of the target to be detected. The controller 402 and the pan-tilt platform 403 are arranged in the robot body 401, that is, the robot body includes the controller and the pan-tilt platform.

[0091] The controller 402 is used to:

[0092] When the robot gimbal is in a first position, determining a first relative position relationship between the target to be detected and the robot gimbal based on image acquisition parameters of the robot gimbal;

[0093] Based on the first relative position relationship, determining the target coordinates of the target to be detected;

[0094] When the robot gimbal is in the second position, determining control parameters of the robot gimbal based on the posture information of the robot gimbal and the target coordinates of the target to be detected;

[0095] Based on the control parameters, the robot gimbal is controlled to capture an image of the target to be detected.

[0096] The controller is also used to perform the following processing:

[0097] Optionally, when the robot gimbal is in the first position, determining a first relative position relationship between the target to be detected and the robot gimbal based on an image acquisition parameter of the robot gimbal includes:

[0098] When the robot gimbal is in a first position, determining a first relative distance between the target to be detected and the robot gimbal based on an image acquisition parameter of the robot gimbal;

[0099] A first relative position relationship between the target to be detected and the robot gimbal is determined according to the first relative distance.

[0100] Optionally, determining a first relative distance between the target to be detected and the robot gimbal based on an image acquisition parameter of the robot gimbal includes:

[0101] Processing the image acquisition parameters of the robot gimbal based on the target processing model to obtain a first relative distance between the target to be detected and the robot gimbal, wherein the image acquisition parameters include a focus parameter and a zoom parameter;

[0102] The target processing model is obtained by training a target data set, wherein the target data set includes a plurality of data samples, and each data sample includes relevant information between the current image acquisition parameters of the robot gimbal and the distance of the acquired target.

[0103] Optionally, when the robot gimbal is in the first position, determining a first relative distance between the target to be detected and the robot gimbal based on an image acquisition parameter of the robot gimbal comprises:

[0104] In response to the robot gimbal detecting that the target to be detected is in the target area of ​​the acquisition field of view when the robot gimbal is at the first position, obtaining current image acquisition parameters of the robot gimbal when the focus processing is completed;

[0105] Based on the current image acquisition parameters, a first relative distance between the target to be detected and the robot gimbal is determined.

[0106] Optionally, determining a first relative position relationship between the target to be detected and the robot gimbal according to the first relative distance includes:

[0107] Determine a first direction parameter of the target to be detected relative to the robot gimbal according to the first relative distance;

[0108] Obtaining a second direction parameter and a third direction parameter corresponding to the target area, wherein the first direction parameter, the second direction parameter and the third direction parameter correspond to different directions respectively;

[0109] The first relative position relationship is determined according to the first direction parameter, the second direction parameter and the third direction parameter.

[0110] Optionally, the first relative position relationship is a first coordinate of the target to be detected in a gimbal optical center coordinate system corresponding to the robot gimbal; and determining the target coordinate of the target to be detected based on the first relative position relationship includes:

[0111] Determine a second coordinate of the target to be detected in the robot coordinate system according to the relative posture of the robot gimbal with respect to the robot body and the first coordinate in the first position;

[0112] The target coordinates are determined according to the robot posture of the robot body in the scene coordinate system and the second coordinates in the case of the first position.

[0113] Optionally, when the robot gimbal is in the second position, determining the control parameters of the robot gimbal based on the posture information of the robot gimbal and the target coordinates of the target to be detected includes:

[0114] Determine the fourth coordinate of the target to be detected in the optical center coordinate of the gimbal according to the third coordinate of the robot body in the scene coordinate system, the target coordinate of the target to be detected, and the corresponding relationship between the scene coordinate system and the gimbal optical center coordinate system in the case of the second position;

[0115] According to the fourth coordinate, control parameters of the robot gimbal are determined.

[0116] Optionally, determining the control parameters of the robot gimbal according to the fourth coordinate includes:

[0117] Determine a rotation angle parameter of the robot gimbal based on the coordinate sub-parameters of each coordinate dimension corresponding to the fourth coordinate, wherein the rotation angle parameter includes at least a horizontal rotation angle and a vertical rotation angle;

[0118] Based on the rotation angle parameter, a control parameter is determined.

[0119] Optionally, it also includes:

[0120] When the gimbal type feature of the robot gimbal is updated, the target processing model is adjusted based on the gimbal type feature, so that the image acquisition parameters of the robot gimbal are processed based on the adjusted target processing model to obtain a first relative distance between the target to be detected and the robot gimbal.

[0121] It should be noted that the specific implementation of the controller in this embodiment can refer to the corresponding content in the previous text and will not be described in detail here.

[0122] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0123] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0124] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0125] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A robot pan-tilt control method, comprising: When the robot gimbal is in a first position, determining a first relative position relationship between the target to be detected and the robot gimbal based on image acquisition parameters of the robot gimbal; Based on the first relative position relationship, determining the target coordinates of the target to be detected; When the robot gimbal is in the second position, determining control parameters of the robot gimbal based on the posture information of the robot gimbal and the target coordinates of the target to be detected; Based on the control parameters, the robot gimbal is controlled to capture an image of the target to be detected.

2. The method according to claim 1, wherein when the robot gimbal is in the first position, determining the first relative position relationship between the target to be detected and the robot gimbal based on the image acquisition parameters of the robot gimbal comprises: When the robot gimbal is in a first position, determining a first relative distance between the target to be detected and the robot gimbal based on an image acquisition parameter of the robot gimbal; A first relative position relationship between the target to be detected and the robot gimbal is determined according to the first relative distance.

3. The method according to claim 2, wherein determining the first relative distance between the target to be detected and the robot gimbal based on the image acquisition parameters of the robot gimbal comprises: Processing the image acquisition parameters of the robot gimbal based on the target processing model to obtain a first relative distance between the target to be detected and the robot gimbal, wherein the image acquisition parameters include a focus parameter and a zoom parameter; The target processing model is obtained by training a target data set, wherein the target data set includes a plurality of data samples, and each data sample includes relevant information between the current image acquisition parameters of the robot gimbal and the distance of the acquired target.

4. The method according to claim 2, wherein when the robot gimbal is in the first position, determining the first relative distance between the target to be detected and the robot gimbal based on the image acquisition parameters of the robot gimbal comprises: In response to the robot gimbal detecting that the target to be detected is in the target area of ​​the acquisition field of view when the robot gimbal is at the first position, obtaining current image acquisition parameters of the robot gimbal when the focus processing is completed; Based on the current image acquisition parameters, a first relative distance between the target to be detected and the robot gimbal is determined.

5. The method according to claim 4, wherein determining a first relative position relationship between the target to be detected and the robot gimbal according to the first relative distance comprises: Determine a first direction parameter of the target to be detected relative to the robot gimbal according to the first relative distance; Obtaining a second direction parameter and a third direction parameter corresponding to the target area, wherein the first direction parameter, the second direction parameter and the third direction parameter correspond to different directions respectively; The first relative position relationship is determined according to the first direction parameter, the second direction parameter and the third direction parameter.

6. The method according to claim 1, wherein the first relative position relationship is a first coordinate of the target to be detected in a gimbal optical center coordinate system corresponding to the robot gimbal; and determining the target coordinate of the target to be detected based on the first relative position relationship comprises: Determine a second coordinate of the target to be detected in the robot coordinate system according to the relative posture of the robot gimbal with respect to the robot body and the first coordinate in the first position; The target coordinates are determined according to the robot posture of the robot body in the scene coordinate system and the second coordinates in the case of the first position.

7. The method according to claim 6, wherein when the robot gimbal is in the second position, determining the control parameters of the robot gimbal based on the posture information of the robot gimbal and the target coordinates of the target to be detected comprises: Determine the fourth coordinate of the target to be detected in the optical center coordinate of the gimbal according to the third coordinate of the robot body in the scene coordinate system, the target coordinate of the target to be detected, and the corresponding relationship between the scene coordinate system and the gimbal optical center coordinate system in the case of the second position; According to the fourth coordinate, control parameters of the robot gimbal are determined.

8. The method according to claim 7, wherein determining the control parameters of the robot gimbal according to the fourth coordinate comprises: Determine a rotation angle parameter of the robot gimbal based on the coordinate sub-parameters of each coordinate dimension corresponding to the fourth coordinate, wherein the rotation angle parameter includes at least a horizontal rotation angle and a vertical rotation angle; Based on the rotation angle parameter, a control parameter is determined.

9. The method according to claim 3, further comprising: When the gimbal type feature of the robot gimbal is updated, the target processing model is adjusted based on the gimbal type feature, so that the image acquisition parameters of the robot gimbal are processed based on the adjusted target processing model to obtain a first relative distance between the target to be detected and the robot gimbal.

10. A robot comprising: Robot body; A controller and a pan-tilt platform, wherein the pan-tilt platform is provided with an image acquisition module, and the image acquisition module is used to acquire an image of a target to be detected; The controller is used to: When the robot gimbal is in a first position, determining a first relative position relationship between the target to be detected and the robot gimbal based on image acquisition parameters of the robot gimbal; Based on the first relative position relationship, determining the target coordinates of the target to be detected; When the robot gimbal is in the second position, determining control parameters of the robot gimbal based on the posture information of the robot gimbal and the target coordinates of the target to be detected; Based on the control parameters, the robot gimbal is controlled to capture an image of the target to be detected.

Images