Robot positioning method, robot and computer readable storage medium

Abstract

The invention is suitable for the technical field of robots, and particularly relates to a robot positioning method, a robot and a computer readable storage medium. In the method, when positioning is carried out through an IMU, a robot can obtain a first acceleration and a first angular velocity of the robot through the IMU, and obtain a first image through an optical capture unit, so as to determine a second acceleration and a second angular velocity of the robot according to a first pixel point in the first image; and determining the target pose of the robot according to the accelerated speed obtained by fusing the first accelerated speed and the second accelerated speed and the angular speed obtained by fusing the first angular speed and the second angular speed, so as to correct the first accelerated speed obtained by the IMU according to the second accelerated speed determined by the first image. And correcting the first angular velocity acquired by the IMU according to the second angular velocity determined by the first image, thereby reducing acceleration error and / or angular velocity error caused by IMU null drift, improving positioning accuracy of the IMU, and improving user experience.

Description

Technical Field

[0001] The present application belongs to the field of robot technology, and in particular, relates to a robot positioning method, a robot and a computer-readable storage medium. Background Art

[0002] Inertial measurement unit (IMU) can be used to measure the angle (or angular velocity) and acceleration of an object, and can be widely used in robotics and other fields. For example, during robot navigation, the robot can be positioned by IMU to navigate according to the positioning results of IMU. However, due to the internal noise, error or other environmental factors (such as temperature changes, electromagnetic interference, etc.) of IMU, the IMU may have zero drift problems, resulting in poor positioning accuracy of IMU. Summary of the invention

[0003] The embodiments of the present application provide a robot positioning method, a robot, and a computer-readable storage medium, which can improve the positioning accuracy of the IMU.

[0004] In a first aspect, an embodiment of the present application provides a robot positioning method, which is applied to a robot, wherein the robot includes an inertial measurement unit and an optical capture unit, and the method may include:

[0005] Acquire a first acceleration and a first angular velocity of the robot by using the inertial measurement unit;

[0006] Acquire a first image through the optical capture unit, and determine a second acceleration and a second angular velocity of the robot according to a first pixel point in the first image;

[0007] fusing the first acceleration and the second acceleration to obtain a fused acceleration, and fusing the first angular velocity and the second angular velocity to obtain a fused angular velocity;

[0008] The target position and posture of the robot is determined according to the fused acceleration and the fused angular velocity.

[0009] In the robot positioning method provided above, when the robot is positioned by IMU, the robot can obtain the first acceleration and the first angular velocity of the robot through IMU. In addition, the robot can also obtain the first image through the optical capture unit, and can determine the second acceleration and the second angular velocity of the robot according to the first pixel in the first image. Then, the robot can fuse the first acceleration and the second acceleration to obtain the fused acceleration, and fuse the first angular velocity and the second angular velocity to obtain the fused angular velocity, so as to determine the target posture of the robot according to the fused acceleration and the fused angular velocity. That is, when the robot is positioned by IMU, the robot can obtain the first image through the optical capture unit, and can correct the first acceleration obtained by IMU according to the second acceleration determined by the first image, and can correct the first angular velocity obtained by IMU according to the second angular velocity determined by the first image, which can reduce the acceleration error and / or angular velocity error caused by IMU zero drift, and can improve the positioning accuracy of IMU, so as to provide accurate positioning for the robot, improve the stability and reliability of robot navigation, and enhance user experience.

[0010] In a possible implementation manner, determining the second acceleration and the second angular velocity of the robot according to the first pixel point in the first image may include:

[0011] Acquire a second image by the optical capturing unit, where the second image is an image acquired by the optical capturing unit before acquiring the first image;

[0012] Determine a first position of the first pixel in the first image, and determine a second position of the first pixel in the second image;

[0013] A second acceleration and a second angular velocity of the robot are determined according to the first position and the second position.

[0014] Exemplarily, determining the second acceleration and the second angular velocity of the robot according to the first position and the second position may include:

[0015] determining an offset distance between the first image and the second image according to the first position and the second position;

[0016] The second acceleration of the robot is determined according to the offset distance and a preset time interval, wherein the preset time interval is an interval between a first moment and a second moment, wherein the first moment is a moment of acquiring the first image, and the second moment is a moment of acquiring the second image.

[0017] Exemplarily, determining the second acceleration and the second angular velocity of the robot according to the first position and the second position may include:

[0018] Determine a first angle corresponding to the robot according to the first position and the third position, wherein the third position is the position of the rotation point corresponding to the robot;

[0019] Determining a second angle corresponding to the robot according to the second position and the third position;

[0020] The second angular velocity of the robot is determined according to the first angle, the second angle and a preset time interval, wherein the preset time interval is the interval between a first moment and a second moment, the first moment is the moment of acquiring the first image, and the second moment is the moment of acquiring the second image.

[0021] In some embodiments, the optical capture unit includes a light emitting diode and an optical sensor, and acquiring the first image through the optical capture unit may include:

[0022] The infrared light is emitted to the target object by the light emitting diode, and the first image is acquired by the optical sensor. The first image is an image reflected by the target object to the optical sensor after the infrared light is emitted to the target object.

[0023] In one example, the optical capture unit further includes an optical lens, and the emitting infrared light to the target object through the light emitting diode may include:

[0024] The infrared light is emitted by the light emitting diode, and is emitted to the target object by the optical lens.

[0025] In some other embodiments, the optical capture unit includes a laser emitter and an optical sensor, and acquiring the first image through the optical capture unit may include:

[0026] The laser is emitted to the target object by the laser emitter, and the first image is acquired by the optical sensor. The first image is an image reflected by the target object to the optical sensor after the laser is emitted to the target object.

[0027] In a second aspect, an embodiment of the present application provides a robot, which may include an inertial measurement unit, an optical capture unit, and a processing unit;

[0028] The inertial measurement unit is used to obtain a first acceleration and a first angular velocity of the robot;

[0029] The optical capture unit is used to acquire a first image, and determine a second acceleration and a second angular velocity of the robot according to a first pixel point in the first image;

[0030] The processing unit is used to fuse the first acceleration and the second acceleration to obtain a fused acceleration, and to fuse the first angular velocity and the second angular velocity to obtain a fused angular velocity, and determine the target posture of the robot according to the fused acceleration and the fused angular velocity.

[0031] In a third aspect, an embodiment of the present application provides a robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the robot implements the robot positioning method described in any one of the first aspects above.

[0032] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the computer implements the robot positioning method described in any one of the first aspects above.

[0033] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a robot, enables the robot to execute the robot positioning method described in any one of the first aspects above.

[0034] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 is a schematic structural diagram of an optical capture unit provided in an embodiment of the present application;

[0037] Figure 2 It is a flowchart of a robot positioning method provided in an embodiment of the present application;

[0038] Figure 3 is an example diagram of an image acquired by an optical capture unit provided in an embodiment of the present application;

[0039] Figure 4is a schematic diagram of a process of determining a second acceleration and a second angular velocity of a robot according to a first pixel point in a first image by an optical capture unit provided by an embodiment of the present application;

[0040] Figure 5 is a schematic diagram of the structure of a robot provided in an embodiment of the present application;

[0041] Figure 6 It is a schematic diagram of the structure of another robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0043] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0044] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0045] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0046] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0047] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0048] IMU can be used to measure the angle (or angular velocity) and acceleration of an object, and can be applied to fields such as robotics. For example, during robot navigation, the robot can be positioned by IMU to navigate according to the positioning results of IMU. However, due to the internal noise, error or other environmental factors (such as temperature changes, electromagnetic interference, etc.) of IMU, the IMU may have zero drift problems, resulting in poor positioning accuracy of IMU.

[0049] It should be understood that the zero drift of the IMU refers to the situation where the measurement value output by the IMU is not zero when there is no movement or no external force.

[0050] To solve the above problems, the embodiments of the present application provide a robot positioning method, a robot and a computer-readable storage medium. Among them, the robot may include an IMU and an optical capture unit. In the method, when the robot is positioned by the IMU, the robot can obtain the first acceleration and the first angular velocity of the robot through the IMU. In addition, the robot can also obtain the first image through the optical capture unit, and can determine the second acceleration and the second angular velocity of the robot according to the first pixel in the first image. Then, the robot can fuse the first acceleration and the second acceleration to obtain the fused acceleration, and can fuse the first angular velocity and the second angular velocity to obtain the fused angular velocity, so as to determine the target posture of the robot according to the fused acceleration and the fused angular velocity. That is, when the robot is positioned by the IMU, the robot can obtain the first image through the optical capture unit, and can correct the first acceleration obtained by the IMU according to the second acceleration determined by the first image, and can correct the first angular velocity obtained by the IMU according to the second angular velocity determined by the first image, which can reduce the acceleration error and / or angular velocity error caused by the zero drift of the IMU, and can improve the positioning accuracy of the IMU, so as to provide accurate positioning for the robot, improve the stability and reliability of the robot navigation, and enhance the user experience.

[0051] It should be understood that the robot positioning method provided in the embodiments of the present application can be applied to various robots that can be positioned by IMU, for example, cleaning robots or commercial robots, etc. The embodiments of the present application do not impose any restrictions on the specific type of robot.

[0052] In an embodiment of the present application, the robot may include an IMU and an optical capture unit. Among them, the IMU can be used to measure the acceleration (hereinafter referred to as the first acceleration) and angular velocity (hereinafter referred to as the first angular velocity) of the robot, such as measuring the first acceleration and the first angular velocity during the movement of the robot. The optical capture unit can be used to obtain an image of the environment in which the robot is located (i.e., the first image), and can determine the acceleration (hereinafter referred to as the second acceleration) and angular velocity (hereinafter referred to as the second angular velocity) of the robot based on the first image. Therefore, the robot can fuse the first acceleration with the second acceleration to correct the first acceleration through the second acceleration, thereby reducing the acceleration error obtained by the IMU. In addition, the robot can fuse the first angular velocity with the second angular velocity to correct the first angular velocity through the second angular velocity, thereby reducing the angular velocity error obtained by the IMU, so that the robot's posture (i.e., the target posture) can be accurately determined by the fused acceleration and the fused angular velocity, thereby improving the accuracy of posture determination.

[0053] In one example, the first image may be an image corresponding to a target object in the environment where the robot is located. For example, during the movement of the robot, the robot may obtain a first image corresponding to the target object, and may determine the second acceleration and second angular velocity of the robot according to the first image corresponding to the target object, so as to correct the first acceleration and first angular velocity obtained by the IMU according to the second acceleration and second angular velocity, thereby improving the accuracy of the acceleration and angular velocity, and thus improving the accuracy of the positioning of the robot.

[0054] It should be noted that the target object can be specifically determined according to the actual application scenario, and the present application embodiment does not impose any limitation on this. For example, the target object can be determined to be the ground according to the actual application scenario. The following will be exemplified by taking the target object as the ground as an example.

[0055] In some embodiments, the optical capture unit may include a light emitting diode and an optical sensor. The light emitting diode may be used to emit infrared light. The optical sensor may be used to obtain an image (e.g., a first image) of the infrared light after it is reflected by the ground. That is, the first image may be an infrared image. In other words, the robot may emit infrared light to the ground through a light emitting diode. After the infrared light is reflected by the ground, it may be transmitted to the optical sensor. The optical sensor may obtain an infrared image (e.g., a first image) corresponding to the ground, so that the second acceleration and second angular velocity of the robot may be determined based on the infrared image corresponding to the ground, so that the second acceleration and second angular velocity of the robot may be determined through an infrared image that is highly accurate and not affected by the environment, thereby reducing the influence of the environment on the determination of the second acceleration and second angular velocity, and improving the accuracy of the second acceleration and second angular velocity, thereby improving the accuracy of the robot's positioning.

[0056] It should be noted that the optical sensor can be specifically determined according to the actual application scenario, and the embodiments of the present application do not impose any restrictions on this. For example, the optical sensor can be determined according to the actual application scenario to be a complementary metal oxide semiconductor (CMOS) sensor.

[0057] In one example, in order to allow the optical sensor to obtain an image containing more information, the optical capture unit may further include one or more sets of optical lenses. The robot may emit infrared light to the ground through a light emitting diode and an optical lens group. For example, the robot may emit infrared light emitted by a light emitting diode to the ground through an optical lens group, so that the infrared light emitted by the light emitting diode can be emitted to more areas of the ground through the optical lens group, so that the image obtained by the optical sensor may include more ground information, so that the second acceleration and second angular velocity of the robot can be accurately determined based on the image obtained by the optical capture unit.

[0058] For example, see Figure 1 , Figure 1 A schematic structural diagram of an optical capture unit provided in an embodiment of the present application is shown.

[0059] like Figure 1 As shown, the optical capture unit may include a light emitting diode 110, an optical lens group 120, and an optical sensor 130. The light emitting diode 110 may be used to emit infrared light. The optical lens group 120 may be used to transmit the infrared light emitted by the light emitting diode 110 to the ground. That is, the infrared light emitted by the light emitting diode 110 may be emitted to the ground after passing through the optical lens group 120. After being reflected by the ground, the infrared light may be transmitted to the optical sensor 130, and the optical sensor 130 may obtain an infrared image corresponding to the ground.

[0060] In other embodiments, the optical capture unit may include a laser emitter and an optical sensor. The laser emitter may be used to emit a laser. The optical sensor may be used to obtain an image (e.g., a first image) after the laser is reflected by the ground. That is, the first image may be a laser image. In other words, the robot may emit a laser to the ground through a laser emitter. After the laser is reflected by the ground, it may be transmitted to the optical sensor. The optical sensor may obtain a laser image corresponding to the ground, so that the second acceleration and second angular velocity of the robot may be determined based on the laser image corresponding to the ground. The second acceleration and second angular velocity of the robot may be determined through a laser image that is highly accurate and not affected by the environment, thereby reducing the influence of the environment on the determination of the second acceleration and the second angular velocity, and improving the accuracy of the second acceleration and the second angular velocity, thereby improving the accuracy of the robot positioning.

[0061] In one example, in order to enable the optical sensor to acquire an image containing more information, the laser emitter may be a surface laser emitter, or the laser emitter may include multiple line laser emitters. Laser emission can be performed through the surface laser emitter or multiple line laser emitters, so that the laser can be emitted to more areas of the ground, and the image acquired by the optical sensor can include more ground information, so that the second acceleration and second angular velocity of the robot can be accurately determined based on the first image.

[0062] It should be noted that optical capture has the characteristics of high precision and high accuracy, and can accurately capture information such as the position and direction of an object. The embodiment of the present application fuses the second acceleration determined according to the first image acquired by the optical capture unit with the first acceleration acquired by the IMU, and fuses the second angular velocity determined according to the first image with the second angular velocity acquired by the IMU, so as to obtain more accurate and reliable acceleration and angular velocity, thereby providing a more accurate and reliable posture for the robot. In addition, optical capture can acquire and process images in a real-time dynamic environment, which can not only meet the requirements of real-time and dynamic performance, but also accurately determine the posture of the robot during the movement of the robot, ensure the accuracy of the posture during the movement of the robot, and improve the stability and reliability of the robot's navigation. In addition, optical capture can be unaffected by the environment, that is, optical capture can accurately acquire and process images in various environments (such as indoors, outdoors, in bright and dark environments), can maintain the accuracy of the robot's posture in different environments, and can improve the environmental adaptability of the robot.

[0063] For example, when the robot is a cleaning robot, the second acceleration and second angular velocity determined according to the first image obtained by the optical capture unit can effectively reduce the positioning deviation caused by the IMU zero drift, ensure that the cleaning robot maintains accurate positioning during movement, and enable the cleaning robot to move more accurately along the predetermined path, thereby improving cleaning efficiency.

[0064] The robot positioning method provided in the embodiment of the present application will be described in detail below with reference to the accompanying drawings and specific application scenarios.

[0065] See also Figure 2 , Figure 2 FIG. 1 is a schematic flow chart of a robot positioning method provided by an embodiment of the present application. The method can be applied to a robot. The robot may include an IMU and an optical capture unit. Figure 2 As shown, the method may include:

[0066] S201: The robot obtains a first acceleration and a first angular velocity of the robot through an IMU.

[0067] The embodiment of the present application does not impose any specific restrictions on the content of the IMU obtaining the first acceleration and the first angular velocity of the robot, and reference may be made to the relevant content of the IMU obtaining acceleration and angular velocity in the prior art.

[0068] S202: The robot acquires a first image through an optical capture unit, and determines a second acceleration and a second angular velocity of the robot according to a first pixel point in the first image.

[0069] In one example, the first image may be an infrared image. The optical capture unit may include a light emitting diode and an optical sensor. The light emitting diode may emit infrared light to the ground, and the infrared light may be transmitted to the optical sensor after being reflected by the ground, so that the optical sensor may obtain an infrared image corresponding to the ground.

[0070] In another example, the first image may be a laser image. The optical capture unit may include a laser transmitter and an optical sensor. The laser transmitter may emit a laser toward the ground, and the laser may be transmitted to the optical sensor after being reflected by the ground, and the optical sensor may obtain the laser image corresponding to the ground.

[0071] In some embodiments, the optical capture unit may capture images according to a specified time interval (hereinafter referred to as a preset time interval). For example, the optical capture unit may capture image A at time T0, image B at time (T0+Δt), image C at time (T0+2*Δt), image D at time (T0+3*Δt), etc., where Δt may refer to a preset time interval.

[0072] In one example, the optical capture unit may include a processor, such as a digital signal processor. After acquiring a certain image (e.g., image B), the optical capture unit (e.g., a processor in the optical capture unit) may match image B with an image (e.g., image A) acquired by the optical capture unit last time, and determine whether image B has changed relative to image A, thereby determining whether the robot has moved. When it is determined that image B has changed relative to image A, the optical capture unit may determine that the robot has moved. At this time, the optical capture unit may determine image B as the first image, and may determine image A acquired last time as the second image, so as to determine the second acceleration and second angular velocity of the robot based on the first image (i.e., image B) and the second image (i.e., image A).

[0073] For example, see Figure 3 , Figure 3 An example diagram of an image acquired by an optical capture unit provided by an embodiment of the present application is shown.

[0074] Assume that the image A acquired by the optical capture unit is as follows Figure 3 As shown in (a) in FIG. 1 , the image B obtained by the optical capture unit is as follows: Figure 3 As shown in (b), the optical capture unit can determine the change of image B relative to image A based on image A and image B, that is, it can be determined that the robot has moved. At this time, the optical capture unit can determine image B as the first image and image A as the second image, so as to determine the second acceleration and second angular velocity of the robot based on image B and image A.

[0075] It should be understood that when it is determined that image B has not changed relative to image A, the optical capture unit can determine that the robot has not moved. At this time, the optical capture unit can continue to acquire images (such as image C) after a preset time interval. After acquiring image C, the optical capture unit can match image C with the image acquired last time (i.e., image B) to determine whether image C has changed relative to image B, thereby determining whether the robot has moved. When it is determined that image C has changed relative to image B, the optical capture unit can determine that the robot has moved. At this time, the optical capture unit can determine image C as the first image, and can determine image B acquired last time as the second image, so as to determine the second acceleration and second angular velocity of the robot based on the first image (i.e., image C) and the second image (i.e., image B).

[0076] That is, when the optical capture unit is performing an image acquisition process, the optical capture unit can determine the first image and the second image based on the acquired images, and can determine the second acceleration and the second angular velocity of the robot based on the first image and the second image. It should be understood that the second image can be an image acquired before acquiring the first image, for example, the optical capture unit can acquire the first image at a first moment, and can acquire the second image at a second moment. The second moment can be earlier than the first moment.

[0077] It should be noted that the second image may be an infrared image, or may be a laser image. For example, when the first image is an infrared image, the second image may be an infrared image. For example, when the first image is a laser image, the second image may be a laser image.

[0078] In a possible implementation, the optical capture unit can obtain the position of a specified pixel point (hereinafter referred to as the first pixel point) in the first image (hereinafter referred to as the first position), and can obtain the position of the first pixel point in the second image (hereinafter referred to as the second position). Then, the optical capture unit can determine whether the first image has changed relative to the second image based on whether the first position corresponding to the first pixel point is the same as the second position, thereby determining whether the robot has moved. When the first position is different from the second position, the optical capture unit can determine that the first image has changed relative to the second image, that is, it can be determined that the robot has moved. When determining that the robot has moved, the optical capture unit can determine the second acceleration and second angular velocity of the robot based on the first position and the second position corresponding to the first pixel point.

[0079] That is, the optical capture unit can determine whether the robot moves according to the position of the same pixel in different images, that is, the position change of the same pixel. When determining that the robot moves, the optical capture unit can determine the second acceleration and second angular velocity of the robot according to the position of the same pixel in different images, that is, the position change of the same pixel.

[0080] It should be noted that the first pixel point can be specifically determined according to the actual application scenario, and the embodiment of the present application does not impose any limitation on this.

[0081] The following is a detailed description of how the optical capture unit determines the second acceleration and the second angular velocity of the robot based on the first position and the second position corresponding to the first pixel point.

[0082] In one example, after acquiring the first position and the second position corresponding to the first pixel point, the optical capture unit can determine the offset distance between the first image and the second image based on the first position and the second position, and can determine the second acceleration of the robot based on the offset distance and a preset time interval.

[0083] For example, when the first position is (x1, y1) and the second position is (x2, y2), the optical capture unit can determine the offset distance between the first image and the second image. It should be understood that the offset distance Δd may refer to the offset distance at the pixel level. Subsequently, the optical capture unit may convert the offset distance at the pixel level corresponding to a pixel point into the actual offset distance according to the actual distance corresponding to a pixel point. Assuming that a pixel point = p millimeters, the actual offset distance ΔD between the first image and the second image = Δd*p.

[0084] After acquiring the actual offset distance ΔD between the first image and the second image, the optical capture unit can determine the speed v of the robot according to the time interval between the first image and the second image (for example, the preset time interval Δt), that is, the speed v of the robot can be determined.

[0085] It should be noted that during the movement of the robot, the optical capture unit can acquire the first image and the second image multiple times, and can obtain multiple speeds v of the robot based on the first image and the second image acquired multiple times, so that the second acceleration of the robot can be determined based on the multiple speeds v of the robot and the preset time interval.

[0086] For example, during the movement of the robot, the optical capture unit can acquire image A at time T0, image B at time (T0+Δt), and image C at time (T0+2*Δt), and can determine that image C changes relative to image B, and image B changes relative to image A. At this time, the optical capture unit can determine the speed of the robot (for example, speed v1) based on image B and image A, and can determine the speed of the robot (for example, speed v2) based on image C and image B. Therefore, the optical capture unit can determine the second acceleration a of the robot based on speed v2, speed v1, and a preset time interval Δt. Among them, the second acceleration a of the robot △v=|v2-v1|.

[0087] In other embodiments, during the movement of the robot, the robot may also rotate around a certain rotation point (e.g., rotation point A). Therefore, the optical capture unit may determine the second angular velocity of the robot based on the first image, the second image, and the rotation point A. Specifically, the optical capture unit may obtain the position corresponding to the rotation point A (hereinafter referred to as the third position). Then, the optical capture unit may determine the angle of the robot at the first moment (i.e., the moment of acquiring the first image) (hereinafter referred to as the first angle) based on the first position corresponding to the first pixel point and the third position corresponding to the rotation point A, and may determine the angle of the robot at the second moment (i.e., the moment of acquiring the second image) (hereinafter referred to as the second angle) based on the second position corresponding to the first pixel point and the third position corresponding to the rotation point A. Subsequently, the optical capture unit may determine the second angular velocity of the robot based on the first angle, the second angle, and a preset time interval.

[0088] Exemplarily, assuming that the third position corresponding to the rotation point A is (xc, yc), the optical capture unit can determine the first angle In addition, the optical capture unit can determine a second angle The optical capture unit can then determine the robot's second angular velocity

[0089] It should be understood that the rotation point A may be a certain position point on the robot. Among them, the rotation point A can be specifically determined according to the actual application scenario, and the embodiments of the present application do not impose any restrictions on this. The third position corresponding to the rotation point A may refer to a position in the same coordinate system as the first pixel point. In the embodiments of the present application, the optical capture unit can convert the rotation point A on the robot to the coordinate system where the first pixel point is located through coordinate conversion to obtain the third position corresponding to the rotation point A. The specific method in which the optical capture unit converts the rotation point A on the robot to the coordinate system where the first pixel point is located through coordinate conversion can be determined according to the actual scenario, and the embodiments of the present application do not impose any restrictions on this.

[0090] The following is an exemplary description of the process in which the optical capture unit determines the second acceleration and the second angular velocity of the robot according to the first pixel point in the first image, with reference to the accompanying drawings.

[0091] See also Figure 4 , Figure 4 A schematic flowchart of the optical capture unit provided in an embodiment of the present application determining the second acceleration and second angular velocity of the robot according to the first pixel point in the first image is shown.

[0092] like Figure 4As shown, during the movement of the robot, the light emitting diode in the optical capture unit can emit infrared light to the ground, or the laser emitter in the optical capture unit can emit laser to the ground. After emitting infrared light or laser, the optical sensor in the optical capture unit can obtain an infrared image after the infrared light is reflected by the ground, or obtain a laser image after the laser is reflected by the ground according to a preset time interval.

[0093] After acquiring the infrared image or laser image reflected from the ground through the optical sensor, the processor in the optical capture unit may match the infrared images or laser images acquired twice adjacently to determine whether the infrared images or laser images acquired twice adjacently have changed. When it is determined that the infrared images or laser images acquired twice adjacently have not changed, the optical sensor in the optical capture unit may acquire the infrared image after the infrared light is reflected from the ground, or acquire the laser image after the laser is reflected from the ground, according to a preset time interval. When it is determined that the infrared images or laser images acquired twice adjacently have changed, the processor in the optical capture unit may determine the second acceleration and second angular velocity of the robot based on one or more infrared images or laser images acquired twice adjacently. That is, the second acceleration and second angular velocity of the robot may be determined based on the first pixel point in the infrared image or laser image.

[0094] It should be noted that the above-mentioned determination of the second acceleration and second angular velocity of the robot according to the first pixel in the first image by the optical capture unit is only an exemplary explanation and should not be understood as a limitation on the embodiments of the present application. In the embodiments of the present application, the optical capture unit can be used only to obtain the first image and the second image, and the processor in the robot (such as a digital signal processor) can determine the second acceleration and second angular velocity of the robot according to the first pixel in the first image and the second pixel in the second image. In other words, the specific content of determining the second acceleration and second angular velocity of the robot according to the first pixel in the first image can be performed by the optical capture unit, or can be performed by the processor in the robot.

[0095] S203: The robot fuses the first acceleration and the second acceleration to obtain a fused acceleration, and fuses the first angular velocity and the second angular velocity to obtain a fused angular velocity, and determines the target posture of the robot according to the fused acceleration and the fused angular velocity.

[0096] In the embodiment of the present application, after obtaining the first acceleration and the second acceleration, the robot may fuse the first acceleration with the second acceleration to obtain a fused acceleration. After obtaining the first angular velocity and the second angular velocity of the robot, the robot may fuse the first angular velocity and the second angular velocity to obtain a fused angular velocity. Subsequently, the robot may determine the target posture of the robot according to the fused acceleration and the fused angular velocity.

[0097] It should be noted that the embodiments of the present application do not impose any restrictions on the specific manner of fusing the first acceleration and the second acceleration, and the specific manner of fusing the first angular velocity and the second angular velocity, which can be specifically determined according to the actual application scenario.

[0098] For example, a first confidence level may be set for the first acceleration, and a second confidence level may be set for the second acceleration, so as to perform a weighted calculation of the acceleration according to the first confidence level and the second confidence level to obtain a fused acceleration. Since the accuracy of the second acceleration is generally higher than the accuracy of the first acceleration, the second confidence level may be higher than the first confidence level. Similarly, a third confidence level may be set for the first angular velocity, and a fourth confidence level may be set for the second angular velocity, so as to perform a weighted calculation of the angular velocity according to the third confidence level and the fourth confidence level to obtain a fused angular velocity. Since the accuracy of the second angular velocity is generally higher than the accuracy of the first angular velocity, the fourth confidence level may be higher than the third confidence level.

[0099] It should be understood that specific values ​​of the first confidence level, the second confidence level, the third confidence level, and the fourth confidence level may be determined according to actual application scenarios.

[0100] For example, the robot can fuse the first acceleration and the second acceleration by any one of Kalman filter, particle filter, information fusion, Bayesian filtering, complementary filter, deep learning, or fuzzy logic, etc., to obtain a fused acceleration. Similarly, the robot can fuse the first angular velocity and the second angular velocity by any one of Kalman filter, particle filter, information fusion, Bayesian filtering, complementary filter, deep learning, or fuzzy logic, etc., to obtain a fused angular velocity.

[0101] It should be noted that the embodiment of the present application does not impose any restrictions on the specific method of determining the target position of the robot based on the fused acceleration and the fused angular velocity, which can be determined specifically according to the actual application scenario.

[0102] In an embodiment of the present application, when the robot is positioned by the IMU, the robot can obtain the first acceleration and the first angular velocity of the robot through the IMU. In addition, the robot can also obtain the first image through the optical capture unit, and can determine the second acceleration and the second angular velocity of the robot according to the first pixel in the first image. Then, the robot can fuse the first acceleration and the second acceleration to obtain the fused acceleration, and can fuse the first angular velocity and the second angular velocity to obtain the fused angular velocity, so as to determine the target posture of the robot according to the fused acceleration and the fused angular velocity. That is, when the robot is positioned by the IMU, the robot can obtain the first image through the optical capture unit, and can correct the first acceleration obtained by the IMU according to the second acceleration determined by the first image, and can correct the first angular velocity obtained by the IMU according to the second angular velocity determined by the first image, which can reduce the acceleration error and / or angular velocity error caused by the zero drift of the IMU, and can improve the positioning accuracy of the IMU, so as to provide accurate positioning for the robot, improve the stability and reliability of the robot navigation, and enhance the user experience.

[0103] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0104] Corresponding to the robot positioning method described in the above embodiment, Figure 5 A structural block diagram of a robot provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0105] Reference Figure 5 , the robot may include an inertial measurement unit 501, an optical capture unit 502 and a processing unit 503;

[0106] The inertial measurement unit 501 is used to obtain a first acceleration and a first angular velocity of the robot;

[0107] The optical capture unit 502 is used to acquire a first image, and determine a second acceleration and a second angular velocity of the robot according to a first pixel point in the first image;

[0108] The processing unit 503 is used to fuse the first acceleration and the second acceleration to obtain a fused acceleration, and to fuse the first angular velocity and the second angular velocity to obtain a fused angular velocity, and determine the target posture of the robot according to the fused acceleration and the fused angular velocity.

[0109] In a possible implementation, the optical capture unit 502 is also used to acquire a second image, determine a first position of the first pixel in the first image, and determine a second position of the first pixel in the second image; determine a second acceleration and a second angular velocity of the robot based on the first position and the second position, and the second image is an image acquired by the optical capture unit before acquiring the first image.

[0110] Exemplarily, the optical capture unit 502 is also used to determine an offset distance between the first image and the second image based on the first position and the second position; determine a second acceleration of the robot based on the offset distance and a preset time interval, wherein the preset time interval is an interval between a first moment and a second moment, the first moment being a moment for acquiring the first image, and the second moment being a moment for acquiring the second image.

[0111] Exemplarily, the optical capture unit 502 is also used to determine a first angle corresponding to the robot based on the first position and the third position, where the third position is the position of the rotation point corresponding to the robot; determine a second angle corresponding to the robot based on the second position and the third position; determine the second angular velocity of the robot based on the first angle, the second angle and a preset time interval, where the preset time interval is the interval between a first moment and a second moment, where the first moment is the moment of acquiring the first image, and the second moment is the moment of acquiring the second image.

[0112] In some embodiments, the optical capture unit includes a light emitting diode and an optical sensor;

[0113] The light emitting diode is used to emit infrared light to the target object;

[0114] The optical sensor is used to acquire the first image, where the first image is an image that is reflected from the target object to the optical sensor after the infrared light is emitted to the target object.

[0115] In one example, the optical capture unit further includes an optical lens;

[0116] The light emitting diode is used to emit the infrared light;

[0117] The optical lens is used to emit the infrared light to the target object.

[0118] In some other embodiments, the optical capture unit includes a laser emitter and an optical sensor;

[0119] The laser emitter is used to emit laser to the target object;

[0120] The optical sensor is used to acquire the first image, where the first image is an image reflected from the target object to the optical sensor after the laser is emitted to the target object.

[0121] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0122] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0123] Figure 6 This is a schematic diagram of the structure of another robot provided in an embodiment of the present application. Figure 6 As shown, the robot 6 of this embodiment includes: at least one processor 60 ( Figure 6 Only one is shown in the figure), a memory 61, a computer program 62 stored in the memory 61 and executable on the at least one processor 60, an IMU 63, and an optical capture unit 64. The IMU 63 can obtain a first acceleration and a first angular velocity of the robot. The optical capture unit 64 can obtain a first image. When the processor 60 executes the computer program 62, the steps in any of the above-mentioned robot positioning method embodiments are implemented.

[0124] The robot may include, but is not limited to, a processor 60, a memory 61, an IMU 63, and an optical capture unit 64. Those skilled in the art will appreciate that Figure 6 This is only an example of the robot 6 and does not constitute a limitation on the robot 6. The robot 6 may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0125] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0126] In some embodiments, the memory 61 may be an internal storage unit of the robot 6, such as a hard disk or memory of the robot 6. In other embodiments, the memory 61 may also be an external storage device of the robot 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the robot 6. Further, the memory 61 may also include both an internal storage unit and an external storage device of the robot 6. The memory 61 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0127] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in each of the above robot positioning method embodiments can be implemented.

[0128] An embodiment of the present application provides a computer program product. When the computer program product runs on a robot, the robot can implement the steps in the above-mentioned robot positioning method embodiments when executing the computer program product.

[0129] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may at least include: any entity or device that can carry the computer program code to the device / robot, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, a computer-readable storage medium cannot be an electric carrier signal and a telecommunication signal.

[0130] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0131] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0132] In the embodiments provided in the present application, it should be understood that the disclosed devices / robots and methods can be implemented in other ways. For example, the device / robot embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A robot positioning method, characterized in that: Applied to a robot, the robot comprising an inertial measurement unit and an optical capture unit, the method comprising: Acquire a first acceleration and a first angular velocity of the robot by using the inertial measurement unit; Acquire a first image through the optical capture unit, and determine a second acceleration and a second angular velocity of the robot according to a first pixel point in the first image; fusing the first acceleration and the second acceleration to obtain a fused acceleration, and fusing the first angular velocity and the second angular velocity to obtain a fused angular velocity; The target position and posture of the robot is determined according to the fused acceleration and the fused angular velocity.

2. The method according to claim 1, characterized in that The determining the second acceleration and the second angular velocity of the robot according to the first pixel point in the first image comprises: Acquire a second image by the optical capturing unit, where the second image is an image acquired by the optical capturing unit before acquiring the first image; Determine a first position of the first pixel in the first image, and determine a second position of the first pixel in the second image; A second acceleration and a second angular velocity of the robot are determined according to the first position and the second position.

3. The method according to claim 2, characterized in that Determining a second acceleration and a second angular velocity of the robot according to the first position and the second position includes: determining an offset distance between the first image and the second image according to the first position and the second position; The second acceleration of the robot is determined according to the offset distance and a preset time interval, wherein the preset time interval is an interval between a first moment and a second moment, wherein the first moment is a moment of acquiring the first image, and the second moment is a moment of acquiring the second image.

4. The method according to claim 2, characterized in that: Determining a second acceleration and a second angular velocity of the robot according to the first position and the second position includes: Determine a first angle corresponding to the robot according to the first position and the third position, wherein the third position is the position of the rotation point corresponding to the robot; Determining a second angle corresponding to the robot according to the second position and the third position; The second angular velocity of the robot is determined according to the first angle, the second angle and a preset time interval, wherein the preset time interval is the interval between a first moment and a second moment, the first moment is the moment of acquiring the first image, and the second moment is the moment of acquiring the second image.

5. The method according to any one of claims 1 to 4, characterized in that The optical capture unit includes a light emitting diode and an optical sensor, and acquiring the first image through the optical capture unit includes: The infrared light is emitted to the target object by the light emitting diode, and the first image is acquired by the optical sensor. The first image is an image reflected by the target object to the optical sensor after the infrared light is emitted to the target object.

6. The method according to claim 5, characterized in that The optical capture unit further includes an optical lens, and the emitting infrared light to the target object through the light emitting diode includes: The infrared light is emitted by the light emitting diode, and is emitted to the target object by the optical lens.

7. The method according to any one of claims 1 to 4, characterized in that The optical capture unit includes a laser emitter and an optical sensor, and acquiring the first image through the optical capture unit includes: The laser is emitted to the target object by the laser emitter, and the first image is acquired by the optical sensor. The first image is an image reflected by the target object to the optical sensor after the laser is emitted to the target object.

8. A robot, characterized in that: The robot includes an inertial measurement unit, an optical capture unit and a processing unit; The inertial measurement unit is used to obtain a first acceleration and a first angular velocity of the robot; The optical capture unit is used to acquire a first image, and determine a second acceleration and a second angular velocity of the robot according to a first pixel point in the first image; The processing unit is used to fuse the first acceleration and the second acceleration to obtain a fused acceleration, and to fuse the first angular velocity and the second angular velocity to obtain a fused angular velocity, and determine the target posture of the robot according to the fused acceleration and the fused angular velocity.

9. A robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the robot is enabled to implement the robot positioning method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a computer, the computer is enabled to implement the robot positioning method according to any one of claims 1 to 7.

Images