Automatic recharging method and device for self-moving robot

Abstract

The invention provides an automatic recharging method and device for a self-moving robot. The method comprises the following steps: acquiring a first distance between the self-moving robot and a charging pile; if the first distance is greater than a preset distance, acquiring a first current environment image, comparing the first current environment image with a preset reference environment image, and correcting a track route of the self-moving robot returning to the charging pile; if the first distance is smaller than the preset distance, a second current environment image and a first depth image are obtained, first depth information of the charging pile is obtained according to the first depth image, and a track route of the self-moving robot returning to the charging pile is corrected according to the first depth information and the second current environment image. Therefore, the image processing technology is used, the pile return charging process of the self-moving robot is divided into two stages, different track correction methods are used in the two stages, and the accuracy of the self-moving robot return charging path can be improved on the premise of not depending on the RTK positioning technology.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to an automatic charging method and device for a self-mobile robot. Background Art

[0002] In the prior art, self-mobile robots (such as lawn mowers) mostly use real-time kinematic (RTK) positioning for autonomous navigation. Usually, the charging pile is set at a location with good RTK signal on the lawn, so that the self-mobile robot can accurately return to the pile for charging. However, in indoor environments or places with poor RTK signals, the positioning accuracy of the self-mobile robot cannot be guaranteed, resulting in a low charging return rate of the self-mobile robot; and after the user changes the position of the charging pile, the corresponding error between the map position and the actual position of the charging pile occurs, and the self-mobile robot cannot accurately return to the pile for charging. Summary of the Invention

[0003] This application provides a method and device for automatic charging of a self-mobile robot. Using image processing technology, the charging process of the self-mobile robot returning to the pile is divided into two stages, and different trajectory correction methods are used in the two stages, which can improve the accuracy of the charging path of the self-mobile robot without relying on RTK positioning technology, and reduce the calculation amount and detection time.

[0004] In a first aspect, this application provides an automatic charging method for a self-mobile robot, which is applied to the self-mobile robot in an automatic charging system. The automatic charging system includes the self-mobile robot and a charging pile. The method includes: obtaining a first distance between the self-mobile robot and the charging pile; if the first distance is greater than a preset distance, obtaining a first current environmental image, comparing the first current environmental image with a preset reference environmental image, and correcting the trajectory of the self-mobile robot returning to the charging pile; if the first distance is less than the preset distance, obtaining a second current environmental image and a first depth image, obtaining first depth information of the charging pile according to the first depth image, and correcting the trajectory of the self-mobile robot returning to the charging pile according to the first depth information and the second current environmental image.

[0005] In some embodiments, the reference environmental image is an image obtained when the self-mobile robot first leaves the charging pile. A feature point library is constructed based on the reference environmental image, and the feature point library stores key feature point information in the reference environmental image.

[0006] In some embodiments, if the first distance is greater than a preset threshold, a first current environmental image is acquired, and the first current environmental image is compared with the preset reference environmental image. Correcting the trajectory of the self - mobile robot returning to the charging pile further includes: obtaining the first current two - dimensional coordinates of the first feature points in the first current environmental image; obtaining the first initial three - dimensional coordinates of the first feature points from the feature point library; projecting the first initial three - dimensional coordinates onto the two - dimensional image plane through the internal parameter matrix and the external parameter matrix of the automatic charging system to obtain the projected first initial two - dimensional coordinates; and correcting the trajectory of the self - mobile robot returning to the charging pile by comparing the first current two - dimensional coordinates of the first feature points with the first initial two - dimensional coordinates of the first feature points.

[0007] In some embodiments, if the first distance is less than a preset distance, a second current environmental image and a first depth image are acquired. Obtaining the first depth information of the charging pile according to the first depth image, and correcting the trajectory of the self - mobile robot returning to the charging pile according to the first depth information and the position information of the self - mobile robot further includes: obtaining the first position, the first contour and the rotation state of the charging pile based on the second current environmental image; calculating the second current three - dimensional coordinates and the rotation angle of the charging pile relative to the self - mobile robot according to the first position, the first contour, the rotation state and the first depth information of the charging pile; and correcting the trajectory of the self - mobile robot returning to the charging pile according to the second current three - dimensional coordinates and the rotation angle.

[0008] In some embodiments, the first feature points include environmental feature points and charging - pile feature points. Correcting the trajectory of the self - mobile robot returning to the charging pile by comparing the first current two - dimensional coordinates of the first feature points with the first initial two - dimensional coordinates of the first feature points includes: obtaining a first positional relationship between the current two - dimensional coordinates of the charging - pile feature points and the current two - dimensional coordinates of the environmental feature points, where the first current two - dimensional coordinates include the current two - dimensional coordinates of the charging - pile feature points and the current two - dimensional coordinates of the environmental feature points; and obtaining a second positional relationship between the initial two - dimensional coordinates of the charging - pile feature points and the initial two - dimensional coordinates of the environmental feature points, where the first initial two - dimensional coordinates include the initial two - dimensional coordinates of the charging - pile feature points and the initial two - dimensional coordinates of the environmental feature points; determining whether the charging pile has been moved according to the difference between the first positional relationship and the second positional relationship; if the charging pile has been moved, obtaining the first current three - dimensional coordinates of the charging pile in the first current environmental image, and correcting the trajectory of the self - mobile robot according to the first current three - dimensional coordinates and the first initial three - dimensional coordinates.

[0009] In some embodiments, obtaining the first current three-dimensional coordinates of the charging pile in the first current environmental image includes: identifying the second position and the second contour of the charging pile in the first current environmental image; obtaining a second depth image, and obtaining the second depth information of the charging pile according to the second depth image; calculating the first current three-dimensional coordinates of the charging pile relative to the self-mobile robot according to the second position, the second contour and the second depth information of the charging pile.

[0010] In some embodiments, the first initial three-dimensional coordinates are converted into the first initial two-dimensional coordinates by the following formula: p u = K·(R·P u + T) where p u is the first initial two-dimensional coordinate, K is the internal parameter matrix of the camera, R is the rotation matrix, T is the translation vector, and P u is the first initial three-dimensional coordinate.

[0011] In some embodiments, calculating the second current three-dimensional coordinates and the rotation angle of the charging pile relative to the self-mobile robot according to the first position, the first contour, the rotation state and the first depth information of the charging pile includes: determining the second current three-dimensional coordinates of the charging pile according to the first position, the first contour and the first depth information; obtaining the initial contour of the charging pile in the reference environmental image; determining the rotation state according to the initial contour and the first contour; if the rotation state is rotated, calculating the rotation angle according to the rotation matrix and the first depth information; if the rotation state is not rotated, determining that the rotation angle is 0.

[0012] In some embodiments, calculating the rotation angle according to the rotation matrix and the first depth information includes: calculating the rotation angles of the self-mobile robot on the X-axis, Y-axis and Z-axis according to the rotation matrix; adjusting the rotation angle of the self-mobile robot according to the rotation angle of each axis; wherein, the rotation angle of each axis is calculated by the following formula: α = atan2(R 32 , R 33 ), β = atan2(-R 31 , R 32 2 + R 33 2 ), γ = atan2(R 21 , R 11 ), where α, β, γ are the rotation angles around the X, Y, Z axes respectively, R is the rotation matrix, R 31 represents the element of the third row and the first column of the rotation matrix, and R 32 represents the element of the third row and the second column of the rotation matrix, and R33 Represents the element in the third row and third column.

[0013] In a second aspect, the present application provides an automatic charging device for a self - mobile robot, which is applied to the self - mobile robot in an automatic charging system. The automatic charging system includes the self - mobile robot and a charging pile. The device includes: an acquisition unit for acquiring a first distance between the self - mobile robot and the charging pile; a processing unit for, if the first distance is greater than a preset distance, acquiring a first current environmental image, comparing the first current environmental image with a preset reference environmental image, and correcting the trajectory of the self - mobile robot returning to the charging pile; and, if the first distance is less than the preset distance, acquiring a second current environmental image and a first depth image, obtaining first depth information of the charging pile according to the first depth image, and correcting the trajectory of the self - mobile robot returning to the charging pile according to the first depth information and the second current environmental image.

[0014] In a third aspect, the present application provides an automatic charging system, including a self - mobile robot and a charging pile. The self - mobile robot is configured to execute the step instructions in any one of the methods in the first aspect.

[0015] In a fourth aspect, the present application provides a self - mobile robot, including a control module for executing the step instructions in any one of the methods in the first aspect; a driving module for responding to the driving signal of the control module to adjust the traveling speed and direction of the self - mobile robot; and an operation module including a mowing module, where the mowing module is used to adjust the height and rotation speed of the cutting cutter head.

[0016] In a fifth aspect, the present application provides a self - mobile robot, including a processor and a memory. A computer program is stored in the memory. When the processor calls the computer program in the memory, it executes the step instructions in any one of the methods in the first aspect.

[0017] It can be seen that in the embodiments of the present application, the first distance between the mobile robot and the charging pile is obtained; if the first distance is greater than the preset distance, the first current environment image is obtained, and the first current environment image is compared with the preset reference environment image to correct the trajectory of the mobile robot returning to the charging pile; if the first distance is less than the preset distance, the second current environment image and the first depth image are obtained, the first depth information of the charging pile is obtained according to the first depth image, and the trajectory of the mobile robot returning to the charging pile is corrected according to the first depth information and the second current environment image. Therefore, compared with the method of using real-time kinematic (RTK) positioning technology in the prior art, the present application uses image processing technology to divide the process of the mobile robot returning to the charging pile into two stages, and different trajectory correction methods are used in the two stages, which can improve the accuracy of the return charging path of the mobile robot without relying on RTK positioning technology, as well as reduce the amount of calculation and detection time. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to these drawings. Among them,

[0019] Figure 1 is a schematic structural diagram of the automatic charging system provided by the embodiment of the present application;

[0020] Figure 2 is an example diagram of the mobile robot provided by the embodiment of the present application;

[0021] Figure 3 is a schematic flow diagram of an automatic charging method for a mobile robot provided by the embodiment of the present application;

[0022] Figure 4 is a schematic diagram of a scenario where the mobile robot returns to the charging pile provided by the embodiment of the present application;

[0023] Figure 5 is another schematic diagram of a scenario where the mobile robot returns to the charging pile provided by the embodiment of the present application;

[0024] Figure 6 is a schematic diagram of the relative position relationship between the mobile robot and each feature point provided by the embodiment of the present application;

[0025] Figure 7 is another schematic diagram of the relative position relationship between the mobile robot and each feature point provided by the embodiment of the present application;

[0026] Figure 8It is a functional unit block diagram of an automatic charging device for a self - mobile robot provided by an embodiment of the present application;

[0027] Figure 9 It is a schematic structural diagram of a control module provided by an embodiment of the present application;

[0028] Figure 10 It is a structural block diagram of a self - mobile robot provided by an embodiment of the present application;

[0029] Figure 11 It is a schematic diagram of a trajectory route provided by an embodiment of the present application. Detailed implementation manners

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0031] The terms "first", "second", etc. in the specification and claims of the present application and the above - mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non - exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but in some embodiments also includes steps or units not listed, or in some embodiments also includes other steps or units inherent to these processes, methods, products or devices.

[0032] Referring to "embodiment" in this article means that the specific features, structures or characteristics described in combination with the embodiment can be included in at least one embodiment of the present application. The phrase appears in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0033] The "and / or" in the embodiments of the present application describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent the following three situations: A exists alone; A and B exist simultaneously; B exists alone. Among them, A and B can be singular or plural.

[0034] In the embodiments of the present application, the symbol " / " can indicate that the associated objects before and after are in an "or" relationship. Additionally, the symbol " / " can also represent a division sign, that is, for performing a division operation. For example, A / B can represent A divided by B.

[0035] The "at least one (item)" or its similar expressions in the embodiments of the present application refer to any combination of these items, including any combination of single items (pieces) or plural items (pieces), which means one or more, and multiple means two or more. For example, at least one (item) of a, b, or c can represent the following seven situations: a, b, c, a and b, a and c, b and c, a, b, and c. Among them, each of a, b, and c can be an element or a set containing one or more elements.

[0036] The "equal to" in the embodiments of the present application can be used in combination with "greater than", applicable to the technical solutions adopted when greater than, and can also be used in combination with "less than", applicable to the technical solutions adopted when less than. When "equal to" is used in combination with "greater than", it is not used in combination with "less than"; when "equal to" is used in combination with "less than", it is not used in combination with "greater than".

[0037] In the prior art, the movement strategy is determined by a distance threshold and target detection, without combining environmental features for global path planning, which may cause the robot to easily deviate from the optimal recharge route in a complex environment. When the target data is lost, it only relies on rotating in place or random actions to search for the target, with a large degree of uncertainty, easily resulting in recharge failure or inefficiency. Moreover, without combining depth information, it may be affected by factors such as light changes and obstacle occlusion, resulting in unstable target recognition, affecting the recharge accuracy, and being unable to optimize the recharge path according to the surrounding environmental features, leading to low path adjustment efficiency.

[0038] To solve the above technical problems, the present application provides a method and device for automatic recharge of a self-mobile robot. Using image processing technology, the process of the self-mobile robot recharging at the charging pile is divided into two stages, and different trajectory correction methods are used in the two stages, which can improve the accuracy of the recharge path of the self-mobile robot without relying on RTK positioning technology, as well as reduce the calculation amount and detection time.

[0039] The following uses specific embodiments to elaborate in detail on the technical solutions of the present application and how the technical solutions of the present application solve the above technical problems. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The following will describe the embodiments of the present application in conjunction with the accompanying drawings.

[0040] Please refer to Figure 1 , Figure 1 which is a schematic structural diagram of the automatic recharge system provided by the embodiments of the present application. As Figure 1As shown, the automatic charging system 1 includes a self - mobile robot 10 and a charging pile 20.

[0041] Among them, the self - mobile robot 10 can walk automatically, prevent collisions, and automatically return to the charging pile to charge within a certain range. Exemplarily, the self - mobile robot 10 can be a lawn mowing robot. See Figure 2 , the self - mobile robot 10 can walk automatically, prevent collisions, automatically return to charge within a certain range, has safety detection and battery power detection, has a certain climbing ability, and is especially suitable for lawn mowing and maintenance in places such as family courtyards and public green spaces. Its features are: automatic mowing, grass clippings cleaning, automatic rain avoidance, automatic charging, automatic obstacle avoidance, small in size, electronic virtual fence, network control, etc.

[0042] Specifically, see Figure 2 、 Figure 10 , the self - mobile robot 10 includes a control module 10a1, a drive module 10a2, and an operation module 10a3. Those skilled in the art can understand that Figure 2 、 Figure 10 the electronic device structure shown in does not constitute a limitation on the electronic device. It may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the self - mobile robot 10 also includes components such as a power supply 10a4. Among them:

[0043] The control module 10a1 is the control center of the self - mobile robot 10. See Figure 9 , the control module 10a1 may specifically include a central processing unit 10a11 (Central Process Unit, CPU), a memory 10a12, and a communication bus 10a13. The control module 10a1 also includes components such as input / output ports, timer / counters, digital - to - analog converters, and analog - to - digital converters. The central processing unit 10a11 executes various functions of the lawn mowing robot and processes data by running or executing software programs and / or modules stored in the memory 10a12, and by calling data stored in the memory 10a12; Preferably, the central processing unit 10a11 can integrate an application processor and a modulation - demodulation processor. Among them, the application processor mainly processes the operating system and application programs, etc., and the modulation - demodulation processor mainly processes wireless communication. It can be understood that the above - mentioned modulation - demodulation processor may not be integrated into the central processing unit 10a11.

[0044] The drive module 10a2 is electrically connected to the control module 10a1 and is used to respond to the control signal transmitted by the control module 10a1, adjust the traveling speed and traveling direction of the self - mobile robot 10, and realize the self - movement function of the self - mobile robot 10.

[0045] The operation module 10a3 includes a mowing module 10a31. The mowing module 10a31 is electrically connected to the control module 10a1, and the mowing module 10a31 is configured to adjust the height and rotation speed of the cutting cutter head in response to the control signal transmitted by the control module 10a1, so as to implement the mowing operation.

[0046] The power supply 10a4 can be logically connected to the control module 10a1 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 10a4 can also include any components such as one or more DC or AC power supplies, a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0047] Although not shown, the self-mobile robot 10 may further include a communication module, etc., which will not be elaborated here. The communication module is used for receiving and sending signals during the process of receiving and transmitting information, and realizes signal receiving and sending with a mobile device, a base station or a server by establishing a communication connection with the mobile device, the base station or the server.

[0048] Wherein, the self-mobile robot 10 further includes a 2D camera 101 and a depth camera 102. The 2D camera 101 is used to capture the scene image on the two-dimensional plane to form a two-dimensional photo or video picture. For example, we use a mobile phone to take pictures of scenery and people in daily life. In addition to being able to obtain the appearance information of an object like a 2D camera, the depth camera 102 can accurately sense the depth of the object, provide three-dimensional space information, and can be used for real-time reconstruction of a three-dimensional scene, enabling the system to accurately judge the position, posture, and distance of the object.

[0049] Wherein, the charging pile 20 is used to charge the self-mobile robot 10. In some embodiments, a robot charging port is provided on the self-mobile robot 10, and a charging pile charging port matching the robot charging port is provided on the charging pile 20. By adjusting the position and orientation of the self-mobile robot 10, the robot charging port is aligned with the charging pile charging port.

[0050] In some embodiments, the robot charging port can be a metal sheet, and the charging pile charging port can be a metal contact. The metal contact is electrically connected by contacting the charging metal sheet to achieve contact charging.

[0051] Please refer to Figure 3 , Figure 3 which is a schematic flow chart of an automatic recharging method for a self-mobile robot provided by an embodiment of the present application. The method includes the following steps S301 - step S303:

[0052] Step S301, obtaining a first distance between the self-mobile robot and the charging pile.

[0053] Step S302, if the first distance is greater than a preset distance, obtain a first current environment image, compare the first current environment image with a preset reference environment image, and correct the trajectory of the self-mobile robot returning to the charging pile.

[0054] Among them, when the distance between the robot and the charging pile is relatively far, a first current environment image is obtained through a 2D camera on the self-mobile robot, and image comparison is performed with a preset reference environment image to determine the environmental change, so as to adjust the route of returning to the charging pile and ensure that the robot can accurately return to the charging pile.

[0055] Exemplarily, the value of the first distance can be 3. When the distance between the robot and the charging pile is greater than 3 meters, the first current environment image is captured using the camera, and the first current environment image is compared with the reference environment image.

[0056] Specifically, see Figure 4 , Figure 4 which is a schematic diagram of a scenario for the self-mobile robot to return to the charging pile provided by an embodiment of the present application.

[0057] In some embodiments, the reference environment image is an image obtained when the self-mobile robot first leaves the charging pile. A feature point library is constructed based on the reference environment image, and the feature point library stores key feature point information in the reference environment image.

[0058] In some embodiments, the key feature point information includes the three-dimensional coordinates of each feature point. When the robot first leaves the charging pile, a reference environment image is obtained through 3D detection using a depth camera, and the three-dimensional coordinates of each feature point are obtained based on the reference environment image.

[0059] Among them, the feature points include charging pile feature points and environmental feature points. For the environmental feature points, objects with positions that are not easily moved can be selected as feature points. For example, in an indoor scenario, the corners of walls, doors and windows, and fixed furniture such as wardrobes and bookshelves can be selected as feature points. For example, in an outdoor scenario, buildings, highway trees, etc. can be selected as feature points.

[0060] Step S303, if the first distance is less than a preset distance, obtain a second current environment image and a first depth image, obtain first depth information of the charging pile according to the first depth image, and correct the trajectory of the self-mobile robot returning to the charging pile according to the first depth information and the second current environment image.

[0061] Among them, when the distance between the robot and the charging pile is relatively close, a second current environmental image is obtained through a 2D camera, the depth information of the charging pile is obtained through a depth camera, the accurate position of the robot is calculated based on the second current environmental image and the depth information, and then the trajectory route for returning to charge is planned to ensure precise docking with the charging pile.

[0062] When specifically implemented, see Figure 5 , Figure 5 which is another schematic diagram of the scene of the self-mobile robot returning to the pile for charging provided by the embodiment of the present application.

[0063] It can be seen that in the present application, when the distance between the robot and the charging pile is relatively far, a first current environmental image is obtained through a 2D camera, and the trajectory route of the robot is corrected according to the first current environmental image to ensure the preliminary accuracy of the robot's return path. When the distance between the robot and the charging pile is relatively close, the depth information of the charging pile is obtained through a depth camera, and the trajectory route of the robot is accurately corrected according to the depth information to ensure precise docking with the charging pile. The above method can greatly reduce the system's dependence on 3D detection, and can reduce the calculation amount and detection time while ensuring the calculation accuracy.

[0064] It can be seen that in the embodiment of the present application, the first distance between the self-mobile robot and the charging pile is obtained; if the first distance is greater than the preset distance, the first current environmental image is obtained, and the first current environmental image is compared with the preset reference environmental image to correct the trajectory route of the self-mobile robot returning to the charging pile; if the first distance is less than the preset distance, the second current environmental image and the first depth image are obtained, the first depth information of the charging pile is obtained according to the first depth image, and the trajectory route of the self-mobile robot returning to the charging pile is corrected according to the first depth information and the position information of the self-mobile robot. Therefore, compared with the method of using real-time kinematic (RTK) positioning technology in the prior art, the present application uses image processing technology to divide the process of the self-mobile robot returning to the pile for charging into two stages, and different trajectory correction methods are used in the two stages, which can improve the accuracy of the return charging path of the self-mobile robot without relying on RTK positioning technology, and reduce the calculation amount and detection time.

[0065] In some embodiments, if the first distance is greater than a preset threshold, a first current environmental image is acquired, and the first current environmental image is compared with the preset reference environmental image. Modifying the trajectory of the self-mobile robot returning to the charging pile further includes: acquiring the first current two-dimensional coordinates of the first feature points in the first current environmental image; acquiring the first initial three-dimensional coordinates of the first feature points from the feature point library; projecting the first initial three-dimensional coordinates onto a two-dimensional image plane through the internal parameter matrix and the external parameter matrix of the automatic charging system to obtain the projected first initial two-dimensional coordinates; and modifying the trajectory of the self-mobile robot returning to the charging pile by comparing the first current two-dimensional coordinates of the first feature points with the first initial two-dimensional coordinates of the first feature points.

[0066] Wherein, in this embodiment, by comparing the differences between the position coordinates of the first feature points in the first current environmental image and the first feature points in the feature point library, the position coordinates of the robot in the coordinate system corresponding to the reference environmental image can be determined, and then the trajectory of the robot returning to the charging pile can be planned according to the positional relationship between the robot and the charging pile.

[0067] Specifically, see Figure 11 , assuming that the robot takes a reference environmental image at position A where it leaves the charging pile, establishes a coordinate system with the charging pile as the origin, acquires the first initial three-dimensional coordinates of each feature point, converts the first initial three-dimensional coordinates into the first initial two-dimensional coordinates through coordinate transformation technology, and establishes a first coordinate system. When the robot returns to charge, it takes a first current environmental image at position B which is at a first distance from the charging pile, and establishes a coordinate system with the charging pile as the origin, and acquires the first current two-dimensional coordinates of each feature point. In the case where the charging pile and the feature points do not move, by translating and rotating the first current two-dimensional coordinates to coincide with the first initial two-dimensional coordinates, translation and rotation parameters are obtained, and the two-dimensional coordinates of the robot in the first coordinate system are determined according to the translation and rotation parameters, as shown at position B in Figure 11 . The robot determines the return charging trajectory L1 according to the two-dimensional coordinates of position B.

[0068] Of course, the return charging trajectory L1 is not limited to that shown in Figure 11 , and other routes can also be designed, and the present application does not make specific limitations thereto. Among them, the factors affecting the robot's trajectory mainly include: the moving direction of the robot and the relative displacement in that moving direction. The robot can flexibly adjust the moving direction and the relative displacement according to the two-dimensional coordinates in the first coordinate system to ensure that the robot can accurately return to the charging pile position.

[0069] Among them, the factors affecting the success of the robot's recharge also include the robot's posture. The robot's posture refers to the posture formed by the robot under different working conditions, usually determined by factors such as the angles of each joint of the robot, the orientation of the fuselage, and the extension state of components. Exemplarily, the mowing robot can adjust the height of the entire fuselage according to different mowing heights, or, when the mowing robot avoids obstacles, it can avoid obstacles by adjusting the forward direction of the wheels and the rotation angle of the fuselage. When the robot needs to return to the charging pile for charging, the robot's posture needs to be restored to the initial state so that the robot can be successfully stored in the charging pile.

[0070] Among them, the internal parameter matrix of the automatic recharge system mainly refers to the internal parameter matrix of the camera, and the external parameter matrix includes the rotation matrix and the translation matrix. The first initial three-dimensional coordinates are converted into the first initial two-dimensional coordinates through the following formula (1):

[0071] p i = K·(R·P i + T) Formula (1)

[0072] Among them, p i is the first initial two-dimensional coordinate, and P i is the first initial three-dimensional coordinate. K is the internal parameter matrix of the camera, which describes the characteristics of the camera itself, such as the focal length, the position of the principal point, etc., and determines how the three-dimensional space points are projected onto the two-dimensional image plane. R is the rotation matrix, which represents the rotation relationship between the robot coordinate system and the camera coordinate system. Through the rotation matrix, the direction of the three-dimensional points in space can be adjusted to match the perspective of the camera. T is the translation matrix, which is used to represent the translation relationship between the robot coordinate system and the camera coordinate system, that is, the position offset of the three-dimensional points in space. Through the operation of these three parameters and the first initial three-dimensional point coordinates P i finally, the coordinates of this point on the two-dimensional image plane are obtained, that is, the first initial two-dimensional coordinate. In this embodiment, the robot can accurately convert the three-dimensional coordinates in the feature point library to the two-dimensional plane coordinate system that is the same as the image captured by the camera, which provides an important basis for the robot to judge its own position, environmental changes, and subsequent path correction, and improves the accuracy and consistency of the robot's environmental perception.

[0073] Exemplarily, assume the internal parameter matrix of the camera This means that the focal length of the camera is 800 in the x and y directions, and the position of the principal point in the image plane is (320, 240). The rotation matrix The translation matrix Assume a first initial three-dimensional coordinate P1 = (2, 3, 4), then through the formula p i = K·(R·P iWith the above parameters and (+T), a first initial two-dimensional coordinate p1 = (582.48, 790.95) is calculated.

[0074] Among them, the internal parameter matrix is usually determined, while the rotation matrix and translation matrix need to be adjusted and tested until the optimal pose is found, so that the first initial two-dimensional coordinate after projecting the first initial three-dimensional coordinate is closer to the actual two-dimensional coordinates of each feature point in the reference environment image.

[0075] Specifically, when implemented, the rotation matrix and translation matrix are determined by minimizing the reprojection error: The formula for the minimum reprojection error is:

[0076]

[0077] where E is the reprojection error, p i is the first initial two-dimensional coordinate, that is, the reprojection coordinate calculated by the coordinate transformation method shown in the above embodiments, is the actual two-dimensional coordinate of each feature point in the reference environment image. The reprojection error is used to characterize the degree of difference between the actual two-dimensional coordinate and the first initial two-dimensional coordinate projected onto the image plane after coordinate transformation. By minimizing this error, the optimal pose (i.e., the rotation matrix and translation matrix) can be found, so that the projected first two-dimensional coordinate is more likely to be close to the actual two-dimensional coordinate.

[0078] Furthermore, the method for determining whether the charging pile has been moved is as follows: In some embodiments, the first feature points include environmental feature points and charging pile feature points. By comparing the first current two-dimensional coordinates of the first feature points with the first initial two-dimensional coordinates of the first feature points, correcting the trajectory of the self-mobile robot returning to the charging pile includes: obtaining a first positional relationship between the current two-dimensional coordinates of the charging pile feature points and the current two-dimensional coordinates of the environmental feature points, where the first current two-dimensional coordinates include the current two-dimensional coordinates of the charging pile feature points and the current two-dimensional coordinates of the environmental feature points; and obtaining a second positional relationship between the initial two-dimensional coordinates of the charging pile feature points and the initial two-dimensional coordinates of the environmental feature points, where the first initial two-dimensional coordinates include the initial two-dimensional coordinates of the charging pile feature points and the initial two-dimensional coordinates of the environmental feature points; determining whether the charging pile has been moved according to the difference between the first positional relationship and the second positional relationship; if the charging pile has been moved, obtaining the first current three-dimensional coordinates of the charging pile in the first current environment image, and correcting the trajectory of the self-mobile robot according to the first current three-dimensional coordinates and the first initial three-dimensional coordinates.

[0079] Among them, the first feature points include environmental feature points and charging pile feature points. Compared with only comparing the first initial two-dimensional coordinates and the first current two-dimensional coordinates of the charging pile feature points to determine the position state of the charging pile, the relative position relationship between the environmental feature points and the charging pile feature points can improve the accuracy of determining the position state of the charging pile and avoid the possibility of misjudgment.

[0080] For specific implementation, please refer to Figure 6 , Figure 6 which is a schematic diagram of the relative position relationship between the self-mobile robot and each feature point provided by the embodiment of the present application. See Figure 6 , assuming that the first feature points include feature point 1, feature point 2, and feature point 3. When the self-mobile robot is at the first position, the two-dimensional coordinates of the above three feature points are obtained according to the current environmental image. When the self-mobile robot is at the second position, see Figure 7 , and the two-dimensional coordinates of the above three feature points are obtained according to the current environmental graphic. In Figure 6 and Figure 7 the two scenarios, the positions of the self-mobile robot are different, resulting in different two-dimensional coordinates of the three feature points. However, according to the position coordinates of the three feature points, the position relationship of the three feature points can be determined to be the same. Similarly, on the premise that the charging pile has not been moved, the first position relationship of each feature point in the first current environmental image is theoretically the same as the second position relationship of each feature point in the reference environmental image. If the difference between the first position relationship and the second position relationship exceeds the preset range, it is determined that the charging pile has been moved; if the difference between the first position relationship and the second position relationship does not exceed the preset range, it is determined that the charging pile has not been moved.

[0081] In some embodiments, if the charging pile is moved, the first current three-dimensional coordinates of the charging pile in the first current environmental image are obtained, and the trajectory route of the self-mobile robot is corrected according to the first current three-dimensional coordinates and the first initial three-dimensional coordinates.

[0082] Among them, if the charging pile is moved, switch to the 3D calculation mode to determine the difference between the first current three-dimensional coordinates of the charging pile in the first current environmental image and the first initial three-dimensional coordinates of the charging pile in the feature point library, and correct the trajectory route of the robot according to the difference.

[0083] Among them, only the first current two-dimensional coordinates of each feature point are included in the first current environment image. The first current three-dimensional coordinates of the charging pile in the first current environment image need to be converted from the first current two-dimensional coordinates to the first current three-dimensional coordinates through coordinate transformation. The specific method includes the following steps: obtaining the first current three-dimensional coordinates of the charging pile in the first current environment image includes: identifying the second position and the second contour of the charging pile in the first current environment image; obtaining a second depth image, and obtaining the second depth information of the charging pile according to the second depth image; calculating the first current three-dimensional coordinates of the charging pile relative to the self-mobile robot according to the second position, the second contour and the second depth information of the charging pile.

[0084] Among them, the second position of the charging pile is the first current two-dimensional coordinates of the charging pile in the first current environment image. The border coordinates of the charging pile are obtained through the second contour, and then the position coordinates of the charging pile relative to the robot are calculated according to the first current two-dimensional coordinates, the border coordinates and the second depth information.

[0085] In some embodiments, if the charging pile is not moved, the trajectory of the self-mobile robot is corrected according to the first current two-dimensional coordinates and the first initial two-dimensional coordinates of the charging pile.

[0086] Among them, correcting the trajectory is actually correcting the position coordinates of the robot, and re-planning the traveling direction and displacement of the robot according to the corrected position coordinates. The principle is that when the position coordinates of the robot are inaccurate, in the same coordinate system, the coordinates of the feature points at the same position are deviated. Therefore, the current coordinate system can be adjusted according to the difference in the position coordinates of the same feature points, and then the accurate positioning of the robot can be obtained. After the positioning of the robot is accurate, an accurate charging-back trajectory can be planned.

[0087] It can be seen that in this embodiment, when the distance between the robot and the charging pile is large, the trajectory is directly corrected by the difference in the position coordinates of the first feature points in the first current environment image and the reference environment image, reducing the calculation amount and detection time, and improving the efficiency of the robot's charging-back.

[0088] In some embodiments, if the first distance is less than a preset distance, for the acquired second current environmental image and the first depth image, the first depth information of the charging pile is obtained according to the first depth image, and the trajectory route for the self-mobile robot to return to the charging pile is corrected based on the first depth information and the position information of the self-mobile robot, further including: obtaining the first position, the first contour and the rotation state of the charging pile based on the second current environmental image; calculating the second current three-dimensional coordinates and the rotation angle of the charging pile relative to the self-mobile robot according to the first position, the first contour, the rotation state and the first depth information of the charging pile; and correcting the trajectory route for the self-mobile robot to return to the charging pile according to the second current three-dimensional coordinates and the rotation angle.

[0089] Among them, obtaining the first position, the first contour and the rotation state of the charging pile based on the second current environmental image includes: using a trained deep learning object detection model to detect the outer contour and position of the charging pile; outputting the first position and the first contour; and judging whether the charging pile rotates according to the first contour by using the deep learning object detection model to determine the rotation state of the charging pile.

[0090] Among them, the first position is the second current two-dimensional coordinates of the charging pile in the second current environmental image.

[0091] Among them, the deep learning object detection model includes, but is not limited to, YOLO, SSD or Faster R-CNN models. Specifically, a charging pile image data set in different scenarios is constructed, covering different angles, illuminations, occlusions and rotation conditions of the charging pile, so as to ensure that the deep learning object detection model can accurately identify the position and pose of the charging pile in different environments.

[0092] Among them, the rotation state refers to the state of whether the charging pile in the second current environmental image rotates relative to the charging pile in the reference environmental image. If it is in a rotation state, further 3D detection is performed, and the accurate rotation angle is determined in combination with the depth image. The rotation angle includes the three-axis attitude RPY (R: Roll, roll angle, P: pitch, pitch angle, Y: Yaw, horizontal rotation) and / or the three-axis position XYZ (representing the X-axis, Y-axis, and X-axis), etc.; if it is in a non-rotation state, the rotation angle is directly determined to be 0.

[0093] Among them, based on the known camera parameters and the depth information in the depth image, combined with the 2D bounding box (i.e., the first contour) output by the deep learning object detection model, the three-dimensional coordinates of the charging pile in the three-dimensional space and the angle relative to the robot are calculated. According to the determined three-dimensional coordinates and rotation angle of the charging pile, the robot will automatically adjust its own position and posture to ensure the accuracy when docking with the charging pile. This process will perform fine angle adjustment according to the depth information to correct the small errors that may occur in the training model.

[0094] In some embodiments, calculating the second three-dimensional coordinates and the rotation angle of the charging pile relative to the self-mobile robot according to the first position, the first contour, the rotation state, and the first depth information of the charging pile includes: determining the second three-dimensional coordinates of the charging pile according to the first position, the first contour, and the first depth information; obtaining the initial contour of the charging pile in the reference environment image; determining the rotation state according to the initial contour and the first contour; if the rotation state is rotated, calculating the rotation angle according to the rotation matrix and the first depth information; if the rotation state is not rotated, determining the rotation angle to be 0.

[0095] Among them, the first depth information refers to the Z-axis distance between the charging pile and the robot. Combining the two-dimensional visual information (including the first position and the first contour of the charging pile), the relative position of the charging pile in the 3D space can be calculated.

[0096] Specifically, based on the bounding box and the center point coordinates of the charging pile, combined with the depth information, the current three-dimensional coordinates P of the robot are obtained. current and the current three-dimensional coordinates P of the charging pile relative to the robot. charging , the displacement that the robot needs to move:

[0097] ΔP = P charging - P current Formula (3)

[0098] P curren t = (x current , y current , z current )

[0099] P c h arg i ng = (x charging , y charging , z chargibg )

[0100] Among them, (x current , y current ) is the current two-dimensional coordinates of the robot, zcurrent is the depth information of the robot; (x charging , y charging ) is the current three-dimensional coordinates of the charging pile, and z charging is the depth information of the charging pile.

[0101] Among them, the specific method for determining the rotation state includes: obtaining the initial contour of the charging pile in the reference environment image; determining the rotation state by comparing the initial contour and the first contour. When the rotation state of the charging pile is determined to be rotated, the specific rotation angle is calculated through the rotation matrix and the first depth information; when the rotation state is determined to be unrotated, the rotation angle is determined to be 0.

[0102] In some embodiments, calculating the rotation angle according to the rotation matrix and the first depth information includes: calculating the rotation angles of the self-mobile robot on the X-axis, Y-axis, and Z-axis according to the rotation matrix; adjusting the rotation angle of the self-mobile robot according to the rotation angle of each axis; among them, the rotation angle of each axis is calculated by the following formula:

[0103] α = atan2(R 32 , R 33 )

[0104] β = atan2(-R 31 , R 32 2 + R 33 2 )

[0105] γ = atan2(R 21 , R 11 )

[0106] Among them, α, β, and γ are the rotation angles around the X, Y, and Z axes respectively, R is the rotation matrix, R 31 represents the element in the first column of the third row of the rotation matrix, R 32 represents the element in the second column of the third row, and R 33 represents the element in the third column of the third row.

[0107] It can be seen that in this embodiment, when the distance between the robot and the charging pile is relatively close, the 2D camera is started to capture the second current environment image of the charging pile, and the captured second current environment image is input into the trained deep learning object detection model to output the position information of the charging pile and the approximate range of the rotation angle. Only after detecting that the charging pile rotates, precise 3D calculation is performed. Before that, the approximate rotation angle is extracted using the 2D features of the image, avoiding high-computation 3D detection throughout the detection process and saving computing resources.

[0108] In the embodiment of the present application, the process of the robot returning to the charging pile is divided into two stages. When the robot is far from the charging pile, a rough positioning method is adopted: the current environment image of the robot is obtained through a 2D camera, and the return charging trajectory of the robot is corrected by comparing the position relationship of each feature point in the current environment image with that in the reference environment image; when the robot is close to the charging pile, the image of the charging pile is obtained by using the 2D camera, and a rough rotation angle range is output through a trained model; if the charging pile is in a rotating state, a depth camera is started to obtain the 3D information of the charging pile, and the accurate rotation angle and posture are calculated by combining the depth information to complete the posture correction; according to the result of the fine positioning, the robot dynamically adjusts its own position and angle to ensure accurate docking with the charging pile; if the charging pile is in a non-rotating state, the trajectory correction is continued according to the rough positioning method in the first stage.

[0109] Therefore, through two-stage positioning, the dependence of the system on 3D detection can be significantly reduced, and the calculation amount and detection time can be reduced. Although the frequency of 3D detection is reduced, high-precision posture correction can still be guaranteed at critical moments to ensure the reliability of docking. And this solution is applicable in different scenarios, and can handle both ordinary lighting environments and docking problems in complex postures.

[0110] The above mainly introduces the solution of the embodiment of the present application from the perspective of the execution process of the method side. It can be understood that in order for the server to implement the above functions, it includes the corresponding hardware structure and / or software module for executing each function. Those skilled in the art should easily realize that, combined with the units and algorithm steps of each example described in the embodiments provided in this article, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0111] The embodiment of the present application can divide the server into functional units according to the above method examples. For example, each functional unit can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated unit can be implemented in the form of hardware or in the form of a software program module. It should be noted that the division of units in the embodiment of the present application is illustrative, only a logical function division, and there may be other division methods in actual implementation.

[0112] In the case of adopting an integrated unit, please refer to Figure 8 , Figure 8The block diagram of the functional units of an automatic charging device for a self - mobile robot provided by an embodiment of the present application is applied to the self - mobile robot in an automatic charging system. The automatic charging system includes the self - mobile robot and a charging pile. The automatic charging device 8 includes:

[0113] An acquisition unit 801, configured to acquire a first distance between the self - mobile robot and the charging pile;

[0114] A processing unit 802, configured to, if the first distance is greater than a preset distance, acquire a first current environmental image, compare the first current environmental image with a preset reference environmental image, and correct the trajectory of the self - mobile robot returning to the charging pile; and, if the first distance is less than the preset distance, acquire a second current environmental image and a first depth image, obtain first depth information of the charging pile according to the first depth image, and correct the trajectory of the self - mobile robot returning to the charging pile according to the first depth information and the position information of the self - mobile robot.

[0115] It can be seen that in the embodiment of the present application, a first distance between the self - mobile robot and the charging pile is acquired; if the first distance is greater than the preset distance, a first current environmental image is acquired, the first current environmental image is compared with the preset reference environmental image, and the trajectory of the self - mobile robot returning to the charging pile is corrected; if the first distance is less than the preset distance, a second current environmental image and a first depth image are acquired, first depth information of the charging pile is obtained according to the first depth image, and the trajectory of the self - mobile robot returning to the charging pile is corrected according to the first depth information and the second current environmental image. Therefore, compared with the method of using real - time kinematic (RTK) positioning technology in the prior art, the present application uses image processing technology to divide the process of the self - mobile robot returning to the charging pile into two stages, and different trajectory correction methods are used in the two stages, which can improve the accuracy of the self - mobile robot's charging path without relying on RTK positioning technology, and reduce the calculation amount and detection time.

[0116] In some embodiments, the reference environmental image is an image acquired when the self - mobile robot first leaves the charging pile. A feature point library is constructed based on the reference environmental image, and the feature point library stores key feature point information in the reference environmental image.

[0117] In some embodiments, when the first distance is greater than a preset threshold, a first current environmental image is acquired, and the first current environmental image is compared with the preset reference environmental image to correct the trajectory of the self-mobile robot returning to the charging pile. The processing unit 802 is further configured to: acquire the first current two-dimensional coordinates of the first feature points in the first current environmental image; acquire the first initial three-dimensional coordinates of the first feature points from the feature point library; project the first initial three-dimensional coordinates onto a two-dimensional image plane through the internal parameter matrix and the external parameter matrix of the automatic charging system to obtain the projected first initial two-dimensional coordinates; and correct the trajectory of the self-mobile robot returning to the charging pile by comparing the first current two-dimensional coordinates of the first feature points with the first initial two-dimensional coordinates of the first feature points.

[0118] In some embodiments, when the first distance is less than a preset distance, a second current environmental image and a first depth image are acquired, the first depth information of the charging pile is acquired according to the first depth image, and the trajectory of the self-mobile robot returning to the charging pile is corrected according to the first depth information and the position information of the self-mobile robot. The processing unit 802 is further configured to: acquire the first position, the first contour, and the rotation state of the charging pile based on the second current environmental image; calculate the second current three-dimensional coordinates and the rotation angle of the charging pile relative to the self-mobile robot according to the first position, the first contour, the rotation state, and the first depth information of the charging pile; and correct the trajectory of the self-mobile robot returning to the charging pile according to the second current three-dimensional coordinates and the rotation angle.

[0119] In some embodiments, the first feature points include environmental feature points and charging pile feature points. When the processing unit 802 corrects the trajectory of the self-mobile robot returning to the charging pile by comparing the first current two-dimensional coordinates of the first feature points with the first initial two-dimensional coordinates of the first feature points, it includes: obtaining a first positional relationship between the current two-dimensional coordinates of the charging pile feature points and the current two-dimensional coordinates of the environmental feature points, where the first current two-dimensional coordinates include the current two-dimensional coordinates of the charging pile feature points and the current two-dimensional coordinates of the environmental feature points; and obtaining a second positional relationship between the initial two-dimensional coordinates of the charging pile feature points and the initial two-dimensional coordinates of the environmental feature points, where the first initial two-dimensional coordinates include the initial two-dimensional coordinates of the charging pile feature points and the initial two-dimensional coordinates of the environmental feature points; determining whether the charging pile has been moved according to the difference between the first positional relationship and the second positional relationship; if the charging pile has been moved, obtaining the first current three-dimensional coordinates of the charging pile in the first current environmental image, and correcting the trajectory of the self-mobile robot according to the first current three-dimensional coordinates and the first initial three-dimensional coordinates.

[0120] In some embodiments, when the processing unit 802 obtains the first current three-dimensional coordinates of the charging pile in the first current environmental image, it includes: identifying a second position and a second contour of the charging pile in the first current environmental image; obtaining a second depth image, and obtaining second depth information of the charging pile according to the second depth image; calculating the first current three-dimensional coordinates of the charging pile relative to the self-mobile robot according to the second position, the second contour and the second depth information of the charging pile.

[0121] In some embodiments, the first initial three-dimensional coordinates are converted into the first initial two-dimensional coordinates through the following formula: p i = K·(R·P i + T) where p i is the first initial two-dimensional coordinate, K is the internal parameter matrix of the camera, R is the rotation matrix, T is the translation vector, and P i is the first initial three-dimensional coordinate.

[0122] In some embodiments, the processing unit 802 calculates the second current three-dimensional coordinates and the rotation angle of the charging pile relative to the self-mobile robot according to the first position, the first contour, the rotation state, and the first depth information of the charging pile, including: determining the second current three-dimensional coordinates of the charging pile according to the first position, the first contour, and the first depth information; obtaining the initial contour of the charging pile in the reference environment image; determining the rotation state according to the initial contour and the first contour; if the rotation state is rotated, calculating the rotation angle according to the rotation matrix and the first depth information; if the rotation state is not rotated, determining that the rotation angle is 0.

[0123] In some embodiments, the processing unit 802 calculates the rotation angle according to the rotation matrix and the first depth information, including: calculating the rotation angles of the self-mobile robot on the X-axis, Y-axis, and Z-axis according to the rotation matrix; adjusting the rotation angle of the self-mobile robot according to the rotation angle of each axis; wherein, the rotation angle of each axis is calculated by the following formula: α = atan2(R 32 , R 33 ), β = atan2(-R 31 , R 32 2 +R 33 2 ), γ = atan2(R 21 , R 11 ), where α, β, and γ are the rotation angles around the X, Y, and Z axes respectively, R is the rotation matrix, R 31 represents the element in the first column of the third row of the rotation matrix, R 32 represents the element in the second column of the third row, and R 33 represents the element in the third column of the third row.

[0124] The embodiments of the present application provide a computer-readable storage medium, on which a computer program / instructions are stored, and when the computer program / instructions are executed by a processor, the steps of the method according to any possible embodiment are implemented.

[0125] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0126] In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0127] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the above division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical or other form.

[0128] The units described above as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0129] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0130] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of the present application. The foregoing memory includes: various media such as a USB flash drive, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disc that can store program codes.

[0131] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable memory. The memory may include: a flash drive, a read-only memory (abbreviation: ROM), a random access memory (abbreviation: RAM), a magnetic disk, an optical disk, etc.

[0132] The above has introduced the embodiments of the present application in detail. Specific examples are used herein to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation on the present application.

Claims

1. An automatic charging method for a self-mobile robot, characterized in that, A self - mobile robot applied to an automatic charging system, the automatic charging system includes the self - mobile robot and a charging pile, and the method includes: Obtain a first distance between the self - mobile robot and the charging pile; If the first distance is greater than a preset distance, obtain a first current environment image, compare the first current environment image with a preset reference environment image, and correct the trajectory of the self - mobile robot returning to the charging pile; If the first distance is less than the preset distance, obtain a second current environment image and a first depth image, obtain first depth information of the charging pile according to the first depth image, and correct the trajectory of the self - mobile robot returning to the charging pile according to the first depth information and the second current environment image.

2. The method according to claim 1, wherein The reference environment image is an image obtained when the self - mobile robot first leaves the charging pile. A feature point library is constructed based on the reference environment image, and the feature point library stores key feature point information in the reference environment image.

3. The method according to claim 1, wherein If the first distance is greater than a preset threshold, obtain a first current environment image, compare the first current environment image with the preset reference environment image, and correcting the trajectory of the self - mobile robot returning to the charging pile further includes: Obtain a first current two - dimensional coordinate of a first feature point in the first current environment image; Obtain a first initial three - dimensional coordinate of the first feature point from the feature point library; Project the first initial three - dimensional coordinate onto a two - dimensional image plane through the internal parameter matrix and external parameter matrix of the automatic charging system to obtain a projected first initial two - dimensional coordinate; Correct the trajectory of the self - mobile robot returning to the charging pile by comparing the first current two - dimensional coordinate of the first feature point with the first initial two - dimensional coordinate of the first feature point.

4. The method according to claim 1, wherein If the first distance is less than the preset distance, obtain the second current environment image and the first depth image, obtain the first depth information of the charging pile according to the first depth image, and correcting the trajectory of the self - mobile robot returning to the charging pile according to the first depth information and the position information of the self - mobile robot further includes: Obtain a first position, a first contour, and a rotation state of the charging pile based on the second current environment image; Calculate a second current three - dimensional coordinate and a rotation angle of the charging pile relative to the self - mobile robot according to the first position, the first contour, the rotation state, and the first depth information of the charging pile; Correct the trajectory of the self - mobile robot returning to the charging pile according to the second current three - dimensional coordinate and the rotation angle.

5. The method according to claim 3, wherein The first feature points include environmental feature points and charging - pile feature points. Correcting the trajectory of the self - mobile robot returning to the charging pile by comparing the first current two - dimensional coordinate of the first feature point with the first initial two - dimensional coordinate of the first feature point includes: Obtain a first positional relationship between the current two-dimensional coordinates of the charging pile feature points and the current two-dimensional coordinates of the environmental feature points, where the first current two-dimensional coordinates include the current two-dimensional coordinates of the charging pile feature points and the current two-dimensional coordinates of the environmental feature points; and, obtain a second positional relationship between the initial two-dimensional coordinates of the charging pile feature points and the initial two-dimensional coordinates of the environmental feature points, where the first initial two-dimensional coordinates include the initial two-dimensional coordinates of the charging pile feature points and the initial two-dimensional coordinates of the environmental feature points; Determine whether the charging pile has been moved according to the difference between the first positional relationship and the second positional relationship; If the charging pile has been moved, obtain the first current three-dimensional coordinates of the charging pile in the first current environmental image, and correct the trajectory of the self-mobile robot according to the first current three-dimensional coordinates and the first initial three-dimensional coordinates.

6. The method according to claim 5, wherein The obtaining the first current three-dimensional coordinates of the charging pile in the first current environmental image includes: Identify the second position and the second contour of the charging pile in the first current environmental image; Obtain a second depth image, and obtain the second depth information of the charging pile according to the second depth image; Calculate the first current three-dimensional coordinates of the charging pile relative to the self-mobile robot according to the second position, the second contour and the second depth information of the charging pile.

7. The method according to claim 3, characterized in that The first initial three-dimensional coordinates are converted into the first initial two-dimensional coordinates through the following formula: p i = K·(R·P i + T) where p i is the first initial two-dimensional coordinate, K is the internal parameter matrix of the camera, R is the rotation matrix, T is the translation vector, and P i is the first initial three-dimensional coordinate.

8. The method according to claim 4, wherein The calculating the second current three-dimensional coordinates and the rotation angle of the charging pile relative to the self-mobile robot according to the first position, the first contour, the rotation state and the first depth information of the charging pile includes: Determine the second current three-dimensional coordinates of the charging pile according to the first position, the first contour and the first depth information; Obtain the initial contour of the charging pile in the reference environmental image; Determine the rotation state according to the initial contour and the first contour; If the rotation state is rotated, calculate the rotation angle according to the rotation matrix and the first depth information; If the rotation state is not rotated, determine that the rotation angle is 0.

9. The method according to claim 8, characterized in that, The calculating the rotation angle according to the rotation matrix and the first depth information includes: Calculate the rotation angles of the self-mobile robot on the X-axis, Y-axis, and Z-axis according to the rotation matrix; Adjust the rotation angle of the self-mobile robot according to the rotation angle of each axis; Wherein, the rotation angle of each axis is calculated through the following formula: α = atan2(R 32 , R 33 ), β = atan2(-R 31 , R 32 2 +R 33 2 ), γ = atan2(R 21 , R 11 ), where α, β, and γ are the rotation angles about the X, Y, and Z axes respectively, R is the rotation matrix, and R 31 represents the element in the first column of the third row of the rotation matrix, and R 32 represents the element in the second column of the third row, and R 33 represents the element in the third column of the third row.

10. An automatic charging device for a self - mobile robot, characterized in that, Applied to a self-mobile robot in an automatic charging system, the automatic charging system includes the self-mobile robot and a charging pile, and the device includes: An obtaining unit, configured to obtain a first distance between the self-mobile robot and the charging pile; A processing unit, configured to, if the first distance is greater than a preset distance, obtain a first current environmental image, compare the first current environmental image with a preset reference environmental image, and correct the trajectory of the self-mobile robot returning to the charging pile; and, if the first distance is less than the preset distance, obtain a second current environmental image and a first depth image, obtain first depth information of the charging pile according to the first depth image, and correct the trajectory of the self-mobile robot returning to the charging pile according to the first depth information and the second current environmental image.

11. An automatic recharge system, characterized in that, It includes a self-mobile robot and a charging pile, and the self-mobile robot is configured to execute the step instructions in the method according to any one of claims 1-9.

12. A self-mobile robot, characterized in that, It includes A control module, configured to execute the step instructions in the method according to any one of claims 1-9; A driving module, configured to adjust the traveling speed and traveling direction of the self-mobile robot in response to a driving signal from the control module; An operation module, including a mowing module, and the mowing module is configured to adjust the height and rotation speed of a cutting cutter head.

13. A self-mobile robot, characterized in that, It includes a processor and a memory, and a computer program is stored in the memory. When the processor calls the computer program in the memory, it executes the step instructions in the method according to any one of claims 1 to 9.

Images