A charging control method, terminal device and storage medium for a robot

Abstract

This application is applicable to the field of image processing technology, and provides a charging control method, a terminal device and a storage medium for a robot. The method includes: first obtaining a first image collected by an infrared camera on the robot, determining a first area of a reflective mark on a charging pile for charging the robot in the first image, determining the pose of the robot relative to the charging pile based on the position of the first area in the first image and the position of the reflective mark on the charging pile, and finally controlling the robot to charge on the charging pile based on the pose of the robot relative to the charging pile. By collecting the first image through the infrared camera and identifying the area of the reflective mark in the first image, this application can avoid the problem of being unable to accurately identify the charging pile when there are objects with similar external structures to the charging pile near the charging pile. This application can accurately obtain the pose of the robot relative to the charging pile, and thus accurately charge the robot.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to a charging control method, terminal device and storage medium of a robot. Background Art

[0002] Most robots are powered by batteries, so that the robot's movements are not restricted by power lines, making the robot move more freely. In addition, in order to further improve the robot's intelligence, the robot can detect the location of the charging pile to achieve the effect of automatic charging.

[0003] At present, most robots are equipped with laser radars, which are used to identify the location of the charging pile. Then, the robot calculates the position relationship between the robot and the charging pile based on the identified location of the charging pile, and then controls the robot to charge automatically. If there is an object with a similar appearance and structure to the charging pile near the charging pile, the laser radar will identify the object as a charging pile, causing the robot to be unable to charge. Summary of the invention

[0004] The embodiments of the present application provide a charging control method, a terminal device, and a storage medium for a robot, which can reduce the problem of the robot being unable to charge due to inaccurate identification of a charging pile.

[0005] In a first aspect, an embodiment of the present application provides a charging control method for a robot, wherein at least one reflective mark is provided on a side wall of a charging pile for charging the robot, and the method comprises:

[0006] Acquire a first image captured by an infrared camera on the robot, wherein the first image includes the reflective mark;

[0007] Determining a first area of ​​the reflective mark in the first image;

[0008] Determine the position of the robot relative to the charging pile based on the position of the first area on the first image and the preset position of the reflective mark on the charging pile;

[0009] Based on the posture of the robot relative to the charging pile, the robot is controlled to charge on the charging pile.

[0010] In a second aspect, an embodiment of the present application provides a charging control device for a robot, comprising:

[0011] An image acquisition module, used to acquire a first image captured by an infrared camera on the robot, wherein the first image includes the reflective mark;

[0012] A first area determination module, used to determine a first area of ​​the reflective mark in the first image;

[0013] A posture determination module, used to determine the posture of the robot relative to the charging pile based on the position of the first area on the first image and the preset position of the reflective mark on the charging pile;

[0014] A control module is used to control the robot to charge on the charging pile based on the posture of the robot relative to the charging pile.

[0015] In a third aspect, an embodiment of the present application provides a terminal device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the charging control method for the robot described in any one of the first aspects above is implemented.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the robot charging control method described in any one of the first aspects above is implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes the robot charging control method described in any one of the above-mentioned first aspects.

[0018] Compared with the prior art, the beneficial effects of the first aspect of the present application are as follows: the present application first obtains the first image captured by the infrared camera on the robot, determines the first area of ​​the reflective mark on the charging pile for charging the robot in the first image, determines the position of the robot relative to the charging pile based on the position of the first area on the first image and the position of the reflective mark on the charging pile, and finally controls the robot to charge on the charging pile based on the position of the robot relative to the charging pile. Compared with the problem of inaccurate identification of the charging pile caused by using a laser radar to determine the position of the charging pile, the present application captures the first image by an infrared camera and identifies the area of ​​the reflective mark in the first image, which can avoid the problem of being unable to accurately identify the charging pile when there is an object with a similar appearance and structure to the charging pile near the charging pile. The present application can accurately obtain the position of the robot relative to the charging pile, and then accurately charge the robot.

[0019] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0021] Figure 1 It is a schematic diagram of the application scenario of the charging control method for a robot provided by an embodiment of the present application;

[0022] Figure 2 It is a schematic flowchart of the charging control method for a robot provided by an embodiment of the present application;

[0023] Figure 3 It is a schematic diagram of the first image provided by an embodiment of the present application;

[0024] Figure 4 It is a schematic flowchart of the method for determining the first area from the first image provided by an embodiment of the present application;

[0025] Figure 5 It is a schematic flowchart of the method for determining the second area provided by an embodiment of the present application;

[0026] Figure 6 It is a schematic flowchart of the method for determining the first area from the second area provided by an embodiment of the present application;

[0027] Figure 7 It is a schematic diagram of the structure of the charging control device for a robot provided by an embodiment of the present application;

[0028] Figure 8 It is a schematic diagram of the structure of the terminal device provided by an embodiment of the present application. Detailed implementation manners

[0029] It should be understood that when used in the description of the present application specification and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0030] It should also be understood that the term "and / or" used in the description of the present application specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0031] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0032] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0033] When using the lidar on the robot to detect the position of the charging pile, it is necessary to use the lidar to scan the charging pile to obtain the point cloud data of the charging pile, and match the scanned point cloud data with the cross-sectional shape of the charging pile at this height to obtain the relative position of the robot and the charging pile. However, if there are moving objects in the environment around the charging pile, the moving objects will interfere with the recognition of the charging pile and cause misrecognition of the charging pile.

[0034] Based on the above problems, this application proposes a charging control method for a robot. By collecting an image of a reflective mark with a preset shape set on the charging pile through an infrared camera, the pose of the robot relative to the charging pile is determined based on the image of the reflective mark, avoiding the problem of inaccurate recognition of the charging pile due to interfering objects around the charging pile.

[0035] Figure 1 FIG. is a schematic diagram of an application scenario of the charging control method for the robot provided in the embodiment of this application. The above charging control method for the robot can be used for the robot to automatically charge using the charging pile. Among them, the infrared camera 10 is used to collect the first image, and the first image includes the reflective mark set on the charging pile. The processor 20 is used to obtain the first image collected by the infrared camera 10, process the first image to obtain the pose of the robot relative to the charging pile, and control the robot to charge on the charging pile according to the pose of the robot relative to the charging pile.

[0036] Figure 2 shows a schematic flowchart of the charging control method for the robot provided in this application. Referring to Figure 2 , the details of this method are as follows:

[0037] S101, obtain a first image collected by an infrared camera on the robot, and the first image includes the reflective mark.

[0038] In this embodiment, at least one reflective identifier is provided on the side wall of the charging pile for charging the robot. The shape of the reflective identifier can be set as needed. For example, the shape of the reflective identifier can be set as a circle, a rectangle, a square, etc. If there are multiple reflective identifiers, the shapes of the multiple reflective identifiers can be the same or different. When there are multiple reflective identifiers, the multiple reflective identifiers can be set according to a preset rule. For example, the preset rule can be to set one reflective identifier in the first row and three reflective identifiers in the second row. The preset rule can also be to set the reflective identifiers symmetrically left and right, etc.

[0039] Specifically, on the surface of the charging pile where the reflective identifier is provided, a first coordinate system is established with the height direction (gravity direction) of the charging pile as the vertical axis (Z-axis), the width direction of the charging pile as the horizontal axis (X-axis), and the thickness direction of the charging pile as the longitudinal axis (Y-axis). The X-axis, Y-axis, and Z-axis of the first coordinate system are perpendicular to each other. Determine the coordinates (X-axis coordinate and Z-axis coordinate) of the center point of the reflective identifier in the first coordinate system. The coordinates of the center points of all the reflective identifiers form a second matrix.

[0040] In this embodiment, the infrared camera can shield ambient light interference and only obtain images in the infrared band. The infrared camera includes an infrared fill light, and the infrared fill light can emit infrared rays. Since the infrared fill light is provided in the infrared camera, the infrared camera can be used to collect a first image in the case of insufficient light, for example Figure 3 As shown, in the area circled by the ellipse in the figure, the white area is the reflective identifier. In addition, the infrared camera can illuminate at a relatively long distance. The robot can use the infrared camera to collect the first image of the charging pile at a position more than ten meters away from the charging pile. The first image can be a grayscale image.

[0041] S102, determine a first area of the reflective identifier in the first image.

[0042] In this embodiment, since the reflective identifier has a reflective effect, the effect presented in the first image is that the area of the reflective identifier is brighter than other areas, that is, the grayscale value is higher.

[0043] Specifically, input the first image into the trained convolutional neural network to obtain the first area.

[0044] S103, based on the position of the first area on the first image and the preset position of the reflective identifier on the charging pile, determine the pose of the robot relative to the charging pile.

[0045] Specifically, a first matrix is generated based on the coordinates of the center point of the first region on the first image; a second matrix is generated based on the coordinates of the center point of the reflective mark on the charging pile; based on the first matrix, the second matrix, the internal parameter matrix of the infrared camera, and the distortion parameters of the infrared camera, the pose of the robot relative to the charging pile is determined.

[0046] Specifically, based on the first matrix, the second matrix, the internal parameter matrix of the infrared camera, and the distortion parameters of the infrared camera, the pose of the infrared camera relative to the charging pile is determined; based on the position of the infrared camera on the robot, the pose of the robot relative to the charging pile is determined.

[0047] In this embodiment, a second coordinate system is established in the first image. Specifically, the lower left corner of the first image can be used as the origin, the lower border of the first image as the horizontal axis, and the left border of the first image as the vertical axis to establish the second coordinate system.

[0048] The coordinates of the center points of each first region in the second coordinate system are determined, and the coordinates of the center points of all first regions in the second coordinate system form the first matrix. When establishing the first matrix, according to the position of the first region, each first region is corresponding to the reflective mark on the charging pile, and the first matrix is generated according to the creation rule of the second matrix. For example, the second matrix is created according to the counterclockwise order of the reflective marks. When creating the first matrix, it is also necessary to create it according to the counterclockwise order of the first regions, and determine the first first region in the first regions according to the first reflective mark in the second matrix.

[0049] In this embodiment, the infrared camera is calibrated to obtain the internal parameter matrix and distortion parameters of the infrared camera. In addition, the internal parameter matrix and distortion parameters of the infrared camera can also be pre-stored or obtained from an external storage device.

[0050] In this embodiment, the position of the infrared camera on the robot can be pre-set, for example, the coordinates of the infrared camera on the robot.

[0051] In this embodiment, based on the first matrix, the second matrix, the internal parameter matrix of the infrared camera, and the distortion parameters of the infrared camera, the PnP (P4P) algorithm is used to calculate the pose and spatial coordinates of the infrared camera relative to the charging pile, and a rotation vector and a translation matrix are obtained. The parameters of the rotation vector are taken as negative numbers, and the inverse of the translation matrix is calculated to obtain a 6-element pose matrix including coordinates, rotation angle, left-right swing angle (0 degrees), and up-down swing angle (0 degrees), thereby obtaining the pose relationship between the charging pile and the infrared camera.

[0052] The purpose of the PnP (perspective-n-point) algorithm is to solve the method of 3D-2D point pair motion. Simply put, it is how to estimate the pose of the camera (i.e., the attitude of the camera in a specified coordinate system A) when the coordinates of n three-dimensional space points (relative to a specified coordinate system A) and their two-dimensional projection positions are known.

[0053] S104, based on the pose of the robot relative to the charging pile, control the robot to charge on the charging pile.

[0054] Specifically, when the robot uses the charging pile to charge, the robot may be at a certain distance from the charging pile. Therefore, after determining the pose of the robot relative to the charging pile, it is also necessary to determine the pose of the robot relative to the charging position, and then control the robot to reach the charging position and charge at the charging position.

[0055] In the embodiment of the present application, first obtain the first image collected by the infrared camera on the robot, determine the first area of the reflective mark on the charging pile for charging the robot in the first image, determine the pose of the robot relative to the charging pile based on the position of the first area in the first image and the position of the reflective mark on the charging pile, and finally control the robot to charge on the charging pile based on the pose of the robot relative to the charging pile. Compared with using lidar to determine the position of the charging pile, which causes inaccurate identification of the charging pile, the present application collects the first image through an infrared camera and identifies the area of the reflective mark in the first image, which can avoid the problem of inaccurate identification of the charging pile when there are objects with similar external structures to the charging pile near the charging pile. The present application can accurately obtain the pose of the robot relative to the charging pile and then accurately charge the robot.

[0056] As Figure 4 shown, in a possible implementation manner, the implementation process of step S102 may include:

[0057] S1021, determine the target pixel points among the pixel points of the first image whose gray values are greater than the first preset threshold.

[0058] In this embodiment, each pixel point in the first image corresponds to a gray value. Search for the gray values greater than the first preset threshold among all the gray values, and record the gray values greater than the first preset threshold as the first gray value. The first preset threshold can be set as needed.

[0059] Search for the pixel points corresponding to the first gray value, and the pixel points corresponding to the first gray value are the target pixel points.

[0060] In addition, perform binarization processing on the first image to obtain a binary image. Specifically, set the first gray value to 255 and other gray values to 0 to obtain a binary image.

[0061] S1022. Determine a second region in the first image based on the position of the target pixel point in the first image.

[0062] In this embodiment, if two target pixel points are adjacent pixel points in the first image, then the two target pixel points are within a third region. Adjacent pixel points can be adjacent vertically or horizontally.

[0063] As an example, if target pixel point A is the pixel point at the second row and the third column in the first image, and target pixel point B is the pixel point at the second row and the fourth column, then target pixel point A and target pixel point B are adjacent pixel points, and target pixel point A and target pixel point B are within the same third region.

[0064] If there are non-target pixel points between two target pixel points, then the two target pixel points are within different third regions.

[0065] As an example, if target pixel point A is the pixel point at the second row and the third column in the first image, and target pixel point B is the pixel point at the second row and the fifth column, then target pixel point A and target pixel point B are non-adjacent pixel points, and target pixel point A and target pixel point B are within different third regions.

[0066] In this embodiment, after determining all the third regions based on all the target pixel points, perform another screening using the perimeter and / or area of the third regions to obtain the second region in the third regions.

[0067] In addition, after obtaining the binary image, the white region part in the binary image is the third region.

[0068] S1023. If the number of the second regions is greater than or equal to the number of the reflective markings, determine the first region from the second regions based on the coordinates of the target pixel points on the contour line of the second region.

[0069] In this embodiment, according to the coordinates of the target pixel points on the contour line of the second region, it can be determined whether the shape of the second region matches the shape of the reflective marking, and then the first region is determined according to the second region whose shape matches the shape of the reflective marking.

[0070] In this embodiment, if the number of the second regions is less than the number of the reflective markings, discard the first image, indicating that the first region cannot be determined based on this first image, and another first image needs to be acquired again.

[0071] In this embodiment, since the gray value of the region of the reflective marking in the first image is different from the gray values of other regions, therefore, the second region can be determined according to the gray value, and then the first region can be more accurately determined according to the coordinates of the target pixel points on the contour line of the second region.

[0072] As Figure 5 shown, in a possible implementation, the implementation process of step S1022 may include:

[0073] S201. Based on the position of the target pixel point in the first image, determine a third region in the first image.

[0074] Specifically, please refer to the description of obtaining the third region in step S1022, which will not be elaborated here.

[0075] S202. Calculate first data of the third region, where the first data includes area and / or perimeter.

[0076] Specifically, calculate the area and / or perimeter of the third region according to the coordinates of the target pixel points on the contour of the third region.

[0077] S203. Determine second data in the first data that meets a preset requirement, and the third region corresponding to the second data is the second region.

[0078] In this embodiment, when the first data includes the area, the preset requirement includes that the area of the third region is within a first preset interval; when the first data includes the perimeter, the preset requirement includes that the perimeter of the third region is within a second preset interval; when the first data includes the area and the perimeter, the preset requirement includes that the area of the third region is within the first preset interval and the perimeter of the third region is within the second preset interval.

[0079] In this embodiment, after determining the third region using the position of the target pixel point, use the area and / or perimeter of the third region to screen the third region, and remove the third regions with too small area and / or too small perimeter in the third region to obtain the second region. By screening the third region with the first data, a more accurate region of the reflective mark can be obtained.

[0080] As Figure 6 shown, in a possible implementation, the implementation process of step S1023 may include:

[0081] S301. Based on the coordinates of the target pixel points on the contour of the second region, determine a fourth region in the second region that matches the shape of the reflective mark.

[0082] Optionally, based on the coordinates of the target pixel points on the contour of the second region, calculate the perimeter and area of the second region; calculate the first ratio of the perimeter of the second region to the area of the second region; if the first ratio is within a third preset interval, the second region corresponding to the first ratio within the third preset interval is the fourth region.

[0083] In this embodiment, the ratios of the perimeters and areas of reflective signs with different shapes are different. For example, the ratio of the perimeter to the area of a circular reflective sign is different from that of a rectangular reflective sign. Therefore, the ratio of the perimeter and area of the second region can be used to determine whether the shape of the second region matches the shape of the reflective sign.

[0084] If the first ratio is not within the third preset interval, remove the second region corresponding to the first ratio that is not within the third preset interval.

[0085] Optionally, based on the coordinates of the pixel points on the contour of the second region, determine the smoothness of the contour line of the second region; if the smoothness is within a fourth preset interval, the second region corresponding to the smoothness within the fourth preset interval is the fourth region.

[0086] In this embodiment, since the smoothness of the contour lines of reflective signs with different shapes is different, therefore, according to the smoothness of the contour line of the second region, it can be determined whether the second region matches the shape of the reflective sign. Specifically, input the coordinates of the pixel points on the contour of the second region into the calculation model to obtain the smoothness of the contour line of the second region.

[0087] If the smoothness is not within the fourth preset interval, it means that the second region corresponding to this smoothness does not match the shape of the reflective sign, and the second region corresponding to the smoothness not within the preset interval is discarded.

[0088] Optionally, if the number of reflective signs is multiple, according to the coordinates of the target pixel points on the contour of the second region, determine the coordinates of the center point of each second region, and obtain the third invariant moment of the second region according to the coordinates of the center point of the second region. For example, there are 3 second regions, and each second region corresponds to a third invariant moment. If the similarity between the third invariant moment and the invariant moment of the preset reflective sign is greater than the preset value, the second region corresponding to the third invariant moment with the similarity greater than the preset value is the fourth region.

[0089] In this embodiment, the main idea of invariant moments is to use several moments of regions insensitive to transformation as shape features. Invariant moments are a set of moments calculated from a digital graph, usually used to describe the global features of the graph and provide geometric feature information of the graph. The geometric feature information may include size, position, orientation, shape, etc. Invariant moments can be first-order, second-order, third-order, Hu ((Visual pattern recognition by moment invariants)) moments, etc.

[0090] In this embodiment, if the shapes of multiple reflective markers are different, for example, there are two circular reflective markers and two square reflective markers. It is also necessary to determine the shape type of the fourth region. Specifically, each preset interval corresponds to a shape type.

[0091] As an example, if the third preset interval includes a first interval and a second interval, the shape type corresponding to the first interval is circular, and the shape type corresponding to the second interval is rectangular. The first ratio corresponding to the fourth region A is within the first interval, the first ratio corresponding to the fourth region B is within the first interval, and the first ratio corresponding to the fourth region C is within the second interval. The shape types of the fourth region A and the fourth region B are both circular, and the shape type of the fourth region C is rectangular.

[0092] S302, if the number of the fourth regions is equal to the number of the reflective markers, based on the coordinates of the center points of the fourth regions, obtain the first invariant moment.

[0093] In this embodiment, if the number of the fourth regions is equal to the number of the reflective markers, it is possible to continue to determine whether the fourth region is the first region. If the number of the fourth regions is less than or greater than the number of the reflective markers, discard the first image and stop searching for the first region.

[0094] In this embodiment, when there are multiple reflective markers with different shapes, if the number of the fourth regions is equal to the number of the reflective markers and the number of shape types included in the fourth regions is the same as the number of shape types included in the reflective markers, based on the coordinates of the center points of the fourth regions, obtain the first invariant moment.

[0095] For example, there are 4 reflective markers, two circular and two rectangular. The number of the fourth regions is 4, and the shape types of the fourth regions include two circles and two rectangles. Based on the coordinates of the center points of the fourth regions, obtain the first invariant moment.

[0096] In this embodiment, calculate the first invariant moment based on the coordinates of the center points of all the fourth regions. For example, there are 4 fourth regions, and the first invariant moment is obtained according to the coordinates of the 4 center points. The first invariant moment reflects the invariant moment of the shape formed by the center points of all the fourth regions.

[0097] S303. Calculate the similarity between the first invariant moment and a preset invariant moment, where the preset invariant moment is determined based on the coordinates of the center point of the reflective identification mark on the charging pile.

[0098] In this embodiment, the first invariant moment and the preset invariant moment are input into a similarity calculation model to obtain the similarity between the first invariant moment and the preset invariant moment.

[0099] S304. If the similarity is greater than a second preset threshold, the fourth region is the first region corresponding to the reflective identification mark.

[0100] In this embodiment, the second preset threshold can be set as needed. Using the similarity between the first invariant moment and the preset invariant moment to determine the first region can determine whether the distribution of all fourth regions is the same as the distribution of all reflective identification marks, and further determine whether the fourth region is the region where the reflective identification mark is located, ensuring the accuracy of the determined first region.

[0101] In a possible implementation manner, the above method may further include:

[0102] S10. Obtain a first image collected by an infrared camera on the robot, where the first image includes the reflective identification mark.

[0103] S20. Determine target pixel points in the pixel points of the first image whose gray values are greater than a first preset threshold.

[0104] S30. Based on the positions of the target pixel points in the first image, determine a third region in the first image.

[0105] S40. Calculate first data of the third region, where the first data includes area and / or perimeter. Determine second data that meets preset requirements in the first data, and the third region corresponding to the second data is the second region.

[0106] S50. If the number of second regions is greater than or equal to the number of reflective identification marks, based on the coordinates of the target pixel points on the contour of the second region, determine a fourth region that matches the shape of the reflective identification mark from the second regions.

[0107] S60. If the number of fourth regions is equal to the number of reflective identification marks, based on the coordinates of the center points of the fourth regions, obtain a first invariant moment. Calculate the similarity between the first invariant moment and a preset invariant moment, where the preset invariant moment is determined based on the coordinates of the center point of the reflective identification mark on the charging pile; if the similarity is greater than a second preset threshold, the fourth region is the first region corresponding to the reflective identification mark.

[0108] S70, generate a first matrix based on the coordinates of the center point of the first region on the first image.

[0109] S80, generate a second matrix based on the coordinates of the center point of the reflective identifier on the charging pile.

[0110] S90, determine the pose of the infrared camera relative to the charging pile based on the first matrix, the second matrix, the internal parameter matrix of the infrared camera, and the distortion parameters of the infrared camera;

[0111] S100, determine the pose of the robot relative to the charging pile based on the position of the infrared camera on the robot, and control the robot to charge on the charging pile based on the pose of the robot relative to the charging pile.

[0112] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0113] Corresponding to the charging control method of the robot described in the above embodiments, Figure 7 The structural block diagram of the charging control device of the robot provided by the embodiment of the present application is shown. For the sake of convenience of description, only the parts related to the embodiment of the present application are shown.

[0114] Refer to Figure 7 , the device 400 may include: an image acquisition module 410, a first region determination module 420, a pose determination module 430, and a control module 440.

[0115] Among them, the image acquisition module 410 is used to acquire a first image collected by an infrared camera on the robot, and the reflective identifier is included in the first image;

[0116] The first region determination module 420 is used to determine a first region of the reflective identifier in the first image;

[0117] The pose determination module 430 is used to determine the pose of the robot relative to the charging pile based on the position of the first region on the first image and the preset position of the reflective identifier on the charging pile;

[0118] The control module 440 is used to control the robot to charge on the charging pile based on the pose of the robot relative to the charging pile.

[0119] In a possible implementation manner, the first region determination module 420 may specifically be used for:

[0120] Determine the target pixel points among the pixel points of the first image whose grayscale values are greater than the first preset threshold;

[0121] Based on the positions of the target pixel points in the first image, determine a second region in the first image;

[0122] If the number of the second regions is greater than or equal to the number of the reflective markings, based on the coordinates of the target pixel points on the contour lines of the second regions, determine the first region from the second regions.

[0123] In a possible implementation manner, the first region determination module 420 may specifically be configured to:

[0124] Based on the positions of the target pixel points in the first image, determine a third region in the first image;

[0125] Calculate first data of the third region, where the first data includes area and / or perimeter;

[0126] Determine second data that meets the preset requirements in the first data, and the third region corresponding to the second data is the second region. Wherein, when the first data includes the area, the preset requirements include that the area of the third region is within a first preset interval; when the first data includes the perimeter, the preset requirements include that the perimeter of the third region is within a second preset interval; when the first data includes the area and the perimeter, the preset requirements include that the area of the third region is within the first preset interval and the perimeter of the third region is within the second preset interval.

[0127] In a possible implementation manner, the first region determination module 420 may specifically be configured to:

[0128] Based on the coordinates of the target pixel points on the contour of the second region, determine a fourth region that matches the shape of the reflective marking from the second region;

[0129] If the number of the fourth regions is equal to the number of the reflective markings, obtain a first moment invariant based on the coordinates of the center points of the fourth regions;

[0130] Calculate the similarity between the first moment invariant and a preset moment invariant, where the preset moment invariant is determined based on the coordinates of the center point of the reflective marking on the charging pile;

[0131] If the similarity is greater than a second preset threshold, the fourth region is the first region corresponding to the reflective marking.

[0132] In a possible implementation manner, the first region determination module 420 may specifically be configured to:

[0133] Calculate the perimeter and area of the second region based on the coordinates of the target pixel points on the contour of the second region;

[0134] Calculate a first ratio of the perimeter of the second region to the area of the second region;

[0135] If the first ratio is within a third preset interval, the second region corresponding to the first ratio within the third preset interval is the fourth region.

[0136] In a possible implementation manner, the first region determination module 420 may specifically be configured to:

[0137] Determine the smoothness of the contour line of the second region based on the coordinates of the pixel points on the contour of the second region;

[0138] If the smoothness is within a fourth preset interval, the second region corresponding to the smoothness within the fourth preset interval is the fourth region.

[0139] In a possible implementation manner, the pose determination module 430 may specifically be configured to:

[0140] Generate a first matrix based on the coordinates of the center point of the first region on the first image;

[0141] Generate a second matrix based on the coordinates of the center point of the reflective mark on the charging pile;

[0142] Determine the pose of the robot relative to the charging pile based on the first matrix, the second matrix, the internal parameter matrix of the infrared camera, and the distortion parameters of the infrared camera.

[0143] In a possible implementation manner, the pose determination module 430 may specifically be configured to:

[0144] Determine the pose of the infrared camera relative to the charging pile based on the first matrix, the second matrix, the internal parameter matrix of the infrared camera, and the distortion parameters of the infrared camera;

[0145] Determine the pose of the robot relative to the charging pile based on the position of the infrared camera on the robot.

[0146] It should be noted that for the information interaction, execution process, etc. between the above-mentioned device / units, since they are based on the same concept as the method embodiment of the present application, their specific functions and the technical effects brought are specifically described in the method embodiment part, and will not be elaborated here.

[0147] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiment and will not be elaborated here.

[0148] An embodiment of this application also provides a terminal device. Refer to Figure 8 , the terminal device 500 may include: at least one processor 510, a memory 520, and a computer program stored in the memory 520 and executable on the at least one processor 510. When the processor 510 executes the computer program, it implements the steps in any of the foregoing method embodiments, such as Figure 2 the steps S101 to S104 in the illustrated embodiment. Alternatively, when the processor 510 executes the computer program, it implements the functions of each module / unit in the foregoing device embodiments, such as Figure 7 the functions of the illustrated modules 410 to 440.

[0149] Exemplarily, the computer program can be divided into one or more modules / units. One or more modules / units are stored in the memory 520 and executed by the processor 510 to complete this application. The one or more modules / units can be a series of computer program segments capable of completing specific functions, and these program segments are used to describe the execution process of the computer program in the terminal device 500.

[0150] Those skilled in the art can understand that Figure 8 this is only an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown in the figure, or combine some components, or different components, such as input / output devices, network access devices, buses, etc.

[0151] The processor 510 may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0152] The memory 520 may be an internal storage unit of the terminal device or an external storage device of the terminal device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. The memory 520 is used to store the computer program and other programs and data required by the terminal device. The memory 520 may also be used to temporarily store data that has been output or is to be output.

[0153] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the buses in the drawings of this application are not limited to only one bus or one type of bus.

[0154] The charging control method for the robot provided by the embodiments of this application can be applied to terminal devices such as computers, tablet computers, laptop computers, netbooks, personal digital assistants (PDAs), etc. The embodiments of this application do not impose any restrictions on the specific types of terminal devices.

[0155] In the above embodiments, the descriptions of the various embodiments each have their own emphases. For parts not detailed or recorded in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0156] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0157] In the embodiments provided in this application, it should be understood that the disclosed terminal devices, apparatuses, and methods can be implemented in other ways. For example, the terminal device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in an electrical, mechanical, or other form.

[0158] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0159] In addition, the functional units in each embodiment of this application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0160] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-mentioned method embodiments of this application, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by one or more processors, the steps of the above-mentioned method embodiments can be implemented.

[0161] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-mentioned embodiment methods of this application, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by one or more processors, the steps of the above-mentioned various method embodiments can be implemented.

[0162] Similarly, as a computer program product, when the computer program product runs on a terminal device, it enables the terminal device to implement the steps in the above-mentioned various method embodiments when executed.

[0163] Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0164] The above-mentioned embodiments are only used to illustrate the technical solutions of this application, rather than to limit them; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of this application, and should all be included in the protection scope of this application.

Claims

1. A charging control method for a robot, characterized in that, at least one reflective mark is provided on the side wall of a charging pile for charging the robot; the method includes: acquiring a first image collected by an infrared camera on the robot, where the first image includes the reflective mark; determining a first area of the reflective mark in the first image, including: determining target pixel points in the pixel points of the first image whose gray values are greater than a first preset threshold; based on the positions of the target pixel points in the first image, determining a second area in the first image, including: based on the positions of the target pixel points in the first image, determining a third area in the first image, calculating first data of the third area, the first data including area and / or perimeter, determining second data that meets preset requirements in the first data, and the third area corresponding to the second data is the second area, where if two target pixel points are adjacent pixel points in the first image, the two target pixel points are in one third area, and if there are non-target pixel points between two target pixel points, the two target pixel points are in different third areas, when the first data includes the area, the preset requirements include that the area of the third area is within a first preset interval, when the first data includes the perimeter, the preset requirements include that the perimeter of the third area is within a second preset interval, and when the first data includes the area and the perimeter, the preset requirements include that the area of the third area is within the first preset interval and the perimeter of the third area is within the second preset interval; if the number of the second areas is greater than or equal to the number of the reflective marks, determining the first area from the second areas based on the coordinates of the target pixel points on the contour line of the second area; determining the pose of the robot relative to the charging pile based on the position of the first area on the first image and the preset position of the reflective mark on the charging pile; controlling the robot to charge on the charging pile based on the pose of the robot relative to the charging pile.

2. The charging control method for a robot according to claim 1, characterized in that, the determining the first area from the second areas based on the coordinates of the target pixel points on the contour line of the second area includes: determining a fourth area that matches the shape of the reflective mark from the second areas based on the coordinates of the target pixel points on the contour of the second area; if the number of the fourth areas is equal to the number of the reflective marks, obtaining a first invariant moment based on the coordinates of the center points of the fourth areas; calculating the similarity between the first invariant moment and a preset invariant moment, where the preset invariant moment is determined based on the coordinates of the center points of the reflective marks on the charging pile; if the similarity is greater than a second preset threshold, the fourth area is the first area corresponding to the reflective mark.

3. The charging control method for a robot according to claim 2, characterized in that, Determining a fourth region that matches the shape of the reflective identifier from the second region based on the coordinates of the target pixel points on the contour of the second region includes: Calculating the perimeter and area of the second region based on the coordinates of the target pixel points on the contour of the second region; Calculating a first ratio of the perimeter of the second region to the area of the second region; If the first ratio is within a third preset interval, the second region corresponding to the first ratio within the third preset interval is the fourth region.

4. The charging control method for a robot according to claim 2, wherein, Determining a fourth region that matches the shape of the reflective identifier from the second region based on the coordinates of the target pixel points on the contour of the second region includes: Determining the smoothness of the contour line of the second region based on the coordinates of the pixel points on the contour of the second region; If the smoothness is within a fourth preset interval, the second region corresponding to the smoothness within the fourth preset interval is the fourth region.

5. The charging control method for a robot according to any one of claims 1 to 4, wherein, Determining the pose of the robot relative to the charging pile based on the position of the first region on the first image and the preset position of the reflective identifier on the charging pile includes: Generating a first matrix based on the coordinates of the center point of the first region on the first image; Generating a second matrix based on the coordinates of the center point of the reflective identifier on the charging pile; Determining the pose of the robot relative to the charging pile based on the first matrix, the second matrix, the internal parameter matrix of the infrared camera, and the distortion parameters of the infrared camera.

6. The charging control method for a robot according to claim 5, wherein, Determining the pose of the robot relative to the charging pile based on the first matrix, the second matrix, the internal parameter matrix of the infrared camera, and the distortion parameters of the infrared camera includes: Determining the pose of the infrared camera relative to the charging pile based on the first matrix, the second matrix, the internal parameter matrix of the infrared camera, and the distortion parameters of the infrared camera; Determining the pose of the robot relative to the charging pile based on the position of the infrared camera on the robot.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the charging control method for a robot according to any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, wherein, When the computer program is executed by a processor, it implements the charging control method for a robot according to any one of claims 1 to 6.

Images