Automatic recharging method, device, robot and electronic equipment

Abstract

The present invention proposes an automatic recharging method, device, robot and electronic equipment, which belongs to the field of control technology. The method includes: using a target detection model to generate a global detection result of an environmental image, and obtaining the positioning information of the robot relative to the charging pile according to the global detection result; based on the positioning information, when it is determined that the robot is located in front of the charging surface of the charging pile, local detection is performed on the environmental image to obtain multiple key point positions of the charging pile, and based on the multiple key point positions, the current position of the robot relative to the charging pile is obtained; based on the current position, the position and / or posture of the robot is adjusted to allow the charging interface of the robot to dock with the charging pile. In this way, by first roughly positioning, and then using deep local detection to accurately locate the position of the charging pile, and adjusting the robot posture accordingly, the influence of external conditions is weakened or even eliminated, and the robot can effectively achieve accurate positioning of the charging pile and efficient recharging.

Description

Technical Field

[0001] The present invention relates to the field of control technology, and in particular to an automatic recharging method, device, robot and electronic equipment. Background Art

[0002] With the rapid development of robot technology, automatic recharging has become a must-have feature for various types of robots, such as mowing robots, sweeping robots, and inspection robots. Automatic recharging means that after completing a task or when the battery is low, the robot automatically returns to the charging station or charging base for charging.

[0003] At present, common automatic recharging methods include: (1) induction signal method, through the induction signal emitted by the charging pile, the robot detects and analyzes these signals, and then adjusts the robot's own posture to accurately dock with the charging interface of the charging pile; (2) guide line method, using the pre-set guide line, the robot detects and follows the guide line to achieve automatic recharging; (3) lidar detection method, through the point cloud data obtained by lidar, after correction, the position of the charging pile is identified to achieve accurate recharging; (4) grass boundary recognition method, the robot identifies the grass boundary and moves along the boundary until it finds the charging pile to achieve recharging; (5) pure visual automatic recharging.

[0004] However, the current purely visual automatic recharging method is easily affected by insufficient generalization when detecting the corner points of the charging pile or by external conditions such as lighting, resulting in poor recharging positioning accuracy. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide an automatic recharging method, device, robot and electronic device, which integrate target detection technology, deep learning algorithm and image processing technology to achieve accurate positioning and efficient recharging of charging piles by the robot.

[0006] In order to achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows:

[0007] In a first aspect, the present invention provides an automatic recharging method, the method comprising:

[0008] Obtaining an environmental picture taken in real time by a visual camera; wherein the visual camera is arranged on the robot;

[0009] Inputting the environment image into a target detection model, obtaining a global detection result using the target detection model, and obtaining positioning information of the robot relative to the charging pile according to the global detection result;

[0010] According to the positioning information, determining whether the robot is located in front of the charging surface of the charging pile;

[0011] If yes, perform local detection on the environment image to obtain multiple key point positions of the charging pile, and obtain the current position of the robot relative to the charging pile based on the multiple key point positions;

[0012] Based on the current position, the position and / or posture of the robot is adjusted so that the charging interface of the robot is docked with the charging pile.

[0013] Optionally, the positioning information includes an actual distance and orientation of the robot relative to the charging pile, and a surface category of the charging pile in the environment picture;

[0014] After the step of determining whether the robot is located in front of the charging surface of the charging pile according to the positioning information, the method further includes:

[0015] If the robot is not located in front of the charging surface of the charging pile, a motion path is obtained according to the current position of the robot and the target position in front of the charging surface;

[0016] According to the motion path, the robot is controlled to move to the target position in front of the charging surface, and returns to execute the step of obtaining the environment picture taken in real time by the visual camera.

[0017] Optionally, the step of obtaining a motion path according to the current position of the robot and the target position in front of the charging surface comprises:

[0018] Calculate the current position coordinates of the robot in the world coordinate system;

[0019] Taking the position coordinates as a starting point and the target position in front of the charging surface as a target point, path planning is performed to obtain a motion path.

[0020] Optionally, the detection result includes a bounding box of the charging pile;

[0021] The step of performing local detection on the environment image to obtain the positions of multiple key points of the charging pile includes:

[0022] Cropping the environment image to obtain a partial image of the bounding box;

[0023] Enlarging the local image, and detecting the enlarged local image to obtain a local detection result;

[0024] When the local detection result indicates that each key area of ​​the charging pile is not blocked, extracting a key point position from each key area;

[0025] When the local detection result indicates that at least one key area of ​​the charging pile is blocked, a cleaning alarm is issued.

[0026] Optionally, the step of extracting a key point position from each of the key areas comprises:

[0027] For each of the key areas, the image of the key area is enlarged to obtain an original key image;

[0028] Performing grayscale conversion and morphological operations on the original key image to obtain a preprocessed key image;

[0029] Edge detection is performed on the preprocessed key image to extract the pixel point with the largest gradient change as the target point, and the coordinates of the target point are used as a key point position.

[0030] Optionally, the step of obtaining the current position of the robot relative to the charging pile according to the positions of the plurality of key points includes:

[0031] Inputting the positions of the plurality of key points into a regression model, and using the regression model to generate angles and distances to obtain the current position of the robot relative to the charging pile;

[0032] The method for obtaining the regression model includes:

[0033] Acquire multiple key point images and key point positions in each of the key point images; wherein each of the key point images includes all key points of the charging pile;

[0034] Marking the key point positions of the sample images according to the distance and angle of the robot relative to the charging pile when taking the key point images, so as to obtain a plurality of sample data;

[0035] Combining the error function and the model complexity function, we get the target loss function;

[0036] Based on the sample data and the target loss function, the initial XGBoost model is iteratively trained to obtain a regression model.

[0037] Optionally, the global detection result includes a bounding box of the charging pile and a face category of the charging pile in the environment picture;

[0038] The step of obtaining the positioning information of the robot relative to the charging pile according to the global detection result includes:

[0039] Using the position of the bounding box in the environment image as the orientation of the robot relative to the charging pile;

[0040] The area of ​​the bounding box is calculated, and the area is input into a pre-fitted distance calculation formula to obtain the actual distance of the robot relative to the charging pile.

[0041] Optionally, the step of obtaining an environment picture taken in real time by a visual camera includes:

[0042] Sampling the visual camera to obtain a real-time original picture;

[0043] According to the calibration parameters of the visual camera, the original image is subjected to distortion correction to obtain an environment image.

[0044] Optionally, the method for acquiring the target detection model includes:

[0045] Label each sample image according to the bounding box of the charging pile to obtain a sample data set;

[0046] Based on the sample data set, iteratively train the initial detection model to obtain a teacher model;

[0047] Using the teacher model and the sample data set, knowledge distillation is performed on the student model to obtain a target detection model.

[0048] In a second aspect, the present invention provides an automatic recharging device, including an image acquisition module, a coarse positioning module, a position determination module, a fine positioning module and a precise docking module;

[0049] The picture acquisition module is used to acquire the environment picture taken in real time by the visual camera; wherein the visual camera is arranged on the robot;

[0050] The coarse positioning module is used to input the environment image into the target detection model, obtain a global detection result using the target detection model, and obtain the positioning information of the robot relative to the charging pile according to the global detection result;

[0051] The position determination module is used to determine whether the robot is located in front of the charging surface of the charging pile according to the positioning information;

[0052] The precise positioning module is used to perform local detection on the environment image when the robot is located in front of the charging pile, obtain multiple key point positions of the charging pile, and obtain the current position of the robot relative to the charging pile based on the multiple key point positions;

[0053] The precise docking module is used to adjust the position and / or posture of the robot based on the current position so that the charging interface of the robot can be docked with the charging pile.

[0054] In a third aspect, the present invention provides a robot, comprising a body, and a host and a visual camera arranged on the body;

[0055] The visual camera is used to take real-time photos of the environment to obtain an environmental picture;

[0056] The host is used to implement the automatic recharging method as described in the first aspect.

[0057] In a fourth aspect, the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the automatic recharging method as described in the first aspect.

[0058] The automatic recharging method, device, robot and electronic device provided by the embodiments of the present invention include: obtaining an environmental picture taken by a visual camera provided on the robot, inputting the environmental picture into a target detection model, generating a global detection result, and obtaining the positioning information of the robot relative to the charging pile based on the global detection result; based on the positioning information, when it is determined that the robot is located in front of the charging surface of the charging pile, performing local detection on the environmental picture to obtain multiple key point positions of the charging pile, and obtaining the current position of the robot relative to the charging pile based on the multiple key point positions; based on the current position, adjusting the position and / or posture of the robot so that the charging interface of the robot is docked with the charging pile.

[0059] In this way, the target detection technology is used to roughly locate the position of the charging pile first, so as to quickly narrow the search range and improve the efficiency of subsequent processing. Then, after the robot moves to the front of the charging surface, deep local detection is used to accurately locate the position of the charging pile, and the robot posture is adjusted accordingly to ensure that the robot charging interface is connected to the charging pile. In this way, the influence of external conditions is reduced or even eliminated, and the robot can effectively achieve accurate positioning of the charging pile and efficient recharging.

[0060] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0062] Figure 1 A block diagram of an automatic recharging system provided by an embodiment of the present invention is shown.

[0063] Figure 2 A block diagram of an electronic device provided by an embodiment of the present invention is shown.

[0064] Figure 3 One of the flow charts of the automatic recharging method provided by an embodiment of the present invention is shown.

[0065] Figure 4 Shows Figure 3 Schematic diagram of the process flow of some sub-steps of step 11.

[0066] Figure 5 The second flowchart of the automatic recharging method provided by the embodiment of the present invention is shown.

[0067] Figure 6 Shows Figure 3 Schematic diagram of the process flow of some sub-steps of step 13.

[0068] Figure 7 The third flowchart of the automatic recharging method provided by the embodiment of the present invention is shown.

[0069] Figure 8 Shows Figure 7 Schematic diagram of the process of some sub-steps of step 16.

[0070] Fig. 9 Shows Figure 3 and Figure 7 Schematic diagram of the process of some sub-steps of step 17.

[0071] Fig.10 Shows Fig. 9 A flowchart of some sub-steps of step 177.

[0072] Fig.11 A fourth flowchart of the automatic recharging method provided in an embodiment of the present invention is shown.

[0073] Fig.12 A block diagram of an automatic recharging device provided in an embodiment of the present invention is shown.

[0074] Explanation of the accompanying drawings: 10-automatic recharging system; 110-model training equipment; 120-charging pile; 130-robot; 140-host; 150-visual camera; 20-electronic device; 210-memory; 220-processor; 230-communication module; 30-automatic recharging device; 310-image acquisition module; 320-coarse positioning module; 330-position judgment module; 340-precise positioning module; 350-precise docking module. DETAILED DESCRIPTION

[0075] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0076] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0077] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0078] The automatic recharging method provided by the embodiment of the present invention can be applied to Figure 1 In the automatic recharging system 10 shown, the automatic recharging system 10 includes a model training device 110, a charging pile 120 and a robot 130. The robot 130 includes a main body and a host 140 and a visual camera 150 installed on the main body. The host 140 is communicated with the visual camera 150 and the model training device 110 respectively through wired connection or wireless connection.

[0079] The model training device 110 is used to train a target detection model, a key point detection model and a regression model, and deploy the target detection model, the key point detection model and the regression model on the host 140.

[0080] Among them, the target detection model is trained to recognize the bounding box of the charging pile 120 in the picture, the key point detection model is trained to recognize the key identification area of ​​the charging pile 120 in the picture, and the regression model is trained to output the angle and distance of the robot relative to the charging pile based on the input key point position.

[0081] The visual camera 150 is used to take real-time pictures of the environment in which the robot 130 is located when the robot 130 enters a working state (such as movement, mowing, inspection, etc.), and transmit the pictures to the host 140.

[0082] The host 140 is used to implement the automatic recharging method provided in the embodiment of the present invention.

[0083] The robot 130 may be any robot 130 such as a lawn mowing robot 130 , an inspection robot 130 , a sweeping robot 130 , etc. The host 140 may be, but is not limited to, a computer device, a single-chip microcomputer, a programmable logic controller (PLC), or a field programmable gate array (FPGA).

[0084] Please refer to Figure 2 , is a block diagram of an electronic device 20, which can be Figure 1 The host 140 in the automatic recharging system 10 shown. The electronic device 20 includes a memory 210, a processor 220 and a communication module 230. The memory 210, the processor 220 and the communication module 230 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0085] The memory 210 is used to store programs or data. The memory 210 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0086] The processor 220 is used to read / write data or programs stored in the memory 210 and execute corresponding functions. For example, Figure 1 In the automatic recharging system 10 shown, the processor 220 of the host 140 reads the computer program stored in the memory 210 to implement the automatic recharging method provided in the embodiment of the present invention.

[0087] The communication module 230 is used to establish a communication connection between the electronic device 20 and other communication terminals through the network, and to send and receive data through the network. Figure 1 In the automatic recharging system 10 shown, the host 140 sends and receives data with the visual camera 150 via a network.

[0088] It should be understood that Figure 2 The structure shown is only a schematic diagram of the structure of the electronic device 20. The electronic device 20 may also include Figure 2 More or fewer components as shown, or with Figure 2 Different configurations are shown. Figure 2 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0089] In order to solve the problem that the current pure visual automatic recharging method has poor recharging positioning accuracy due to external conditions, an embodiment of the present invention provides an automatic recharging method, referring to Figure 3 , including steps 11 to 19, such that: Figure 1 In the automatic recharging system 10 shown, the host 140 is Figure 2 In the structure shown, when the processor 220 executes the computer program stored in the memory 210, the above steps 11 to 19 are executed to implement the automatic recharging method.

[0090] Step 11, obtain the environment picture taken by the visual camera in real time.

[0091] Wherein, the visual camera is arranged on the robot.

[0092] Step 13: Input the environment image into the target detection model, use the target detection model to obtain a global detection result, and obtain the positioning information of the robot relative to the charging pile based on the global detection result.

[0093] Step 15: determine whether the robot is located in front of the charging surface of the charging pile based on the positioning information. If yes, proceed to step 17.

[0094] Step 17, perform local detection on the environment image to obtain multiple key point positions of the charging pile, and obtain the current position of the robot relative to the charging pile based on the multiple key point positions.

[0095] Step 19: Based on the current position, adjust the position and / or posture of the robot so that the charging interface of the robot is docked with the charging pile.

[0096] For example, in combination Figure 1In the automatic recharging system 10 shown, after the robot 130 is powered on, the visual camera 150 continuously captures the surrounding environment to obtain a real-time environmental picture. When the robot 130 is low on power or completes work (such as mowing, inspection, and sweeping), it enters the automatic recharging mode, and the host 140 samples the visual camera 150 to obtain the latest current environmental picture, and inputs the environmental picture into the target detection model, which is processed by the target detection model to obtain a global detection result. Furthermore, the host 140 obtains the positioning information of the robot relative to the charging pile based on the global detection result (it can also be understood as the group positioning information of the charging pile 120 relative to the robot 130). The positioning information can be in the form of information such as the front, left, right, lower right corner, lower left corner, upper right corner, and upper left corner.

[0097] When the positioning information indicates that the robot 130 is located in front of the charging surface of the charging pile 120, the host 140 performs a partial detection of the environmental image to obtain multiple key point positions of the charging pile 120 (i.e., the area on the charging pile 120 that has an identifying function), and obtains the current position of the robot relative to the charging pile based on the multiple key point positions (in this case, it is precise positioning information, such as distance and angle, or coordinates, etc.). Furthermore, the host 140 adjusts the position and / or posture of the robot 130 based on the current position by means of PID adjustment, and repeatedly adjusts it according to the latest environmental image so that the charging interface of the robot 130 can be docked with the charging pile 120.

[0098] It should be noted that the host 140 may be pre-set with a docking adjustment rule for the charging pile 120. The host 140 repeatedly adjusts the posture of the robot 130 according to the docking adjustment rule, combined with PID adjustment and the current position, so that the charging interface of the robot 130 docks with the charging pile 120. Alternatively, the current position may be used as the starting point, the preset charging position may be used as the end point, and the A* algorithm may be used to plan the charging movement path, and the robot 130 may be controlled to travel to the charging position according to the charging movement path, and then the posture of the robot 130 may be adjusted at the charging position according to the latest real-time posture of the robot 130 and the preset charging posture, so that the charging interface of the robot 130 docks with the charging pile 120. The above method is only an example, and its implementation method is not limited.

[0099] Compared with the traditional purely visual recharging method, the above automatic recharging method uses the integrated use of image processing, target detection technology, etc., and uses target detection technology to roughly locate the position of the charging pile first, so as to quickly narrow the search range and improve the efficiency of subsequent processing. Then, after the robot moves to the front of the charging surface, deep local detection is used to accurately locate the position of the charging pile, and the robot posture is adjusted accordingly to ensure that the robot charging interface is docked with the charging pile. Thereby, the influence of external conditions (such as lighting, lack of detection generalization, etc.) is weakened or even eliminated, and the robot can effectively realize the precise positioning and efficient recharging of the charging pile.

[0100] In order to eliminate the image error caused by lens distortion in the image taken by the visual camera 150, the idea of ​​performing distortion correction on the image is introduced in the process of obtaining the environment image in step 11, referring to Figure 4 , including steps 111 to 113.

[0101] Step 111, sampling the visual camera to obtain a real-time original picture.

[0102] Step 113, performing distortion correction on the original image according to the calibration parameters of the visual camera to obtain an environment image.

[0103] The calibration parameters may be obtained by using the Zhang Zhengyou calibration method or any other lens calibration method. The calibration parameters include an intrinsic parameter matrix and a distortion vector distCoeffs (including 5 distortion coefficients, namely 3 radial distortion parameters and 2 tangential distortion parameters).

[0104] In step 113, a mapping relationship from the distorted pixel coordinates in the original image to the undistorted coordinate system can be established based on the intrinsic parameter matrix and the distortion vector, and based on the mapping relationship, the initUndistortRectifyMap function in OpenCV is used to calculate the distortion correction and dedistortion transformation mapping, and then the remap function is applied to remap the original image to obtain a dedistorted environment image.

[0105] Through the above steps 111 to 113, the original image taken by the visual camera 150 is subjected to distortion correction to eliminate the image error caused by lens distortion, thereby ensuring the accuracy of subsequent image processing, and thus helping to improve the positioning accuracy of automatic refilling.

[0106] Therefore, in step 13, the environment image after distortion correction in the above manner is input into the target detection model for global detection.

[0107] The target detection model used in step 13 may be any model that has been trained and matured, for example, a YOLOv5 model or a YOLOv4 model, etc. The above method is only an example, and its implementation method is not limited.

[0108] In order to increase the computational complexity of the model, improve the detection speed, and further improve the efficiency of refilling, the concept of knowledge distillation is introduced in the acquisition of the target detection model. Figure 5 , the process of acquiring the target detection model includes steps 21 to 25.

[0109] Step 21, annotate each sample image according to the bounding box of the charging pile to obtain a sample data set.

[0110] Here, we first need to clarify the shape of the charging pile and the image features of its front side (i.e., the charging side) (i.e., the charging pile 120 logo) to ensure that the features in the sample image have high discrimination and clarity, so as to improve the efficiency of subsequent model training.

[0111] During the stage of collecting sample images, the visual camera 150 carried by the robot 130 is used to photograph the charging pile 120 under various conditions, such as different lighting conditions, different viewing angles, with or without obstructions, so as to widely collect original images of the charging pile 120 .

[0112] Then, the original image is enhanced by increasing brightness, jittering colors, and other data enhancement processing. Then, a label annotation tool is used to annotate each original image obtained after data enhancement according to the shape and orientation of the charging pile 120 in the image (such as the left side, right side, upper left corner, upper right corner, lower right corner, lower left corner, etc. of the center of the image) and the face category of the charging pile 120 facing the robot 130 (such as the charging face, back side, left side, and right side, etc.), and all sample images obtained after annotation are used as a sample data set.

[0113] In this process, in order to eliminate the influence of image distortion and ensure the detection accuracy of the target detection model trained subsequently, a camera calibration method (such as Zhang Zhengyou calibration method, etc.) can also be used to accurately calibrate the poem camera on the robot 130 to obtain calibration parameters (such as the internal parameter matrix K and the distortion vector distCoeffs (including 5 distortion coefficients)). Then, the calibration parameters are used to perform distortion correction on each sample image. In this way, the image error caused by lens distortion is eliminated to ensure the accuracy of subsequent image processing.

[0114] Step 23, based on the sample data set, iteratively train the initial detection model to obtain a teacher model.

[0115] The initial detection model may be any target detection model, such as a YOLOv5 model, a YOLOv6 model, etc. The loss function, parameter adjustment, etc. used in iterative training may be any known method, which will not be described in detail here.

[0116] Step 25, using the teacher model and the sample data set, perform knowledge distillation on the student model to obtain the target detection model.

[0117] During the knowledge distillation process, the distillation method, the loss function used, etc. can be set flexibly and are not restricted here.

[0118] In order to improve the accuracy of the target detection model, an improved softmax layer is introduced in the knowledge distillation process. At this time, the knowledge distillation process can be as follows.

[0119] First, a sample image is extracted from the sample data set as the input image, and the input image is predicted by the trained teacher model Φ (excluding the softmax layer) to obtain the teacher output result Z, Z = Φ(X).

[0120] The teacher output result is then processed by the improved softmax layer to obtain the student output result p t . This process can be expressed by the following formula: Among them, T represents the introduced temperature variable. The level of the temperature variable affects the degree of attention paid to negative labels during the student model training process. i Indicates that the student outputs the result p t The i-th component of i Represents the i-th component of the teacher's output.

[0121] Then, based on the real label of the input image, the teacher output result, and the student output result, the loss function is used to calculate the distillation loss value of this iteration. The distillation loss value can be expressed as: Among them, y true represents the true label, p s represents the output of the student model, L hard represents the loss function between the student output and the true label, L soft represents the loss function between the teacher output and the student output, β are weight values.

[0122] Update the current number of iterations. When the number of iterations does not reach the target number, adjust the parameters of the student model and return to the step of extracting sample images from the sample dataset as input images. When the number of iterations is equal to the target number, the student model with the lowest distillation loss value during training is used as the target detection model.

[0123] In the above steps 21 to 25, the data enhancement processing of the sample images is used to enhance the robustness of the target detection model to external conditions such as illumination and angle changes. At the same time, distillation learning is used to enable the smaller model to learn the rich knowledge and feature representation of the larger model, greatly improving the performance of the target detection model, such as detection speed.

[0124] Therefore, after processing the environment image in step 13 using the target detection model trained in steps 21 to 25, the boundary box of the charging pile 120 in the image and the face category of the side of the charging pile 120 facing the robot 130 can be identified, and a global detection result can be obtained.

[0125] In order to subsequently determine whether the robot 130 is located in front of the charging surface, and to facilitate the robot 130 to quickly move to the front of the charging surface when the robot 130 is not located in front of the charging surface, the positioning information obtained in step 13 may include the actual distance and direction of the robot relative to the charging pile, and the face category of the side of the charging pile 120 facing the robot 130 in the environmental image.

[0126] Among them, refer to Figure 6 The process of obtaining positioning information in step 13 may include steps 131 to 133.

[0127] Step 131 : Using the position of the bounding box in the environment image as the position of the robot relative to the charging pile.

[0128] Among them, in order to make the orientation more accurate, the angle between the center point of the bounding box and the center of the image can be used as the orientation. For example, the intrinsic parameter matrix (including focal length and principal point coordinates) of the visual camera 150 can be obtained, and the pixel coordinates of the center point of the bounding box in the environmental picture can be determined. Then, the inverse matrix of the intrinsic parameter matrix is ​​multiplied by the matrix composed of pixel coordinates to obtain the camera coordinates of the pixel coordinates in the camera coordinate system. According to the camera coordinates and the origin of the camera coordinate system, the angle between the center point of the bounding box and the visual camera 150 can be calculated by using trigonometric functions, and the angle can be used as the orientation of the robot relative to the charging pile. In addition, the position of the visual camera 150 and the position of the center point of the bounding box can be converted to the world coordinate system, and then the angle between the two can be calculated by using trigonometric functions as the orientation of the robot relative to the charging pile. And the above method is only an example, and its implementation method is not limited.

[0129] The coordinates of the center point of the image (ie, the principal point of the visual camera 150) are extracted from the coordinates, and the coordinates of the bounding box are determined, and based on this, the center point of the image is calculated.

[0130] Step 133, calculate the area of ​​the bounding box, input the area into a pre-fitted distance calculation formula, and obtain the actual distance of the robot relative to the charging pile.

[0131] Here, the distance calculation formula is preset based on historical data or experimental data, and is obtained after model fitting and training. It represents the relationship between the area of ​​the bounding box in the image and the actual distance (i.e., between the robot 130 and the charging pile 120 in the world coordinate system when the image is obtained). Therefore, when the area of ​​a bounding box is input, the actual distance corresponding to the area can be output.

[0132] It should be understood that the distance calculation formula is not limited, and the distance calculation formula is different in the case of different visual cameras 150, robots 130 of different sizes, or different charging piles 120.

[0133] After obtaining the positioning information in step 13, in step 15, it is possible to determine whether the charging pile 120 is located in front of the charging surface of the robot 130 according to the surface category in the positioning information. For example, when the surface category is a charging surface, the charging pile 120 is located in front of the charging surface of the robot 130, otherwise, the charging pile 120 is not located in front of the charging surface.

[0134] When the charging pile 120 is not in front of the charging surface, in order to enable the robot 130 to quickly reach the charging surface, refer to Figure 7 The automatic recharging method provided in the embodiment of the present invention also includes step 16 and step 18. When it is determined in step 15 that the robot is not located in front of the charging surface of the charging pile, step 16 is executed.

[0135] Step 16, combining the surface type, actual distance and orientation, to obtain a motion path from the current position to the target position in front of the charging surface.

[0136] Step 18: Control the robot to move to the target position in front of the charging surface according to the motion path. After step 18, return to step 11.

[0137] That is, after the robot 130 moves to the target position in front of the charging surface according to the motion path, it returns to step 11. Alternatively, while the robot 130 moves to the target position in front of the charging surface according to the motion path, the above steps 11 to 18 are continuously repeated to correct and adjust the path and the motion of the robot 130 to ensure that it reaches the front of the charging surface quickly and accurately.

[0138] The method of obtaining the motion path can be flexibly selected. For example, any path planning algorithm can be used for planning, or it can be planned according to preset rules, or it can be planned manually. The above methods are only examples, and their implementation methods are not limited.

[0139] In order to shorten the motion path as much as possible and improve the efficiency of automatic recharging, refer to Figure 8 The process of obtaining the motion path in step 16 includes the following steps 161 to 163.

[0140] Step 161, calculate the current position coordinates of the robot in the world coordinate system.

[0141] For example, the origin coordinates of the camera coordinate system can be multiplied by the external parameter matrix (including the rotation matrix and the translation matrix) of the visual camera 150 to obtain the position coordinates of the visual camera 150 in the world coordinate system, and the position coordinates are used as the current position coordinates of the robot 130. The current position coordinates of the robot 130 can also be calculated according to other preset rules, and the implementation method is not limited.

[0142] Step 163 , taking the position coordinates as the starting point and the target position in front of the charging surface as the target point, path planning is performed to obtain a motion path.

[0143] Among them, the target position in front of the charging surface is preset. Based on the starting point and the target point, a rectangular plane can be obtained, and the part of the rectangular plane that overlaps with the charging pile 120 is removed to obtain a path plane, and then the path plane is gridded. Then, taking the position coordinates as the starting point and the target position in front of the charging surface as the target point, the path is planned on the path plane using the rapidly expanding random tree algorithm, the A* algorithm or any path planning algorithm, and the optimal motion path (i.e., the motion path with the shortest distance) can be obtained.

[0144] When it is determined in step 15 that the robot 130 is located in front of the charging surface of the charging pile 120, traditional image processing techniques such as image enhancement, grayscale, binarization, feature extraction, etc. can be used in step 16 to combine multiple features such as color, texture, and edge for local detection. Deep learning algorithms can also be used for local detection, and the implementation method is not restricted.

[0145] In order to quickly and accurately obtain the local detection result, in step 17, a local image of the charging pile 120 is cropped from the environment image, and the key point detection model is used to detect the local image. Fig. 9 The process of obtaining the key point position in step 17 includes steps 171 to 177.

[0146] Step 171, cropping the environment image to obtain a partial image of the bounding box.

[0147] It should be noted that the bounding box here refers to the bounding box of the charging pile.

[0148] Step 173, enlarge the local image, and detect the enlarged local image to obtain a local detection result.

[0149] Here, firstly, the image pyramid technology, vector image magnification or any other image magnification method can be used to enlarge the local image, and the magnification of the local image can be determined by the accuracy requirements and the equipment performance of the machine, for example, it can be 4 times or 5 times, and the specific data is not limited.

[0150] Next, the trained key point detection model (the key point detection model is trained to detect the key points of the key identification area of ​​the charging pile in the picture) can be used to detect the enlarged local picture to obtain multiple key points. Then, for each key point, the local picture is centered on the key point and expanded by a preset number of pixels (such as 1, 2, 7 or 5, etc.) around the key point (such as four directions of up, down, left and right, or eight directions of 45°, 90°, 135°, 180°, 225°, 270°, 314° and 360°, etc.) to obtain the key area of ​​the key point.

[0151] For example, when the local image is a 3*3 image and the key point is the 5th pixel of the local image, after 1 pixel in each of the eight directions of 45°, 90°, 135°, 180°, 225°, 270°, 314° and 360°, the key areas including the 1st, 2nd, 3rd, 4th, 5th, 6th, 7th, 8th and 9th pixels are obtained.

[0152] Among them, the key identification area is a pre-set feature area that needs to be identified, which may include the charging port of the charging pile 120 and the prompt word of the charging port, the keyword area on the charging pile 120, the charging indicator light, a specific pattern or shape, etc.

[0153] The local detection results include the key areas of each key point. In addition, deep learning algorithms or traditional image processing techniques can be used to determine whether the key areas of the charging pile are blocked, and the blockage status of each key area can be obtained. In this case, the local detection results also include the blockage status of each key area.

[0154] Step 175 , when the local detection result indicates that the key areas of the charging pile are not blocked, a key point position is extracted from each key area.

[0155] Step 177 , when the local detection result indicates that at least one key area of ​​the charging pile is blocked, a clearing alarm is issued.

[0156] In step 175 and step 177, it can be determined whether any key area is blocked according to the number of key identification areas identified. When the number of key areas identified is less than the total number of preset key identification areas, or the detection result indicates that the key area is dirty or blocked, it means that at least one key area is blocked.

[0157] In step 175, the center point of the key area can be directly used as the key point position, or the point with the most information (ie, the most conspicuous) in the key area can be used as the key point position, and the line of sight method for extracting the key point position is not limited.

[0158] Considering that the center point may be distorted by the shooting angle, the size of the bounding box, etc., resulting in inaccuracy, in order to accurately extract the key point position, the idea of ​​preprocessing the image and edge detection is introduced in step 175 to extract the key point position. Fig.10 , the process of extracting key point positions includes steps 1751 to 1755.

[0159] Step 1751, for each key area, enlarge the image of the key area to obtain the original key image.

[0160] Similar to the local image magnification in step 173 above, the image of the key area can be magnified by using image pyramid technology, vector image magnification or any other image magnification method.

[0161] Step 1753, grayscale and morphological operations are performed on the original key image to obtain a preprocessed key image.

[0162] Among them, morphological operations include but are not limited to corrosion and expansion. In this way, the original key image is converted into a grayscale image through grayscale conversion to reduce processing complexity and highlight key features. Morphological operations such as corrosion and expansion are applied to remove noise, connect broken lines or separate adhesion areas to highlight key features in the image.

[0163] Step 1755, edge detection is performed on the preprocessed key image to extract the pixel point with the largest gradient change as the target point, and the coordinates of the target point are used as a key point position.

[0164] It should be noted that the edge detection in step 1755 can also be replaced by corner detection. Both edge detection and corner detection are mature image processing methods and will not be described in detail here.

[0165] Through the above steps 1751 to 1755, the pixel point with the largest gradient change is the most conspicuous point with the richest information in the key area, so for the same key area, the key point position extracted each time is the most conspicuous point in the key area, that is, the same position point. In this way, the accuracy of the key point position is ensured.

[0166] After extracting all the key point positions, the method of obtaining the current position of the robot relative to the charging pile (i.e., the precise positioning position) in step 17 can be flexibly selected. For example, the key point positions can be connected to obtain a closed figure, and the current position can be calculated based on the closed figure and trigonometric functions, or the current position can be calculated using preset rules.

[0167] In order to quickly and accurately obtain the current position of the robot relative to the charging pile, the idea of ​​using a pre-trained regression model to generate the current position is introduced in step 17. Specifically, multiple key point positions can be input into the regression model, and the regression model can be used to generate angles and distances to obtain the current position of the robot relative to the charging pile.

[0168] Among them, the regression model is a model obtained after fitting training using historical data. The model learns the relationship between the key point position and the relative position (that is, the current position of the robot relative to the charging pile), so that it can output the current position of the robot relative to the charging pile based on the position of each key point.

[0169] Reference Fig.11 , the process of obtaining the regression model includes steps 31 to 37.

[0170] Step 31, obtaining multiple key point images and the key point positions in each key point image.

[0171] Among them, each key point image includes all the key points of the charging pile.

[0172] Step 33, marking the key point positions of the key point images according to the distance and angle of the robot relative to the charging pile when taking the key point images, so as to obtain a plurality of sample data.

[0173] It can be understood that in steps 31 to 33, the geometric features of the specific identification of the charging pile 120 (such as the charging indicator light, specific pattern or shape, etc.) are analyzed. These features have clear shapes, sizes and relative position relationships, which are convenient for subsequent calculations. When marking, the distance and angle of the robot 130 relative to the charging pile 120 when taking the key point picture is used as the real label of the key point position of the key point picture.

[0174] Step 35, combining the error function and the model complexity function to obtain the target loss function.

[0175] Step 37, based on the sample data and the target loss function, iteratively train the initial XGBoost model to obtain a regression model.

[0176] The output of the XGBoost model can be expressed as: F={f(x)=w q(x)}(q:R m →{1,2,…,t},w∈R t , K represents K trees, F represents the set of CART decision tree structures, q represents the tree structure of samples mapped to leaf nodes, t represents the number of leaf nodes, w represents the real number score of leaf nodes, n represents the number of key point positions, R mRepresents the m-dimensional feature vector of the key point position, and x represents the key point position (i.e. coordinate).

[0177] The objective loss function can be expressed as: Obj = L + Ω, γt is the L1 regularization term, is the L2 regularization term, y i is the true label, w j represents the i-th weight vector.

[0178] Iterative training is to quickly find the parameters that minimize the objective function. By performing a second-order Taylor expansion of the objective function through the XGBoost model, an approximate objective function can be obtained. If the structure part q of the tree is known, the objective function can be used iteratively to find the optimal w j , and obtain the optimal objective function value to obtain the regression model. At the same time, during the iterative training process, the hyperparameters of the XGBoost regression model (learning rate, number of trees, maximum depth of trees, regularization parameters, etc.) can use genetic algorithms to determine the optimal hyperparameter combination.

[0179] Through the above steps 31 to 37, a regression model can be quickly obtained, and the regression model can generate relative position information between the robot 130 and the charging pile 120 (ie, the current position of the robot relative to the charging pile) based on the key point positions.

[0180] By adopting the above steps 11 to 17 and 16 to 18, the robot 130 is moved to the front of the charging surface of the charging pile 120, and the current position (including distance and angle) of the robot relative to the charging pile is obtained by local detection. Then, in step 19, when the distance between the robot 130 and the charging pile 120 is equal to the safety distance threshold, it switches to a more sophisticated docking mode. In this docking mode, the robot 130 will use a fine-tuning mechanism to accurately adjust its posture and position to ensure that the charging interface can dock with the charging pile 120 accurately. Once the docking is successful, the robot 130 will start the charging program and start the battery charging operation. At the same time, the robot 130 will continue to monitor the charging status and be ready to respond to any possible abnormal situation at any time.

[0181] The fine-tuning mechanism may be any known method, which will not be elaborated here.

[0182] Based on the same inventive concept as the above automatic recharging method, refer to Fig.12The embodiment of the present invention also provides an automatic recharging device 30, including an image acquisition module 310, a coarse positioning module 320, a position determination module 330, a fine positioning module 340 and a precise docking module 350. The image acquisition module 310, the coarse positioning module 320, the position determination module 330, the fine positioning module 340 and the precise docking module 350 can be Figure 1 Functional modules of the host 140 in the automatic recharging system 10.

[0183] The image acquisition module 310 is used to acquire the environment image taken in real time by the visual camera, wherein the visual camera is arranged on the robot.

[0184] The coarse positioning module 320 is used to input the environment image into the target detection model, use the target detection model to obtain the global detection result, and obtain the positioning information of the robot relative to the charging pile based on the global detection result.

[0185] The position determination module 330 is used to determine whether the robot is located in front of the charging surface of the charging pile based on the positioning information.

[0186] The precise positioning module 340 is used to perform local detection on the environment image when the robot is located in front of the charging surface, obtain multiple key point positions of the charging pile, and obtain the current position of the robot relative to the charging pile based on the multiple key point positions.

[0187] The precise docking module 350 is used to adjust the position and / or posture of the robot based on the current position so that the charging interface of the robot can be docked with the charging pile.

[0188] Furthermore, it may also include an orientation adjustment module, which is used to: when the robot is not located in front of the charging surface of the charging pile, obtain a motion path according to the current position of the robot and the target position in front of the charging surface, and control the robot to move to the target position in front of the charging surface based on the motion path.

[0189] The automatic recharging device 30 may further include a first model acquisition module and a second model acquisition module. The first model acquisition module and the second model acquisition module may be Figure 1 Functional modules of the model training device 110 in the automatic recharging system 10.

[0190] The first model acquisition model is used to execute steps 21 to 25 in the above automatic recharging method.

[0191] The second model acquisition module is used to execute steps 31 to 37 in the above automatic recharging method.

[0192] The above-mentioned automatic recharging device 30, under the coordinated action of the image acquisition module 310, the coarse positioning module 320, the position judgment module 330, the fine positioning module 340 and the precise docking module 350, uses the integrated use of image processing, target detection technology, etc., and uses the target detection technology to first roughly locate the position of the charging pile, so as to quickly narrow the search range and improve the efficiency of subsequent processing. Then, after the robot moves to the front of the charging surface, deep local detection is used to accurately locate the position of the charging pile, and the robot posture is adjusted accordingly to ensure that the robot charging interface is docked with the charging pile. Thereby, the influence of external conditions (such as light, lack of detection generalization, etc.) is weakened or even eliminated, and the robot can effectively realize the precise positioning and efficient recharging of the charging pile.

[0193] For the specific implementation and effect of the automatic recharging device 30, please refer to the description of the implementation of the automatic recharging method above, such as, for the specific implementation and effect of the image acquisition module 310, please refer to the description of the relevant content of step 11 above, for the specific implementation and effect of the coarse positioning module 320, please refer to the description of the relevant content of step 13 above, for the specific implementation and effect of the position judgment module 330, please refer to the description of the relevant content of step 15 above, for the specific implementation and effect of the precise positioning module 340, please refer to the description of the relevant content of step 17 above, for the specific implementation and effect of the precise docking module 350, please refer to the description of the relevant content of step 19 above, and no further details will be given here.

[0194] In addition, each module in the automatic recharging device 30 can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor 220 in the electronic device 20 in the form of hardware, or can be stored in the memory 210 of the electronic device 20 in the form of software, so that the processor 220 can call and execute the operations corresponding to each module to implement the automatic recharging method provided above.

[0195] Based on the same inventive concept as the above-mentioned automatic recharging method, an embodiment of the present invention further provides a robot 130, including a main body, and a host 140 and a visual camera 150 arranged on the main body.

[0196] The visual camera 150 is used to take real-time photos of the environment to obtain an environmental picture.

[0197] The host 140 is used to implement the automatic recharging method provided above.

[0198] An embodiment of the present invention also provides an electronic device 20, including a processor 220 and a memory 210, wherein the memory 210 stores a computer program that can be executed by the processor 220, and the processor 220 can execute the computer program to implement the automatic recharging method proposed in the embodiment of the present invention.

[0199] The embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is executed by the processor 220, the automatic recharging method proposed in the embodiment of the present invention is implemented.

[0200] In addition, an embodiment of the present invention further provides a computer program product, which includes computer instructions, wherein when the computer instructions are executed by the processor 220, the automatic recharging method proposed in the embodiment of the present invention is implemented.

[0201] In summary, the automatic recharging method, device, robot and electronic device provided by the embodiments of the present invention have at least the following beneficial effects:

[0202] (1) Achieving efficient and accurate recharging of the robot by integrating target detection, deep learning algorithms and traditional image processing;

[0203] (2) Through the introduction of deep learning technology, the robot system's ability to adapt to different environments and lighting conditions has been improved, and its generalization has been enhanced;

[0204] (3) Combined with traditional image processing methods, the output of the deep learning algorithm (i.e., local detection results) is finely adjusted to ensure the accuracy of precise positioning;

[0205] (IV) There is no need to strictly restrict the location of the charging pile. It is only necessary to ensure that the robot can identify the charging pile within a certain range.

[0206] (V) Compared with the solution that solely relies on high-cost sensors or complex mechanical structures, the present invention does not need to introduce other sensors, thereby reducing system costs while ensuring performance.

[0207] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0208] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0209] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.

[0210] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An automatic recharging method, characterized in that: The method comprises: Obtaining an environmental picture taken in real time by a visual camera; wherein the visual camera is arranged on the robot; Inputting the environment image into a target detection model, obtaining a global detection result using the target detection model, and obtaining positioning information of the robot relative to the charging pile according to the global detection result; According to the positioning information, determining whether the robot is located in front of the charging surface of the charging pile; If yes, perform local detection on the environment image to obtain multiple key point positions of the charging pile, and obtain the current position of the robot relative to the charging pile based on the multiple key point positions; Based on the current position, the position and / or posture of the robot is adjusted so that the charging interface of the robot is docked with the charging pile.

2. The automatic recharging method according to claim 1, characterized in that: The positioning information includes the actual distance and orientation of the robot relative to the charging pile, and the surface category of the charging pile in the environment picture; After the step of determining whether the robot is located in front of the charging surface of the charging pile according to the positioning information, the method further includes: If the robot is not located in front of the charging surface of the charging pile, a motion path is obtained according to the current position of the robot and the target position in front of the charging surface; According to the motion path, the robot is controlled to move to the target position in front of the charging surface, and returns to execute the step of obtaining the environment picture taken in real time by the visual camera.

3. The automatic recharging method according to claim 2, characterized in that: The step of obtaining a motion path according to the current position of the robot and the target position in front of the charging surface comprises: Calculate the current position coordinates of the robot in the world coordinate system; Taking the position coordinates as a starting point and the target position in front of the charging surface as a target point, path planning is performed to obtain a motion path.

4. The automatic recharging method according to claim 1, characterized in that: The detection result includes a bounding box of the charging pile; The step of performing local detection on the environment image to obtain the positions of multiple key points of the charging pile includes: Cropping the environment image to obtain a partial image of the bounding box; Enlarging the local image, and detecting the enlarged local image to obtain a local detection result; When the local detection result indicates that each key area of ​​the charging pile is not blocked, extracting a key point position from each key area; When the local detection result indicates that at least one key area of ​​the charging pile is blocked, a cleaning alarm is issued.

5. The automatic recharging method according to claim 4, characterized in that: The step of extracting a key point position from each of the key areas comprises: For each of the key areas, the image of the key area is enlarged to obtain an original key image; Performing grayscale conversion and morphological operations on the original key image to obtain a preprocessed key image; Edge detection is performed on the preprocessed key image to extract the pixel point with the largest gradient change as the target point, and the coordinates of the target point are used as a key point position.

6. The automatic recharging method according to any one of claims 1 to 5, characterized in that: The step of obtaining the current position of the robot relative to the charging pile according to the positions of the plurality of key points comprises: Inputting the positions of the plurality of key points into a regression model, and using the regression model to generate angles and distances to obtain the current position of the robot relative to the charging pile; The method for obtaining the regression model includes: Acquire multiple key point images and key point positions in each of the key point images; wherein each of the key point images includes all key points of the charging pile; Marking the key point positions of the key point images according to the distance and angle of the robot relative to the charging pile when taking the key point images, so as to obtain a plurality of sample data; Combining the error function and the model complexity function, we get the target loss function; Based on the sample data and the target loss function, the initial XGBoost model is iteratively trained to obtain a regression model.

7. The automatic recharging method according to any one of claims 1 to 5, characterized in that: The global detection result includes a bounding box of the charging pile and a face category of the charging pile in the environment picture; The step of obtaining the positioning information of the robot relative to the charging pile according to the global detection result includes: Using the position of the bounding box in the environment image as the orientation of the robot relative to the charging pile; The area of ​​the bounding box is calculated, and the area is input into a pre-fitted distance calculation formula to obtain the actual distance of the robot relative to the charging pile.

8. The automatic recharging method according to any one of claims 1 to 5, characterized in that: The method for acquiring the target detection model includes: Label each sample image according to the bounding box of the charging pile to obtain a sample data set; Based on the sample data set, iteratively train the initial detection model to obtain a teacher model; Using the teacher model and the sample data set, knowledge distillation is performed on the student model to obtain a target detection model.

9. An automatic refilling device, characterized in that: It includes image acquisition module, coarse positioning module, position judgment module, fine positioning module and precise docking module; The image acquisition module is used to acquire the environment image taken in real time by the visual camera; wherein the visual camera is arranged on the robot; The coarse positioning module is used to input the environment image into the target detection model, obtain a global detection result using the target detection model, and obtain the positioning information of the robot relative to the charging pile according to the global detection result; The position determination module is used to determine whether the robot is located in front of the charging surface of the charging pile according to the positioning information; The precise positioning module is used to perform local detection on the environment image when the robot is located in front of the charging pile, obtain multiple key point positions of the charging pile, and obtain the current position of the robot relative to the charging pile based on the multiple key point positions; The precise docking module is used to adjust the position and / or posture of the robot based on the current position so that the charging interface of the robot can be docked with the charging pile.

10. A robot, characterized in that: It includes a body, and a host and a visual camera arranged on the body; The visual camera is used to take real-time photos of the environment to obtain an environmental picture; The host is used to implement the automatic recharging method according to any one of claims 1 to 8.

11. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores a computer program that can be executed by the processor, and the processor can execute the computer program to implement the automatic recharging method as described in any one of claims 1 to 8.

Images