Visual processing-based automatic pile returning and charging method for self-moving equipment

Abstract

The invention discloses a visual processing-based automatic pile returning charging method for self-moving equipment, which comprises the following steps of: judging whether the electric quantity of a battery is higher than 20% in a mowing process by the self-moving equipment, if so, maintaining a normal working mode, and if not, triggering an automatic pile returning mode; the self-moving device walks anticlockwise / clockwise along the grassland boundary and searches for a charging pile identifier in the automatic pile returning mode; and after the camera of the self-moving device detects the charging pile identifier, the processor of the self-moving device drives the self-moving device to return to the charging pile for charging. According to the invention, the problem of insufficient cruising ability caused by lack of a pile returning function of a visual sensing mower is improved, and the user experience of the mower is improved.

Description

Technical Field

[0001] The invention relates to a method for automatically returning a mobile device to a charging pile based on visual processing, and belongs to the technical field of lawn mowers. Background Art

[0002] When the mowing robot autonomously returns to the charging station for charging, it needs to have high positioning and navigation accuracy. The currently widely used charging method usually relies on laying guide lines on the floor of the charging station, which not only increases the cost, but also makes the installation process complicated and inflexible.

[0003] Traditional methods of returning to the pile include the use of ultrasonic sensors, infrared sensors, and global positioning system (GPS) technologies. Among them, the ultrasonic solution is easily affected by the surrounding noise and reduces accuracy; infrared technology is sensitive to strong light or direct sunlight, which may cause signal interference; as for the GPS-based positioning method, in places where there are a lot of obstacles such as trees, buildings, and other places where the GPS signal is weak, the signal reception quality will decrease, thus affecting the robot's precise positioning and navigation capabilities.

[0004] The present invention aims to solve the limitations of the above-mentioned prior art and proposes an innovative solution for returning to the charging pile based on visual recognition technology. Through advanced image processing algorithms and pattern recognition technology, the solution enables the mowing robot to efficiently and accurately identify and locate the charging pile, and maintain good working performance even in complex and changeable outdoor environments, thereby achieving a more reliable and stable automatic return to the charging pile function. This method not only reduces the requirements for external facilities and simplifies the deployment process, but also improves the overall robustness and adaptability of the system. Summary of the invention

[0005] The purpose of the present invention is to overcome the problems existing in the prior art and provide a method for automatically returning a mobile device to a charging pile based on visual processing. Through advanced image processing algorithms and pattern recognition technology, the mowing robot can efficiently and accurately identify and locate the charging pile, and maintain good working performance even in complex and changeable outdoor environments, thereby achieving a more reliable and stable automatic charging function. This method not only reduces the requirements for external facilities and simplifies the deployment process, but also improves the overall robustness and adaptability of the system.

[0006] In order to solve the above technical problems, a method for automatically returning a mobile device to a charging pile based on visual processing of the present invention comprises the following steps: S1: During the mowing process, the self-mobile device determines whether the battery power is higher than a preset threshold. If so, the normal working mode is maintained. If not, the automatic return to pile mode is triggered. S2: The mobile device is in the automatic return mode and walks counterclockwise / clockwise along the grass boundary and looks for the charging pile logo; S3: When the camera of the mobile device detects the charging pile logo, its processor drives the mobile device to return to the charging pile for charging.

[0007] Furthermore, assuming that the moving direction of the self-mobile device is forward, the charging pile logo is set at the front end of the charging pile, the charging pile logo is a QR code beacon, and a guide line is provided in the middle of the bottom plate of the charging pile along the moving direction of the self-mobile device.

[0008] Furthermore, S3 specifically includes the following steps: S301: The self-mobile device determines the pose information of the guide line through the guide line information output by the Transformer model, and the pose information includes the yaw angle and the roll angle, to determine the distance dist from the camera to the end of the guide line, and let the line L between the anchor point P at the end of the guide line and the center of the camera plane O point to the target heading, and the normal of the camera plane points to the current heading of the robot. The angle between the target heading and the current heading of the self-mobile device is called the yaw angle yaw; The guide line is regarded as the central axis of the charging pile, the normal line of the camera plane is regarded as the central axis of the body, and the angle between the two central axes is the roll angle; S302: When the roll angle and yaw angle displayed in the image are both positive, it means that both angles are biased to the left. The mobile device needs to move left to the central axis of the charging pile and then turn right 90°. The formula for the moving angle and distance is as follows: Angle of movement to the left : , the moving distance d: Where: d is the distance of lateral movement, is the distance from the end point P of the guide line to the camera plane, is the yaw angle, is the roll angle; similarly, when the roll angle and yaw angle shown are both negative, it means that both angles are biased to the right. The mobile device needs to move right to the central axis of the charging pile and then turn left 90°. The angle and distance calculation formulas are the same; S303: After the central axis of the body coincides with the central axis of the charging pile, the mobile device moves forward in a straight line at a constant speed to complete the return to the pile.

[0009] Furthermore, S301 trains the guide line dataset through the Transformer model. The Transformer model includes: Shared convolutional neural network backbone: input RGB image, output multi-scale features; Transformer encoder: enhances multi-scale features; Transformer decoder: Through the curve cross-attention mechanism, the curve sequence is updated, and the anchor points are iterated to approximate the visual beacon. The visual beacon is the guide line, and the curve is a curve formed by connecting the anchor points output after the guide line is processed by the Transformer model.

[0010] Furthermore, the Transformer model uses a dynamic anchor point set to interact with the image features and the curve query vector, and obtains the image features corresponding to the anchor points through the offline calibrated camera extrinsics; specifically, the following steps are included: Initialize the dynamic anchor point set, and the position of the anchor point is initialized based on the preset initial conditions; Through the cross attention mechanism, the similarity between the image features and the curve query vector is calculated to generate the attention weight; The image features are weighted summed using the attention weights, the curve sequence is updated, and the visual beacon is gradually approached; Through the offline calibrated camera extrinsics, the anchor point position in the image coordinate system is converted to the coordinate system of the mobile device to obtain the image features corresponding to the anchor point; In the Transformer decoder, a series of iterative updates are performed on the anchor point set. Each iteration adjusts the position of the anchor point according to the updated curve sequence until the preset convergence condition is met, the anchor point position change is less than the preset threshold or the preset number of iterations is reached.

[0011] Furthermore, the field of view of the camera is 135°, the angle between the normal line of the lens plane and the ground is 32°, and the ground within 3 meters in front of the lawn mower is photographed.

[0012] Furthermore, the self-moving device is a lawn mower, and the charging pile is arranged at the boundary of the lawn.

[0013] Furthermore, the preset threshold is 20%.

[0014] Compared with the prior art, the present invention has achieved the following beneficial effects: First, because there is no need to install an electronic fence, the lawn mower proposed in this solution eliminates the labor and time costs required to install and maintain the electronic fence compared to traditional lawn mowers. The lawn mower significantly improves the user experience by reducing the cost of using the lawn mower.

[0015] Secondly, because this solution uses the Transformer model for visual beacon recognition, it can achieve more accurate return to the charging pile compared to traditional lawn mowers. The lawn mower proposed in this solution can accurately identify the charging pile beacon and plan the return trajectory based on the beacon's posture. Because the trajectory planning is done in advance, the lawn mower proposed in this solution can calibrate the posture at a longer distance, thereby greatly improving the success rate of returning to the charging pile.

[0016] Finally, because it is equipped with a domestic AI chip, more new functions can be expanded, such as SLAM mapping, visual inertial odometer, etc. At the same time, the present invention does not need to adjust the camera's viewing angle to confirm the return status of the lawn mower. In summary, compared with traditional lawn mowers, the lawn mower proposed in this solution has stronger scalability while ensuring cost control. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The accompanying drawings are only provided for reference and explanation and are not intended to limit the present invention.

[0018] Figure 1 is a logic diagram of the present invention; Figure 2 Schematic diagram of the Transformer model in the present invention; Figure 3 It is a schematic diagram of a specific embodiment of the present invention; Figure 4 It is a schematic diagram of the lawn mower returning to the pile in the present invention. DETAILED DESCRIPTION

[0019] In the following description of the present invention, the directions or positional relationships indicated by terms such as "up", "down", "front", "back", "left", "right", "inside" and "outside" are based on the directions or positional relationships shown in the accompanying drawings and are only for the convenience of describing the present invention and simplifying the description, and do not mean that the device must have a specific direction.

[0020] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below with reference to specific diagrams.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0022] like Figure 1 As shown, the present invention aims to provide a method for automatically returning a mobile device to a charging pile based on visual processing, comprising the following steps: S1: The self-mobile device determines whether the battery power is higher than 20% during mowing. If so, the normal working mode is maintained. If not, the automatic return to charging pile mode is triggered. The self-mobile device is a lawn mower, and the charging pile is set at the boundary of the lawn.

[0023] S2: The mobile device is in the automatic return mode and walks counterclockwise / clockwise along the grass boundary and looks for the charging pile logo; S3: After the camera of the mobile device detects the charging pile logo, the processor of the mobile device drives the mobile device to return to the charging pile for charging. The camera has a field of view of 135°, and the angle between the normal line of the lens plane and the ground is 32°, shooting the ground within 3 meters in front of the lawn mower.

[0024] If the direction of movement of the mobile device is forward, the charging pile logo is set at the front of the charging pile. The charging pile logo is a QR code beacon, and a guide line is set in the middle of the bottom plate of the charging pile along the direction of movement of the mobile device. The lawn mower searches for the charging pile in the edge walking mode, and can find the location of the charging pile by pure visual navigation even when the satellite signal fails.

[0025] S3 specifically includes the following steps: S301: The present invention trains a guide line data set through a Transformer model, and determines the pose information of the guide line from the guide line information output by the Transformer model from the mobile device, the pose information including the yaw angle and the roll angle, to determine the distance dist from the camera to the end of the guide line, and makes the line L between the anchor point P at the end of the guide line and the center O of the camera plane point to the target heading, the normal of the camera plane points to the current heading of the robot, and the angle between the target heading and the current heading of the robot is called the yaw angle yaw; The guide line is regarded as the central axis of the charging pile, the normal line of the camera plane is regarded as the central axis of the body, and the angle between the two central axes is the roll angle; like Figure 4 As shown, S302: When the roll angle and yaw angle displayed in the image are both positive, it means that both angles are biased to the left. The mobile device needs to move left to the central axis of the charging pile and then turn right 90°. The formula for the moving angle and distance is as follows: Angle of movement to the left :

[0026] Distance moved d:

[0027] Where: d is the distance of lateral movement, is the distance from the end point P of the guide line to the camera plane, is the yaw angle, is the roll angle; similarly, when the roll angle and yaw angle shown are both negative, it means that both angles are biased to the right. The mobile device needs to move right to the central axis of the charging pile and then turn left 90°. The angle and distance calculation formulas are the same; S303: After the central axis of the body coincides with the central axis of the charging pile, the mobile device moves forward in a straight line at a constant speed to complete the return to the pile.

[0028] like Figure 2 As shown, in the guide line dataset of the Transformer model, each guide line image has a corresponding annotation file, which contains the actual guide line location information GT, which represents that when using the guide line dataset to train your own Transformer model, when the prediction and GT both cover a position, the algorithm will calculate the Euclidean distance from the predicted point to the GT value. When the Euclidean distance is less than the maximum allowed distance, that is, 0.05 meters, the algorithm considers the predicted point to be matched. When using the dataset to train the model, the average accuracy AUC, recall rate R, and the harmonic mean of accuracy recall F1 are also calculated to evaluate the performance of the model. The Transformer model includes: Shared convolutional neural network backbone: input RGB image, output multi-scale features; Transformer encoder: enhances multi-scale features; Transformer decoder: Through the curve cross attention mechanism, the curve sequence is updated and the anchor point is iterated to approximate the visual beacon - the guide line.

[0029] like Figure 3 As shown in the figure, the Transformer model uses a dynamic anchor point set to interact with the image features and the curve query vector, and obtains the image features corresponding to the anchor points through the offline calibrated camera extrinsics. Then, in the Transformer decoder, a series of iterations are performed on the anchor point set, which specifically includes the following steps: Initialize the dynamic anchor point set, and the position of the anchor point is initialized based on the preset initial conditions; Through the cross attention mechanism, the similarity between the image features and the curve query vector is calculated to generate the attention weight; The image features are weighted summed using the attention weights, the curve sequence is updated, and the visual beacon is gradually approached; Through the offline calibrated camera extrinsics, the anchor point position in the image coordinate system is converted to the coordinate system of the mobile device to obtain the image features corresponding to the anchor point; In the Transformer decoder, a series of iterative updates are performed on the anchor point set. Each iteration adjusts the position of the anchor point according to the updated curve sequence until the preset convergence condition is met, the anchor point position change is less than the preset threshold or the preset number of iterations is reached.

[0030] The above curve is a curve formed by connecting the anchor points output by the Transformer model after the guide line is processed, that is, the contour points of the guide line; At the same time, Transformer introduces a curve cross-attention mechanism module in the decoder to calculate the similarity between the curve query vector and the image features, and iterate the dynamic anchor point set of the guide line. Transformer introduces a context sampling unit to predict the offset after fusing the reference feature and the query vector to guide the learning of the sampling offset. Transformer uses the predicted offset to sample features. This mechanism effectively compensates for the error between the real-time external parameter and the offline external parameter, ensuring the robustness of the algorithm, so that the lawn mower can still maintain good working performance in complex and changeable outdoor environments.

[0031] In addition, Transformer introduces an auxiliary segmentation branch during the training phase to strengthen the image features of the shared convolutional neural backbone network. The prediction head of the Transformer network completes the output in the form of guide line parameters. Transformer applies a time fusion module that combines sparse curve queues and corresponding anchor point sets to form a continuous multi-frame information record.

[0032] The lawn mower of the present invention is equipped with Rockchip's RK3576 chip; this domestic AI chip has an 8-core CPU and a 6T computing power NPU, and can efficiently run the Transformer model; the program deployed by the RK3576 chip of the lawn mower has a self-starting function when it is turned on; after the lawn mower is turned on, the program in the RK3576 chip will initialize one thread to run the Transformer model; the program in the RK3576 chip will initialize one decoding channel for conversion between NV12 format and RGB format data; the program in the RK3576 chip will obtain NV12 format image data from the camera module at a rate of 30FPS; The program then inputs the NV12 format data into a decoding channel, converts it into a set of RGB format image data, and transmits it to the Transformer thread in real time; in the Transformer thread, the RGB image is calculated using the existing algorithm, the charging pile visual beacon is detected from the central area, and the position information of the visual beacon is calculated; after the trajectory calculation is completed, the calculation result will be sent to the microcontroller to guide the lawn mower to adjust its position step by step and steadily enter the charging pile.

[0033] Because the present invention does not require the installation of an electronic fence, the visual lawn mower proposed in this solution eliminates the labor and time costs required for installing and maintaining an electronic fence compared to a traditional lawn mower. The visual lawn mower significantly improves the user experience by reducing the use cost of the lawn mower user.

[0034] At the same time, the Transformer model is used for visual beacon recognition, which can achieve more accurate return to the charging pile compared to traditional lawn mowers. The visual lawn mower proposed in this solution can accurately identify the charging pile beacon and plan the return trajectory according to the beacon's posture. Because the trajectory planning is done in advance, the visual lawn mower proposed in this solution can calibrate the posture at a longer distance, thereby greatly improving the success rate of returning to the charging pile.

[0035] The above is only a preferred embodiment of the present invention, which shows and describes the basic principles and main features of the present invention and the advantages of the present invention, but does not limit the scope of patent protection of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. In addition to the above embodiments, the present invention may have other implementation modes without departing from the spirit and scope of the present invention. The present invention may also have various changes and improvements, and all technical solutions formed by equivalent substitution or equivalent transformation fall within the scope of protection required by the present invention. The scope of protection required by the present invention is defined by the attached claims and their equivalents. The technical features not described in the present invention can be realized by or using existing technologies, which will not be repeated here.

Claims

1. A method for automatically returning a mobile device to a charging pile based on visual processing, characterized in that: The following steps are involved: S1: During the mowing process, the mobile device determines whether the battery power is higher than a preset threshold. If so, the mobile device maintains a normal working mode. If not, the mobile device triggers an automatic return to pile mode. S2: The mobile device is in the automatic return mode and walks counterclockwise / clockwise along the grass boundary and looks for the charging pile logo; S3: When the camera of the mobile device detects the charging pile logo, its processor drives the mobile device to return to the charging pile for charging.

2. The method for automatically returning a mobile device to a charging pile based on visual processing according to claim 1 is characterized in that: Assuming that the moving direction of the self-mobile device is forward, the charging pile logo is set at the front end of the charging pile, the charging pile logo is a two-dimensional code beacon, and a guide line is provided in the middle of the bottom plate of the charging pile along the moving direction of the self-mobile device.

3. The method for automatically returning a mobile device to a charging pile based on visual processing according to claim 2 is characterized in that: S3 specifically includes the following steps: S301: The self-mobile device determines the position and posture information of the guide line through the guide line information output by the Transformer model, and the position and posture information includes the yaw angle and the roll angle, to determine the distance from the camera to the end of the guide line, and let the line L connecting the anchor point P at the end of the guide line and the center of the camera plane O point to the target heading, and the normal of the camera plane points to the current heading of the robot. The angle between the target heading and the current heading of the self-mobile device is called the yaw angle yaw; The guide line is regarded as the central axis of the charging pile, the normal line of the camera plane is regarded as the central axis of the body, and the angle between the two central axes is the roll angle; S302: When the roll angle and yaw angle displayed in the image are both positive, it means that both angles are biased to the left. The mobile device needs to move left to the central axis of the charging pile and then turn right 90°. The formula for the moving angle and distance is as follows: Left turn angle : , the moving distance d: Where: d is the distance of lateral movement, is the distance from the end point P of the guide line to the camera plane, is the yaw angle, is the roll angle; similarly, when the roll angle and yaw angle shown are both negative, it means that both angles are biased to the right. The mobile device needs to move right to the central axis of the charging pile and then turn left 90°. The angle and distance calculation formulas are the same; S303: After the central axis of the body coincides with the central axis of the charging pile, the mobile device moves forward in a straight line at a constant speed to complete the return to the pile.

4. The method for automatically returning a mobile device to a charging pile based on visual processing according to claim 3 is characterized in that: S301 trains the guide line dataset through the Transformer model. The Transformer model includes: Shared convolutional neural network backbone: input RGB image, output multi-scale features; Transformer encoder: enhances multi-scale features; Transformer decoder: Through the curve cross-attention mechanism, the curve sequence is updated, and the anchor points are iterated to approximate the visual beacon. The visual beacon is the guide line, and the curve is a curve formed by connecting the anchor points output after the guide line is processed by the Transformer model.

5. The method for automatically returning a mobile device to a charging pile based on visual processing according to claim 4 is characterized in that: The Transformer model uses a dynamic anchor point set to interact with the image features and the curve query vector, and obtains the image features corresponding to the anchor points through the offline calibrated camera extrinsics; specifically, the following steps are included: Initialize the dynamic anchor point set, and the position of the anchor point is initialized based on the preset initial conditions; Through the cross attention mechanism, the similarity between the image features and the curve query vector is calculated to generate the attention weight; The image features are weighted summed using the attention weights, the curve sequence is updated, and the visual beacon is gradually approached; Through the offline calibrated camera extrinsics, the anchor point position in the image coordinate system is converted to the coordinate system of the mobile device to obtain the image features corresponding to the anchor point; In the Transformer decoder, a series of iterative updates are performed on the anchor point set. Each iteration adjusts the position of the anchor point according to the updated curve sequence until the preset convergence condition is met, the anchor point position change is less than the preset threshold or the preset number of iterations is reached.

6. The method for automatically returning a mobile device to a charging pile based on visual processing according to claim 1 is characterized in that: The camera has a field of view of 135°, an angle between the normal line of the lens plane and the ground of 32°, and can capture the ground within 3 meters in front of the lawn mower.

7. The method for automatically returning a mobile device to a charging pile based on visual processing according to claim 1, characterized in that: The self-moving device is a lawn mower, and the charging pile is arranged at the boundary of the lawn.

8. The method for automatically returning a mobile device to a charging pile based on visual processing according to claim 1, characterized in that: The preset threshold is 20%.

Images