Return control method, device, computer-readable medium, and self-propelled equipment

Abstract

The present application belongs to the technical field of self-moving equipment, and specifically relates to a return control method, device, computer-readable medium and self-moving equipment. The method includes obtaining a visual image containing a base after the self-moving equipment enters a designated area; determining the base point cloud data of the base and the corresponding offset of the self-moving equipment relative to the base based on the visual image; calculating the distance from the base point cloud data to the preset reference point cloud data; obtaining the initial posture of the self-moving equipment; determining the positioning posture of the self-moving equipment based on the offset, distance, initial posture and preset error equation; and controlling the movement of the self-moving equipment based on the positioning posture so that the self-moving equipment docks with the base. By determining the final error value through the error equation, a more accurate posture of the self-moving equipment can be obtained, thereby improving the position matching accuracy of the self-moving equipment and the base, achieving accurate guidance of the self-moving equipment to return, and improving the accuracy of the return.

Description

Technical Field

[0001] The present application belongs to the technical field of self-moving equipment, and specifically relates to a return control method, device, computer-readable medium, and self-moving equipment. Background Art

[0002] With the development of technology, the application of autonomous equipment is becoming more and more widespread. When an autonomous equipment completes its work or runs out of power, it can return to the base (also called a charging station) by locating the base. This is called the return home of the autonomous equipment.

[0003] In related technical solutions, the base positioning device or surrounding environmental factors may often be affected, resulting in an inability to accurately locate the base position, further making it impossible for the mobile device to accurately navigate and return.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0005] The purpose of this application is to provide a return control method, device, readable medium and self-moving device, which to a certain extent improve the accuracy of the self-moving device when returning.

[0006] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by practice of the present application.

[0007] According to one aspect of an embodiment of the present application, a return control method is provided, the return control method comprising:

[0008] After the mobile device enters the designated area, a visual image including the base is obtained;

[0009] Determine the base point cloud data of the base and the corresponding offset of the mobile device relative to the base according to the visual image;

[0010] Calculate the distance between the base point cloud data and the preset reference point cloud data;

[0011] Get the initial pose from the mobile device;

[0012] Determine the positioning pose of the mobile device based on the offset, distance, initial pose and preset error equation;

[0013] Based on the positioning posture, the self-moving device is controlled to move so that the self-moving device docks with the base.

[0014] According to one aspect of an embodiment of the present application, a return control device is provided, the return control device comprising:

[0015] A first acquisition module is used to acquire a visual image including the base after the mobile device enters the designated area;

[0016] A first determination module is configured to determine base point cloud data of the base and a corresponding offset of the mobile device relative to the base based on the visual image;

[0017] A calculation module, used to calculate the distance between the base point cloud data and the preset reference point cloud data;

[0018] A second acquisition module is used to obtain an initial posture from the mobile device;

[0019] A second determination module is used to determine the positioning posture of the mobile device based on the offset, distance, initial posture and a preset error equation;

[0020] The mobile module is used to control the movement of the self-moving device based on the positioning posture so that the self-moving device can dock with the base.

[0021] In some embodiments of the present application, based on the above technical solution, the first determination module is also used to extract features from the visual image to obtain multiple contour lines of the base; and determine the point cloud data contained in the multiple contour lines as the base point cloud data of the base.

[0022] In some embodiments of the present application, based on the above technical solution, the device also includes a reference point cloud data acquisition module for acquiring a preset three-dimensional model of the base; performing feature extraction on the three-dimensional model of the base to obtain multiple reference contour lines of the base; and determining the point cloud data contained in the multiple reference contour lines as the reference point cloud data corresponding to the base.

[0023] In some embodiments of the present application, based on the above technical solution, the calculation module is also used to determine the reference point cloud data that matches the base point cloud data; use the reference contour line where the reference point cloud data is located as the target contour line; and determine the distance from the base point cloud data to the target contour line as the distance from the base point cloud data to the reference point cloud data.

[0024] In some embodiments of the present application, based on the above technical solution, the second determination module is also used to substitute the offset, distance, and initial posture into a preset error equation for calculation to obtain a total error value; when the total error value is greater than or equal to a preset error threshold, the initial posture is updated according to a preset update strategy, and the step of calculating the total error value is returned based on the updated initial posture; when the total error value is less than the preset error threshold, the latest initial posture is used as the positioning posture of the self-moving device.

[0025] In some embodiments of the present application, based on the above technical solution, the return control device also has a positioning module, which is used to obtain the positioning signal and posture information of the mobile device; when the positioning signal is not in the specified area, the mobile device is controlled to enter the specified area based on the positioning signal and posture information.

[0026] In some embodiments of the present application, based on the above technical solution, the first acquisition module is also used to detect and identify the visual image, determine the objects contained in the visual image and the category labels corresponding to each object; when it is detected that the visual image contains a category label belonging to the base, it is determined that the visual image contains the base.

[0027] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the return control method in the above technical solution is implemented.

[0028] According to one aspect of an embodiment of the present application, a self-moving device is provided, which includes: a vehicle body, including a vehicle body and wheels; and a control module for executing the return control method provided in any embodiment of the present application.

[0029] According to one aspect of an embodiment of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the return control method described in the above technical solution.

[0030] In the technical solution provided in the embodiment of the present application, the self-moving device is first controlled to enter a designated area, and a visual image is acquired in real time. The area in which the base is located is preliminarily determined by the visual image. Then, the corresponding base point cloud data is determined based on the visual image, and the offset of the self-moving device relative to the base, as well as the distance between the base point cloud data and the preset reference point cloud data, are determined. The positioning pose of the self-moving device is determined based on the offset, distance, initial position of the self-moving device, and a preset error equation. In this way, the offset of the self-moving device from the base is obtained by associating and matching the visual image and the base point cloud data, and the positioning pose of the self-moving device is constrained by the offset and error equation. Since the error equation can be used to verify whether the deviation of the positioning pose from the base is optimal, when the positioning pose is obtained by the error equation, a more accurate positioning pose can be obtained. Since this technical solution does not require the use of a positioning device, the factors affecting the positioning device by the environment are reduced, making the positioning robustness of the self-moving device more suitable for indoor and outdoor scenarios, and improving the accuracy of the self-moving device's return.

[0031] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0033] Figure 1 The following schematically shows a flow chart of the steps of the return control method provided in one embodiment of the present application.

[0034] Figure 2 The figure schematically shows the corresponding offset of the self-moving device relative to the base provided by an embodiment of the present application.

[0035] Figure 3 A specific flow chart for implementing step S102 in an embodiment of the present application is schematically shown.

[0036] Figure 4 A specific flow chart for implementing step S103 in an embodiment of the present application is schematically shown.

[0037] Figure 5 A schematic diagram of the distance between the base point cloud data and the reference point cloud data provided in an embodiment of the present application is schematically shown.

[0038] Figure 6 A schematic diagram of the distance between the base point cloud data and the reference point cloud data provided in another embodiment of the present application is schematically shown.

[0039] Figure 7 A flowchart of the return control method steps provided in another embodiment of the present application is schematically shown.

[0040] Figure 8 A specific flow chart for implementing step S105 in an embodiment of the present application is schematically shown.

[0041] Figure 9 The structural block diagram of the return control device provided in an embodiment of the present application is schematically shown.

[0042] Figure 10 The following schematically shows a block diagram of a computer system structure of a mobile device suitable for implementing the embodiments of the present application.

[0043] Figure 11 A schematic diagram of a self-moving device provided by an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0044] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0045] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner.In the following description, many specific details are provided so as to provide a full understanding of the embodiments of the present application. However, it will be appreciated by those skilled in the art that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps etc. can be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0046] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0047] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0048] In related technical solutions, when a mobile device returns to its destination, it typically uses an infrared sensor or depth camera to determine the relative position of the mobile device and the base. However, infrared sensors cannot properly measure the distance between the mobile device and the base in strong outdoor light, and depth cameras cannot accurately measure the distance between the mobile device and the base in dark environments. Therefore, the robustness of related technical solutions is poor.

[0049] To address the aforementioned technical issues, this application proposes a return-to-home control method. This method first controls a self-propelled device to enter a designated area, acquires a visual image in real time, and uses the visual image to preliminarily determine the area within which the base is located. The method then determines the corresponding base point cloud data based on the visual image, the offset of the self-propelled device relative to the base, and the distance between the base point cloud data and the preset reference point cloud data. The positioning pose of the self-propelled device is determined based on the offset, distance, the initial pose of the self-propelled device, and a preset error equation. Thus, the offset of the self-propelled device from the base is determined by associating and matching the visual image with the base point cloud data. The positioning pose of the self-propelled device is then constrained using the offset and error equation. Because the error equation can be used to verify whether the deviation of the positioning pose from the base is optimal, a highly accurate positioning pose can be obtained when the positioning pose is determined using the error equation. Since this technical solution does not require a positioning device, it reduces the environmental influences on the positioning device, resulting in more robust positioning of the self-propelled device. The method is suitable for both indoor and outdoor scenarios, improving the accuracy of the return-to-home (RTH) of the self-propelled device.

[0050] The following describes in detail the return control method, device, computer-readable medium, and mobile device provided by the present application in conjunction with specific implementation methods.

[0051] The method of this embodiment can be applied to the scenario of recharging from a mobile device. Specifically, see Figure 1 , Figure 1 The following schematically illustrates a flow chart of the return control method provided by an embodiment of the present application. The execution subject of the return control method may be a controller, and the method may mainly include the following steps S101 to S106.

[0052] Step S101: After the mobile device enters a designated area, a visual image including a base is acquired.

[0053] When the battery of the mobile device is low, or when a recharge instruction or a return instruction is received from the mobile terminal, the mobile device is controlled to return to the base. The recharge instruction or the return instruction can be used to instruct the mobile device to return to the base. During the process of controlling the mobile device to return to the base, the location of the mobile device is obtained in real time through the GPS (Global Positioning System). By obtaining the location of the mobile device, it is determined whether the mobile device has entered the designated area of ​​the base.

[0054] The autonomous device may include a device with autonomous mobility assistance functions, a semi-autonomous device, or a fully autonomous device. For example, the autonomous device may include a robot, a drone, or a smart car, such as a lawn mower robot, a food delivery robot, a mine sweeper robot, or a cleaning robot. This application does not limit the type of autonomous device.

[0055] Regarding the setting of the designated area, specifically, the designated area is an area range at a preset distance from the base. The designated area can be set according to the position of the base in the actual scene. For example, the designated area can be an area range at a preset distance of 5 meters, 6 meters, 10 meters, etc. from the base. The designated area can make it more likely that the self-mobile device will capture a visual image containing the base through the image acquisition device in the forward direction, that is, improve the success rate of capturing the visual image containing the base. At the same time, after the self-mobile device enters the designated area, a visual image containing the base is acquired. The capture of invalid visual images can be reduced, the time cost of subsequent processing operations can be reduced, and the efficiency of visual image capture can be improved. Therefore, when it is detected that the position of the self-mobile device is in the designated area, the image acquisition device can be started to capture images in the forward direction of the self-mobile device.

[0056] When it is detected that the self-moving device is not in the designated area, the self-moving device can be controlled to move forward into the designated area according to the location of the self-moving device. When it is detected that the location of the self-moving device is in the designated area, the image acquisition device can be activated to acquire visual images.

[0057] Step S102: determining the base point cloud data of the base and the corresponding offset of the mobile device relative to the base based on the visual image.

[0058] Among them, the visual image can be acquired by an image acquisition device, and the image acquisition device may include a visual camera and a laser camera. The visual camera and the laser camera are calibrated with external parameters so that the visual image acquired by the visual camera and the point cloud data acquired by the laser camera are synchronized. That is to say, during the return process of the self-mobile device, each pixel in the acquired visual image can be matched to the corresponding point cloud data. When the base is detected in the visual image, a label of the base can be set in the image area where the base is located, so that the pixels belonging to the base in the image area all carry the same label. By fusing the point cloud data with the visual image, the corresponding base point cloud data belonging to the base can be determined based on the pixel points carrying the label, so as to distinguish the point cloud data corresponding to the base from the point cloud data corresponding to other environmental information, filter out the point cloud data of non-base, and use the point cloud data of the base as the base point cloud data.

[0059] The offset refers to the distance that the mobile device is offset to the left or right relative to the base in the same coordinate system. By determining the offset of the mobile device relative to the base, it is helpful to determine the deflection angle of the mobile device. In other words, the offset can be used to constrain the yaw angle of the mobile device, making the final yaw angle pose more accurate. Figure 2 , Figure 2 The figure schematically shows the corresponding offset of the self-moving device relative to the base provided by an embodiment of the present application.

[0060] In some optional embodiments, see Figure 3 , Figure 3 The specific flow chart for implementing step S102 in one embodiment of the present application is schematically shown. According to the visual image, the base point cloud data of the base in the self-mobile device coordinate system is determined, which may specifically include the following steps S301 to S302.

[0061] Step S301: extract features from the visual image to obtain multiple contour lines of the base.

[0062] After obtaining the visual image, a target detection algorithm is used to extract features from the visual image to obtain multiple contour lines of the base. By performing feature extraction on the visual image containing the base, rather than on all visual images, the amount of data processing is reduced. Among them, the target detection algorithm can include one or more target detection algorithms such as R-CNN (Region-CNN, regional convolutional neural network), SPP (Spatial Pyramid Pooling, spatial pyramid pooling), YOLO (You Only Look Once, a regression method based on deep learning), and the selection of the target detection algorithm is not limited here.

[0063] Step S302 : determining the point cloud data included in the plurality of contour lines as the base point cloud data of the base.

[0064] Since each pixel point in the collected visual image can be matched to the corresponding point cloud data, after multiple contour lines of the base are obtained through feature extraction, each contour line will be matched to the corresponding point cloud data, and the point cloud data contained in each contour line will be determined as the base point cloud data of the base.

[0065] In this way, by performing feature extraction on the visual image to obtain multiple contour lines of the base, and by determining the point cloud data contained in the multiple contour lines as the base point cloud data of the base, it is beneficial to obtain effective base point cloud data while filtering out point cloud data of other irrelevant environmental information.

[0066] Step S103: Calculate the distance between the base point cloud data and the preset reference point cloud data.

[0067] The reference point cloud data represents the point cloud data used as a standard reference for the base. By calculating the distance between the base point cloud data and the preset reference point cloud data, it is helpful to determine the deviation between the actual base and the reference base, and further facilitate the determination of the positioning pose of the self-moving object.

[0068] In some optional embodiments, see Figure 4 , Figure 4 The specific flow chart for implementing step S103 in one embodiment of the present application is schematically shown. Calculating the distance between the base point cloud data and the preset reference point cloud data may specifically include the following steps S401 to S403.

[0069] Step S401: Determine reference point cloud data that matches the base point cloud data.

[0070] The reference point cloud data represents the point cloud data used as a standard reference for the base. When determining the reference point cloud data that matches the base point cloud data, for example, a Normal Distributions Transform (NDT) algorithm can be used to align the base point cloud data with the reference point cloud data, thereby determining the matching reference point cloud data. Using the matching reference point cloud data as a reference facilitates determining the distance between the actual base point cloud data and the reference point cloud data.

[0071] See also Figure 5 , Figure 5A schematic diagram of the distance between the base point cloud data and the reference point cloud data provided by an embodiment of the present application is schematically shown. Among them, L1', L2', L3' and L4' are multiple contour lines that constitute the base, that is, the contour lines actually measured, and L1, L2, L3 and L4 are reference contour lines that constitute the base model. When determining the target contour line, each contour line is matched and combined with each reference contour line, so that each contour line of each matching combination corresponds to a reference contour line. The distance between the contour line and the corresponding reference contour line is calculated, that is, the distance between the contour line where the base point cloud data is located and the reference contour line where the reference point cloud data is located is calculated, and the sum of the various distances in the same matching combination is obtained to obtain the total distance. When the total distance reaches the minimum, the reference point cloud data that matches the base point cloud data is determined, that is, the contour line in the matching combination is successfully matched with the reference contour line, and the matched contour line is used as the target contour line. After calculation, it is determined that L1' where the base point cloud data is located matches L1 where the benchmark point cloud data is located, L2' where the base point cloud data is located matches L2 where the benchmark point cloud data is located, L3' where the base point cloud data is located matches L3 where the benchmark point cloud data is located, and L4' where the base point cloud data is located matches L4 where the benchmark point cloud data is located.

[0072] Step S402: taking the reference contour line where the reference point cloud data is located as the target contour line.

[0073] Since the benchmark point cloud data is distributed in different contour lines, the reference contour line where the matched benchmark point cloud data is located is used as the target contour line, which helps to determine the distance between the base point cloud data and the benchmark point cloud data.

[0074] Step S403 : determining the distance from the base point cloud data to the target contour line as the distance from the base point cloud data to the reference point cloud data.

[0075] Get the straight line equation of the target contour line, for example, the straight line equation is Ax+By+C=0, where A, B, and C are constants. According to the coordinates of the points in the base point cloud data, for example, the coordinates of point O (Xo, Yo), the distance from point O to the target contour line is Then the distance between the base point cloud data and the benchmark point cloud data is obtained.

[0076] See also Figure 6 , Figure 6A schematic diagram of the distance between the base point cloud data and the reference point cloud data provided by another embodiment of the present application is shown. To calculate the distance between the base point cloud data and the reference point cloud data, take the base point cloud data on contour line L1' as an example. Contour line L1' includes points in the base point cloud data, such as a', b', and c'. The distance from a' to the target contour line L1, the distance from b' to the target contour line L1, and the distance from c' to the target contour line L1 are calculated respectively. This allows the distance between the base point cloud data and the reference point cloud data to be calculated. The same principle applies to other contour lines and is not further described here.

[0077] In this way, the reference point cloud data that matches the base point cloud data is first determined, and the distance between the base point cloud data and the reference reference point cloud data is calculated, which is conducive to a more accurate distance and further helps to determine the positioning posture of the mobile device.

[0078] Step S104: Acquire the initial posture of the mobile device.

[0079] The initial posture of the mobile device includes the initial orientation angle of the mobile device and the initial position of the mobile device. The initial posture can be determined by setting an empirical value, and those skilled in the art can set the specific value according to actual needs.

[0080] Step S105 , determining the positioning posture of the mobile device according to the offset, distance, initial posture and a preset error equation.

[0081] After determining the offset, distance, and initial pose through the above steps, the corresponding data is substituted into the preset error equation to determine the positioning pose of the mobile device. The preset error equation is:

[0082]

[0083] Among them, (R k ,t k ) represents the position of the mobile device in the base coordinate system at the kth moment, t k It represents the translation of the mobile device in the base coordinate system at the kth moment. k Also includes roll k represents the roll angle of the mobile device at the kth moment, pitch k Indicates the pitch angle of the mobile device at the kth moment, yaw k Indicates the yaw angle of the mobile device at the kth moment. is the jth point in the i-th contour line of the current base point cloud data, mx+n represents the equation of the line corresponding to the j-th point in the reference point cloud data, m and n are obtained from the reference point cloud data, that is, the corresponding m and n can be obtained according to the obtained target contour line. |((R k x ij +t k )-(m i x+n i ))| is the distance, d is the offset, w d is the weight of the error equation where the offset d is located. The weight value can be set according to actual needs. δ is the compensation value set when the yaw angle or pitch angle is both in the plane. p ,Σ d ,Σ x and Σ q is the information matrix of the corresponding error term.

[0084] In this way, by solving the error equation, the initial position is continuously adjusted to minimize the total error value, and finally a more accurate positioning position of the self-moving device is obtained, thereby improving the accuracy of the docking between the self-moving device and the base.

[0085] Step S106: Based on the positioning posture, the self-moving device is controlled to move so as to dock with the base.

[0086] Among them, the positioning posture includes positioning attitude and positioning position. After obtaining the positioning posture of the self-moving device, the posture and positioning position of the self-moving device are adjusted to correspond to the posture of the target, thereby realizing the docking of the self-moving device with the base.

[0087] In the technical solution provided in the embodiment of the present application, the self-moving device is first controlled to enter the designated area, and a visual image is obtained in real time. The area where the base is located is preliminarily determined by the visual image. Then, the corresponding base point cloud data is determined based on the visual image, and the offset of the self-moving device relative to the base, as well as the distance between the base point cloud data and the preset reference point cloud data are determined. The positioning posture of the self-moving device is determined based on the offset, distance, the initial posture of the self-moving device, and the preset error equation. In this way, the posture is continuously adjusted based on the initial posture of the self-moving device to minimize the error value of the entire error equation, and then a more accurate posture of the self-moving device is obtained, thereby improving the accuracy of the matching between the self-moving device and the base, achieving precise navigation of the self-moving device to return, and improving the accuracy of the return.

[0088] In some optional embodiments, see Figure 7 , Figure 7The following schematically illustrates a flow chart of the return control method provided by another embodiment of the present application. Before calculating the distance between the base point cloud data and the preset reference point cloud data, the method may specifically include the following steps S701 to S703.

[0089] Step S701: Obtain a preset three-dimensional model of the base.

[0090] The three-dimensional model of the base may be a model of the base, for example, obtained by computer-aided design (CAD) modeling, and the modeling method is not limited.

[0091] Step S702 : performing feature extraction on the three-dimensional model of the base to obtain a plurality of reference contour lines of the base.

[0092] By extracting features from the 3D model of the base, we obtain multiple reference contour lines for the base, which facilitates comparison of the contour lines with the reference contour lines. Since the method for extracting features from the 3D model of the base is similar to that for extracting features from visual images, we will not repeat the method for extracting features from the 3D model of the base.

[0093] Step S703 : determining the point cloud data included in the plurality of reference contour lines as the reference point cloud data corresponding to the base.

[0094] By identifying the point cloud data contained in each reference contour line as the base's corresponding benchmark point cloud data, more accurate benchmark point cloud data can be obtained. Using the benchmark point cloud data as a reference facilitates the matching and calibration of the base's point cloud data. The matching relationship between the contour line and the benchmark contour line can more accurately determine the final positioning pose of the autonomous device.

[0095] In some optional embodiments, see Figure 8 , Figure 8 The flowchart for implementing step S105 in one embodiment of the present application is schematically shown. The positioning posture of the mobile device is determined based on the offset, distance, initial posture, and a preset error equation, which may specifically include the following steps S801 to S803.

[0096] In step S801, the offset, distance, and initial posture are substituted into a preset error equation to calculate a total error value.

[0097] By setting a preset error equation, substituting the offset, distance, and initial pose into the error equation, the total error value is calculated. The total error value can be considered as the docking accuracy between the mobile device and the base.

[0098] Step S802: When the total error value is greater than or equal to the preset error threshold, the initial posture is updated according to the preset update strategy, and the process of calculating the total error value is returned to the updated initial posture.

[0099] The preset update strategy is: after setting the initial pose, for example, the initial pose is determined to be R k0 and t k0 , then calculate the total error according to the error equation. When the total error value is greater than or equal to the preset error threshold, adjust R k0 and t k0 , change the initial pose from the original R k0 and t k0 Update to R k1 and t k1 , and then substitute the posture into the error equation to recalculate the total error value. Repeat this cycle. When the total error value is less than the preset error threshold, R at this time is the optimal positioning posture.

[0100] Step S803: When the total error value is less than the preset error threshold, the latest initial posture is used as the positioning posture of the mobile device.

[0101] The error threshold can be pre-set. Those skilled in the art can determine the magnitude of the error threshold based on actual needs and are not limited herein. When the total error value is less than the preset error threshold, it can be assumed that the current positioning posture of the self-moving device is compatible with docking with the base. By continuously adjusting the positioning posture to minimize the total error value, the docking accuracy of the self-moving device and the base can be improved.

[0102] In this way, by calculating the total error value and continuously adjusting the initial posture to minimize the final total error value, accurate docking of the mobile device and the base can be achieved, and further precise navigation of the mobile device for return and recharging can be achieved, thereby improving the reliability and accuracy of the return and recharging.

[0103] In some optional embodiments, the method further comprises:

[0104] Obtain positioning signals and posture information from mobile devices;

[0105] When the positioning signal is not in the designated area, the mobile device is controlled to enter the designated area based on the positioning signal and posture information.

[0106] The self-moving device's positioning signal can be sent by RTK or GNSS, while its attitude information can be obtained by an IMU (Inertial Measurement Unit). Based on the positioning signal and attitude information, the self-moving device is guided into a designated area. This helps maintain the accuracy of the image acquisition device by first controlling the self-moving device within the designated area, resulting in better image acquisition results.

[0107] In some optional embodiments, after the mobile device enters the designated area, obtaining a visual image including the base includes:

[0108] Detect and identify visual images to determine the objects contained in the visual images and the category labels corresponding to each object;

[0109] After obtaining the visual image, an object detection algorithm is used to detect objects in the visual image, thereby determining the categories to which each object in the visual image belongs. The object detection algorithm may include one or more of the following: R-CNN (Region-CNN), SPP (Spatial Pyramid Pooling), YOLO (You Only Look Once, a deep learning-based regression method), and the like. The choice of the object detection algorithm is not limited herein.

[0110] In this way, the object detection algorithm detects and identifies the visual image to determine the category to which each object belongs, which facilitates classification of the visual image and thus determines whether the object belongs to the base or non-base category. After detecting the category to which each object belongs in the visual image, the category of each object is determined to be the base or non-base category, and a category label is assigned to each object. The category label can be a category number, category name, etc., which is not limited here.

[0111] When it is detected that the visual image includes a category label belonging to the pedestal, it is determined that the visual image includes the pedestal.

[0112] Thus, after acquiring the visual image, object detection is performed on the visual image to determine the category of one or more objects contained in the visual image. The corresponding category labels are then added to the objects. This allows the determination of whether the visual image contains a base based on the category labels, thereby filtering out non-base objects. This allows only point cloud data containing the base to be used in the subsequent analysis, filtering out point cloud data corresponding to non-base objects. This not only reduces the amount of data to be calculated, but also helps further improve matching accuracy by filtering out non-base point cloud data.

[0113] It should be noted that although the steps of the method of the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all steps must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0114] The following introduces an embodiment of the device of the present application, which can be used to execute the return control method in the above embodiment of the present application. Figure 9 The structure block diagram of the return control device provided by the embodiment of the present application is schematically shown. Figure 9 As shown, the return control device 900 includes:

[0115] A first acquisition module 901 is configured to acquire a visual image including a base after the mobile device enters a designated area;

[0116] A first determining module 902 is configured to determine base point cloud data of the base and a corresponding offset of the mobile device relative to the base based on the visual image;

[0117] The calculation module 903 is used to calculate the distance between the base point cloud data and the preset reference point cloud data;

[0118] A second acquisition module 904 is used to acquire an initial posture from the mobile device;

[0119] A second determining module 905 is configured to determine the positioning posture of the mobile device based on the offset, the distance, the initial posture, and a preset error equation;

[0120] The movement module 906 is used to control the movement of the self-moving device based on the positioning posture so that the self-moving device can dock with the base.

[0121] In some embodiments of the present application, based on the above technical solution, the first determination module 902 is also used to extract features from the visual image to obtain multiple contour lines of the base; and determine the point cloud data contained in the multiple contour lines as the base point cloud data of the base.

[0122] In some embodiments of the present application, based on the above technical solution, the device also includes a reference point cloud data acquisition module for acquiring a preset three-dimensional model of the base; performing feature extraction on the three-dimensional model of the base to obtain multiple reference contour lines of the base; and determining the point cloud data contained in the multiple reference contour lines as the reference point cloud data corresponding to the base.

[0123] In some embodiments of the present application, based on the above technical solution, the calculation module 903 is also used to determine the reference point cloud data that matches the base point cloud data; use the reference contour line where the reference point cloud data is located as the target contour line; and determine the distance from the base point cloud data to the target contour line as the distance from the base point cloud data to the reference point cloud data.

[0124] In some embodiments of the present application, based on the above technical solution, the second determination module 905 is also used to substitute the offset, distance, and initial posture into a preset error equation for calculation to obtain a total error value; when the total error value is greater than or equal to a preset error threshold, the initial posture is updated according to a preset update strategy, and the step of calculating the total error value is returned according to the updated initial posture; when the total error value is less than the preset error threshold, the latest initial posture is used as the positioning posture of the self-moving device.

[0125] In some embodiments of the present application, based on the above technical solution, the return control device also includes a positioning module, which is used to obtain the positioning signal and posture information of the mobile device; when the positioning signal is not in the specified area, the mobile device is controlled to enter the specified area based on the positioning signal and posture information.

[0126] In some embodiments of the present application, based on the above technical solution, the first acquisition module 901 is also used to detect and identify the visual image, determine the objects contained in the visual image and the category labels corresponding to each object; when it is detected that the visual image contains a category label belonging to the base, it is determined that the visual image contains the base.

[0127] The specific details of the return control device provided in each embodiment of the present application have been described in detail in the corresponding method embodiments and will not be repeated here.

[0128] Figure 10 The block diagram schematically shows a computer system structure of a mobile device for implementing an embodiment of the present application.

[0129] It should be noted that Figure 10 The computer system 1000 shown from the mobile device is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0130] like Figure 10As shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to the program stored in the read-only memory 1002 (ROM) or the program loaded from the storage part 1008 into the random access memory 1003 (RAM). Various programs and data required for system operation are also stored in the random access memory 1003. The CPU 1001, the read-only memory 1002, and the random access memory 1003 are connected to each other via a bus 1004. An input / output interface 1005 (i.e., an I / O interface) is also connected to the bus 1004.

[0131] The following components are connected to the input / output interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output section 1007 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 1008 including a hard disk; and a communication section 1009 including a network interface card such as a local area network card or a modem. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the input / output interface 1005 as needed. Removable media 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1010 as needed, so that computer programs read therefrom can be installed into the storage section 1008 as needed.

[0132] In particular, according to an embodiment of the present application, the processes described in the various method flow charts can be implemented as computer software programs. For example, an embodiment of the present application includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the methods shown in the flow charts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication portion 1009 and / or installed from a removable medium 1011. When the computer program is executed by the central processing unit 1001, the various functions defined in the system of the present application are performed.

[0133] It should be noted that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0134] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0135] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the application, the features and functions of two or more modules or units described above can be concretized in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0136] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0137] Figure 11 A schematic diagram of a self-moving device provided by an embodiment of the present application is shown schematically. Figure 11 As shown, the self-moving device 10 includes: a vehicle body 110, including a vehicle body 1101 and wheels 1102; and a control module 1103, which is used to execute the recharging method of the self-moving device provided in any embodiment of the present application. The specific details of the recharging method of the self-moving device have been described in detail in the corresponding method embodiment and will not be repeated here.

[0138] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of this application and include common knowledge or customary techniques in the art that are not disclosed herein.

[0139] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A return control method, characterized in that: The return control method includes: After the mobile device enters the designated area, a visual image including the base is obtained; Determining, based on the visual image, base point cloud data of the base and an offset of the self-moving device relative to the base; wherein the offset refers to a distance to the left or right of the self-moving device relative to the actual base in the same coordinate system; Calculating the distance between the base point cloud data and the preset reference point cloud data to determine the deviation between the actual base and the reference base; Obtaining an initial position of the mobile device; Determine the positioning posture of the self-moving device in the base coordinate system according to the offset, the distance, the initial posture and a preset error equation; wherein the positioning posture includes the positioning posture and positioning position of the self-moving device in the base coordinate system; Based on the positioning posture, the self-moving device is controlled to move so that the self-moving device is docked with the base.

2. The return control method according to claim 1, characterized in that: Determining base point cloud data of the base according to the visual image includes: Performing feature extraction on the visual image to obtain multiple contour lines of the base; The point cloud data included in the plurality of contour lines is determined as the base point cloud data of the base.

3. The return control method according to claim 1, characterized in that: Before calculating the distance between the base point cloud data and the preset reference point cloud data, the method further includes: Obtain a preset three-dimensional model of the base; Performing feature extraction on the three-dimensional model of the base to obtain a plurality of reference contour lines of the base; The point cloud data included in the plurality of reference contour lines is determined as the reference point cloud data corresponding to the base.

4. The return control method according to claim 3, characterized in that: The calculating the distance between the base point cloud data and the preset reference point cloud data includes: Determining reference point cloud data that matches the base point cloud data; The reference contour line where the benchmark point cloud data is located is used as the target contour line; The distance from the base point cloud data to the target contour line is determined as the distance from the base point cloud data to the reference point cloud data.

5. The return control method according to claim 1, characterized in that: Determining the positioning posture of the self-moving device according to the offset, the distance, the initial posture, and a preset error equation includes: Substituting the offset, the distance, and the initial posture into a preset error equation for calculation to obtain a total error value; When the total error value is greater than or equal to a preset error threshold, updating the initial posture according to a preset update strategy, and returning to the step of calculating the total error value according to the updated initial posture; When the total error value is less than the preset error threshold, the latest initial posture is used as the positioning posture of the self-moving device.

6. The return control method according to any one of claims 1 to 5, characterized in that: The method further comprises: Obtaining positioning signals and posture information of the mobile device; When the positioning signal is not in the designated area, the self-mobile device is controlled to enter the designated area based on the positioning signal and the posture information.

7. The return control method according to claim 1, characterized in that: After the mobile device enters the designated area, obtaining a visual image including the base includes: Detecting and identifying the visual image to determine the objects contained in the visual image and the category labels corresponding to the objects; When it is detected that the visual image includes a category label belonging to a base, it is determined that the visual image includes a base.

8. A return control device, characterized in that: The return control device includes: A first acquisition module is used to acquire a visual image including the base after the mobile device enters the designated area; A first determination module is configured to determine, based on the visual image, base point cloud data of the base and an offset of the self-moving device relative to the base; wherein the offset refers to a distance to the left or right of the self-moving device relative to the actual base in the same coordinate system; A calculation module, configured to calculate the distance between the base point cloud data and the preset reference point cloud data, so as to determine the deviation between the actual base and the reference base; A second acquisition module is used to obtain the initial position of the mobile device; a second determining module, configured to determine a positioning posture of the self-moving device in a base coordinate system based on the offset, the distance, the initial posture, and a preset error equation; wherein the positioning posture includes a positioning attitude and a positioning position of the self-moving device in the base coordinate system; The moving module is used to control the movement of the self-moving device based on the positioning posture so that the self-moving device is docked with the base.

9. A computer-readable medium, characterized in that The computer-readable medium stores a computer program, which, when executed by a processor, implements the return control method according to any one of claims 1 to 7.

10. A self-propelled device, characterized in that: include: A vehicle body, comprising a vehicle body and wheels; as well as A control module, configured to execute the return control method according to any one of claims 1 to 7.

Images