Methods for recharging self-moving devices, self-moving devices, recharging devices and media

Abstract

This application provides a recharging control method for a self-moving device, a self-moving device, a recharging device, and a computer-readable storage medium. The recharging control method for the self-moving device includes: the self-moving device acquiring depth image information of a baffle by sending a pulse signal to the baffle of the self-moving device via a lidar; acquiring intensity image information of the baffle via an image acquisition device; generating a reconstructed image based on the depth image information and the intensity image information; determining a first region based on the reconstructed image; determining a second region based on a standard image; acquiring the intersection-over-union ratio (IoU) of the first region and the second region; when the IoU of the first region and the second region is less than a preset IoU threshold; determining the offset of the self-moving device relative to a base; and controlling the movement of the self-moving device based on the offset until the self-moving device is aligned with the base.

Description

Technical Field

[0001] This application relates to the field of self-moving devices, and more specifically to a method for recharging a self-moving device, a self-moving device, a recharging device, and a computer-readable storage medium. Background Technology

[0002] With the development of technology, the intelligence level of self-moving devices such as robot vacuum cleaners, robot lawnmowers, and inspection robots is constantly improving. After completing their tasks or consuming a certain amount of power, self-moving devices will automatically return to their base and recharge.

[0003] However, in the relevant technical solutions, when determining whether the self-moving device has accurately returned to the preset charging position, it is easily affected by environmental factors, making it difficult to accurately determine the position of the self-moving device, and thus difficult to accurately determine whether the self-moving device is aligned with the base. Therefore, there is a problem of low recharging accuracy of the self-moving device. Summary of the Invention

[0004] This application provides a recharge control method for a self-moving device, a self-moving device, an apparatus, and a computer-readable storage medium to improve the recharge accuracy of the self-moving device.

[0005] Firstly, a method for controlling the recharging of a self-moving device is provided, the method comprising:

[0006] When the self-moving device moves toward the base, the depth image information of the baffle is obtained by the LiDAR. The baffle is set on the self-moving device and the reflectivity of the baffle is greater than the reflection threshold. The base is equipped with LiDAR and image acquisition device.

[0007] The intensity image information of the baffle is obtained through an image acquisition device;

[0008] Based on the depth image information and intensity image information, a reconstructed image is generated. The reconstructed image is used to indicate the first region of the baffle in the reconstructed image.

[0009] Obtain a preset standard image, which is used to indicate the second region of the baffle in the standard image;

[0010] Calculate the intersection-union ratio of the first region and the second region;

[0011] When the crossover ratio is less than the preset crossover threshold, the offset of the self-moving device relative to the base is determined based on the first region and the second region.

[0012] The self-moving device is moved based on the offset control until the intersection-to-intersection ratio is greater than or equal to the preset intersection-to-intersection threshold, at which point the self-moving device is determined to be aligned with the base.

[0013] The self-mobile device recharging control method provided in this application embodiment sets a baffle with a reflectivity greater than a threshold on the self-mobile device. A LiDAR sends pulse signals to the baffle to obtain depth image information of the baffle, and a camera acquires intensity image information of the baffle. A reconstructed image is determined based on the depth and intensity image information. The reconstructed image is compared with a standard image to determine whether the self-mobile device has returned to a preset position. Because the reflectivity of the baffle is higher than the threshold, the LiDAR can quickly distinguish between the pulse signals reflected from the baffle and the pulse signals reflected from the surrounding ambient light. Therefore, interference from ambient light can be minimized, improving the resolution of the LiDAR and resulting in more accurate depth image information. This allows for a more accurate determination of whether the self-mobile device has returned to the charging position, thereby improving the recharging accuracy of the self-mobile device.

[0014] In conjunction with the first aspect, in some implementations of the first aspect, the baffle can extend or retract within the self-moving device, and the lidar is connected to the base via a telescopic device; before acquiring depth image information of the baffle via the lidar, the method further includes: controlling the baffle to extend from inside the self-moving device to the outside when the self-moving device moves to a preset recharge range; and driving the lidar to extend from inside the base by controlling the telescopic device.

[0015] The self-moving device recharging control method provided in this application embodiment allows the self-moving device to control a baffle to extend out of the self-moving device when it moves to a preset recharging range. It also drives the lidar to extend out from inside the base by controlling the telescopic device to obtain depth image information of the baffle. This can prevent the baffle and lidar from extending out of the self-moving device too early, thereby reducing the risk of collisions between the baffle and objects in the environment during operation, and reducing damage to the lidar from collisions with other external objects.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, determining the offset of the self-moving device relative to the base based on the first region and the second region includes: obtaining the center position of the first region of the first region and the center position of the second region of the second region; calculating the deviation between the center position of the first region and the center position of the second region to obtain the offset.

[0017] The recharging control method for a self-moving device provided in this application embodiment determines the distance to be adjusted by calculating the offset of the center position in the first region and the second region. This avoids the self-moving device comparing all position information in the first region and the second region, thereby improving the efficiency of the self-moving device in determining the distance to be adjusted.

[0018] In conjunction with the first aspect, in some implementations of the first aspect, an environmental image of the base is acquired by an image acquisition device; if it is determined from the environmental image that there is an obstacle in the recharge range, a warning message is generated, which is used to indicate that there is an obstacle in the base.

[0019] The self-mobile device recharging control method provided in this application embodiment determines whether there are obstacles around the base. If there are obstacles, the device notifies staff to remove them through a warning message. This avoids the problem that the self-mobile device cannot return to the preset position due to the presence of obstacles on the recharging route, resulting in recharging failure.

[0020] In conjunction with the first aspect, in some implementations of the first aspect, after acquiring environmental images around the base via an image acquisition device, the method further includes: if a slippage area is determined based on the environmental images, then a recharging path for the self-moving device is generated based on the slippage area, the position of the base, and the position of the self-moving device; and the self-moving device is controlled to move towards the base along the recharging path. The recharging control method for the self-moving device provided in this application allows the self-moving device to determine whether a slippage area exists within the recharging range by acquiring environmental images. If such an area exists, the self-moving device generates a recharging path, thereby preventing it from entering the slippage area during its movement towards the base, ensuring safe recharging of the self-moving device, and improving recharging efficiency.

[0021] In conjunction with the first aspect, in some implementations of the first aspect, after calculating the crossover ratio of the first region and the second region, the method further includes: when the crossover ratio is greater than or equal to a preset crossover threshold, determining that the self-moving device is aligned with the base, and sending an indication signal through the indicator light of the self-moving device, wherein the indication signal is used to indicate a signal indicating that the self-moving device has successfully recharged.

[0022] Secondly, a recharge control device for a self-moving device is provided, the device comprising:

[0023] The depth image acquisition unit is used to acquire depth image information of the baffle through LiDAR when the self-moving device moves towards the base. The baffle is set on the self-moving device and the reflectivity of the baffle is greater than the reflection threshold. The base is equipped with LiDAR and image acquisition device.

[0024] An intensity image acquisition unit is used to acquire intensity image information of the baffle through an image acquisition device;

[0025] The reconstructed image acquisition unit is used to generate a reconstructed image based on depth image information and intensity image information. The reconstructed image is used to indicate the first region of the baffle in the reconstructed image.

[0026] A standard image acquisition unit is used to acquire a preset standard image, which is used to indicate the second region of the baffle in the standard image;

[0027] The intersection-union ratio calculation unit is used to calculate the intersection-union ratio of the first region and the second region.

[0028] The offset determination unit is used to determine the offset of the self-moving device relative to the base based on the first region and the second region when the crossover ratio is less than the preset crossover threshold.

[0029] The base alignment unit is used to control the movement of the self-moving device based on the offset until the crossover ratio is greater than or equal to the preset crossover threshold, and then determine that the self-moving device is aligned with the base.

[0030] Thirdly, a self-moving device is provided, including a processor, a processor, and a computer program stored in a memory and executable on the processor, wherein the processor executes the computer program to implement the method in any possible implementation of the first aspect described above.

[0031] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, wherein when the computer program is executed by a device, it implements the method in any possible implementation of the first aspect described above. Attached Figure Description

[0032] Figure 1 This is a schematic architecture diagram of the charging system provided in the embodiments of this application.

[0033] Figure 2 This is a state diagram of the baffle and lidar extending out of the device according to an embodiment of this application.

[0034] Figure 3 This is a schematic flowchart of a recharging method for a self-moving device provided in an embodiment of this application.

[0035] Figure 4 This is an exemplary block diagram of the recharge control device provided in the embodiments of this application.

[0036] Figure 5 This is a schematic structural diagram of the self-moving device provided in the embodiments of this application. Detailed Implementation

[0037] Currently, commercially available self-moving devices using contact charging require the device to be moved to a base and aligned with it before charging can begin. In related technical solutions, the self-moving device uses an infrared sensor or depth camera to determine its relative position to the base when returning to the designated location, and then uses this relative position to return to the designated location. However, infrared sensors cannot accurately calculate the relative position of the self-moving device to the base under strong outdoor light, and depth cameras cannot accurately calculate the relative position of the self-moving device to the base in low-light environments. Therefore, in these technical solutions, both excessively strong and insufficient light intensity in the environment can lead to positioning errors, making it difficult for the self-moving device to accurately return to the designated location.

[0038] To avoid the problems existing in the above-mentioned related technical solutions, this application proposes a recharging method for a self-moving device. This method can minimize the interference of ambient light on the lidar, so that the generated depth image information has high accuracy and can more accurately determine whether the self-moving device has been returned to the charging position, thereby improving the recharging accuracy of the self-moving device.

[0039] The self-recharging method for a self-moving device provided in this application can be applied to a charging system, which can be any system including a self-moving device and a docking station. It is understood that the charging system in this embodiment may also include more self-moving devices and docking stations, and may also include other devices; this application does not impose any limitations on this.

[0040] Figure 1 and Figure 2 A schematic architecture diagram of a charging system applicable to embodiments of this application is shown. (Reference) Figure 1 and Figure 2 The charging system includes a self-moving device 101 and a base 102. The self-moving device 101 is provided with a baffle 1011, and the base 102 is provided with a telescopic device 1021 and a data acquisition device 1022. The data acquisition device 1022 includes a lidar and an image acquisition device. The lidar and the image acquisition device are rigidly connected and located at the same height and in the same direction.

[0041] The self-moving device 101 can be a device that includes self-movement assistance functionality. This self-movement assistance functionality can be implemented via an in-vehicle terminal, and the corresponding self-moving device 101 can be a vehicle equipped with that in-vehicle terminal. The self-moving device 101 can also be a semi-autonomous device or a fully autonomous device. For example, the self-moving device 101 can be an inspection robot, a sweeping robot, a lawnmower robot, etc. In this embodiment, the self-moving device 101 is a robot with self-movement functionality, and it is also equipped with a baffle 1011.

[0042] The baffle 1011 can be any object capable of reflecting pulse signals. In this embodiment, the baffle 1011 can be mounted on the self-moving device 101 for reflecting the pulse signals sent by the lidar.

[0043] For example, the baffle 1011 can extend or retract within the self-moving device 101. Figure 1 In the middle, the baffle 1011 retracts into the self-moving device 101, in Figure 2 In this configuration, the baffle 1011 extends outward from the self-moving device 101. It can be understood that when the self-moving device 101 needs the baffle 1011 to reflect a pulse signal, it controls the baffle 1011 to extend outward from the self-moving device 101, and when the baffle 1011 does not need to reflect a pulse signal, it controls the baffle 1011 to retract into the self-moving device 101.

[0044] In this embodiment, the baffle 1011 can be disposed at any location of the self-moving device 101.

[0045] In some embodiments, a baffle 1011 is disposed at the rear of the self-moving device 101.

[0046] The tail of the self-mobile device 101 can be understood as the end of the self-mobile device 101. For example, as... Figure 1 As shown, the baffle 1011 in the figure is disposed at the tail of the self-moving device 101.

[0047] The recharging method for the self-moving device 101 provided in this application embodiment has a baffle 1011 disposed at the tail of the self-moving device 101, so that the baffle 1011 can better reflect pulse signals, making it easier for the lidar to distinguish the pulse signals reflected by the baffle 1011 from the pulse signals reflected by other objects in the environment, thereby improving the accuracy of the lidar in generating depth image information of the baffle 1011.

[0048] In other embodiments, the baffle 1011 may also be disposed in other locations of the self-moving device 101, such as the right side, left side, and front end of the self-moving device 101.

[0049] The base 102 can be any device that can charge the self-moving device 101. In this embodiment, the base 102 is used to charge the self-moving device 101. In addition, the base 102 can also be equipped with a telescopic device 1021, a lidar, and an image acquisition device.

[0050] The telescopic device 1021 is used to control the lidar and image acquisition device to extend out of the base or retract into the base.

[0051] LiDAR is used to transmit pulse signals to a target, and then compare the received signal reflected back from the target with the transmitted signal to obtain information related to the target.

[0052] In this embodiment of the application, exemplarily, the lidar can extend or retract within the base 102 via the telescopic device 1021 to send pulse signals to the baffle 1011, and then compare the received signal reflected back from the baffle 1011 with the sent pulse signal to obtain the position information of the baffle 1011. Figure 1 In the middle, the lidar is retracted within the base 102, while Figure 2 In this configuration, the lidar extends outward from the base 102. It can be understood that when the lidar needs to send and receive pulse signals, the lidar extends outward from the base 102 via the telescopic device 1021; conversely, when the lidar does not need to send or receive pulse signals, the lidar extends outward from the base 102 via the telescopic device 1021.

[0053] The image acquisition device is used to acquire intensity image information of the baffle 1011.

[0054] In this embodiment, the image acquisition device can extend out of or retract into the base 102 via the telescopic device 1021. Figure 1 In the middle, the image acquisition device retracts into the base 102, Figure 2 In the image acquisition device, the image acquisition device extends out of the base 102. It can be understood that when intensity image information needs to be acquired, the image acquisition device extends out of the base 102 through the telescopic device 1021, and when intensity image information does not need to be acquired, the image acquisition device retracts into the base 102 through the telescopic device 1021.

[0055] In this embodiment, the baffle 1011 can be mounted on the self-moving device 101, and the lidar and image acquisition device are mounted on the image acquisition device, which can extend out of the base 102 or retract into the base 102. The lidar sends pulse signals to the baffle 1011 to obtain depth image information of the baffle 1011, and the image acquisition device obtains intensity image information of the baffle 1011. The self-moving device 101 determines whether to return to a preset position on the base 102 that can charge the self-moving device 101 based on the depth image information and the intensity image information.

[0056] The following, combined with Figure 3 This application provides a detailed description of a recharging method for a self-operated mobile device according to an embodiment of the present application.

[0057] Figure 3This is a schematic flowchart of a recharging method 300 for a self-moving device provided in an embodiment of this application. The execution subject of method 300 can be a self-moving device, a base, or a processor or chip in the self-moving device or the base. For ease of description, a self-moving device is used as an example to describe method 300 in detail.

[0058] In step S310, when the self-moving device moves toward the base, the depth image information of the baffle is acquired by the LiDAR. The baffle is set on the self-moving device and the reflectivity of the baffle is greater than the reflection threshold. The base is equipped with a LiDAR and an image acquisition device.

[0059] For example, such as Figure 1 As shown, the baffle is located at the rear of the self-moving device.

[0060] It should be understood that the baffle can also be placed in other locations of the self-moving device, such as the right side, left side, and front end of the self-moving device, and this application does not limit it in any way.

[0061] In this application, the reflectivity of the baffle is greater than a threshold value. The threshold value can be any value greater than the reflectivity of all objects in the environment in which the self-moving device is located. For example, the reflectivity of the baffle is 200 dBZ. If the reflectivity of common objects in the environment in which the self-moving device is located, such as grass, ground, and walls, is less than 100 dBZ, then the threshold value can be 150 dBZ.

[0062] In practice, the lidar sends a pulse signal to the baffle, the baffle reflects the pulse signal sent by the lidar back to the lidar, and after receiving the pulse signal reflected by the baffle, the lidar sends the pulse signal to the self-moving device. The self-moving device generates depth image information of the baffle based on the received pulse signal.

[0063] For example, a lidar can acquire the pulse signal returned by the baffle using multiple installed single-photon avalanche diode (SPAD) detectors and send the returned pulse signal to a self-moving device. It should be understood that lidar can also acquire pulse signals using other types of detectors.

[0064] For example, the self-moving device obtains the number of pulse signals and the return time of the pulse signals based on the received pulse signals, and generates a histogram to represent the pulse signals based on the number of pulse signals and the return time of the pulse signals. Here, the number of pulse signals represents the illumination intensity of the pulse signals at the corresponding spatial location, and the shape of the histogram represents the return time of the pulse signals. Then, a loss function is used to focus on the relationship between the number of pulse signals and the return time to improve the resolution of the pulse signals at the spatial location, thereby generating depth image information to represent the baffle information.

[0065] Depth image information includes image information such as the position and geometry of the baffle.

[0066] In step S320, the self-moving device acquires the intensity image information of the baffle through the image acquisition device.

[0067] Intensity image information can be understood as image information including a two-dimensional image of the baffle, obtained through an image acquisition device.

[0068] In practice, for example, the self-mobile device acquires the intensity image of the baffle through an image acquisition device. The image acquisition device performs image preprocessing such as grayscale conversion and illumination compensation on the acquired intensity image of the baffle and sends the processed intensity image of the baffle to the self-mobile device. The self-mobile device determines the position and boundary information of the baffle in the intensity image based on the acquired intensity image of the baffle, and determines the intensity image information of the baffle.

[0069] For example, the image acquisition device may be a CMOS (Complementary Metal-Oxide-Semiconductor) camera, a CCD (Charge Coupled Device) camera, or other types of cameras, and this application does not limit it in any way.

[0070] In step S330, the self-moving device generates a reconstructed image based on the depth image information and the intensity image information. The reconstructed image is used to indicate the first region of the baffle in the reconstructed image.

[0071] Reconstructed images can be understood as high-resolution images, including baffle information, generated after the mobile device registers and aligns depth and intensity image information.

[0072] The first region can be understood as the area where the baffle is located in the reconstructed image.

[0073] In the implementation, for example, the self-mobile device introduces a multi-scale feature extraction and attention mechanism module based on the convolutional neural network algorithm. The feature extraction module extracts multi-scale features from the depth image information and intensity image information, and uses the features in the intensity image information as guiding features to generate a reconstructed image. The self-mobile device then uses OpenCV to determine the region where the baffle is located in the reconstructed image.

[0074] In step S340, the self-moving device acquires a preset standard image, which is used to indicate the second area of ​​the baffle in the standard image.

[0075] A standard image can be understood as the image when the mobile device is aligned with the base.

[0076] The second region can be understood as the area where the baffle is located in the standard image.

[0077] In this embodiment of the application, a standard image can be acquired using an image acquisition device.

[0078] In step S350, the self-moving device calculates the intersection-union ratio of the first region and the second region.

[0079] The intersection-union ratio can be used to represent the degree of overlap between the region where the baffle is located in the reconstructed image and the region where the baffle is located in the standard image.

[0080] For example, when calculating the intersection-union ratio of the first region and the second region, the self-moving device can convert the points in the world coordinate system in the reconstructed image to the image coordinate system according to the transformation relationship between the world coordinate system and the standard camera coordinate system used to acquire the standard image. Then, it can combine the position information of the baffle detected by the deep learning algorithm to calculate the intersection-union ratio of the first region in the reconstructed image and the second region in the standard image.

[0081] In step S360, when the crossover ratio is less than the preset crossover threshold, the self-moving device determines the offset of the self-moving device relative to the base based on the first region and the second region.

[0082] The preset intersection threshold is a value used to determine whether the self-moving device should be moved back to a position where the base can charge it.

[0083] Offset can be understood as the distance that the self-moving device needs to move in the X, Y, and Z axes of the world coordinate system when the self-moving device is adjusted to the standard posture in the first region, with the posture of the self-moving device in the second region as the standard posture.

[0084] For example, the self-moving device compares the obtained intersection-union ratio with a preset intersection-union threshold. When it is determined that the intersection-union ratio is less than the preset intersection-union threshold, it calculates the difference between the first region and the second region on the X and Y axes. Based on the transformation relationship between the world coordinate system and the standard camera coordinate system used to acquire the standard image, the calculated difference is converted into the distance that the self-moving device needs to move in the X, Y, and Z axes in the world coordinate system.

[0085] In some embodiments, when the crossover ratio is greater than or equal to a preset crossover threshold, it is determined that the self-moving device is aligned with the base, and an indication signal is sent through the indicator light of the self-moving device, wherein the indication signal is used to indicate that the self-moving device has successfully returned to charging.

[0086] In practice, when the crossover ratio is greater than or equal to the preset crossover threshold, it indicates that the self-moving device has been returned to a position where the base can charge it. An indicator signal can be sent through the indicator light to indicate that the self-moving device has successfully docked with the base and can start charging.

[0087] In step S370, the self-moving device controls the movement of the self-moving device based on the offset until the intersection-to-union ratio is greater than or equal to the preset intersection-to-union threshold, and then determines that the self-moving device is aligned with the base.

[0088] For example, after obtaining the offset, the self-moving device adjusts its position according to the offset. After the position adjustment is completed, it obtains the depth image information and intensity image information of the current position, and then generates a reconstructed image based on the intensity image information and depth image information. The first region is determined based on the reconstructed image, and then the intersection-union ratio of the first region and the second region is determined. The relationship between the intersection-union ratio and the preset intersection-union threshold is judged. If the intersection-union ratio is less than the preset intersection-union threshold, the self-moving device recalculates its offset relative to the base and controls itself to move according to the offset. The above process is repeated until the intersection-union ratio of the first region and the second region is greater than or equal to the preset intersection-union threshold, and the self-moving device is determined to be aligned with the base.

[0089] The self-moving device recharging control method provided in this application embodiment sets a baffle with a reflectivity greater than a threshold on the self-moving device. A laser radar sends pulse signals to the baffle to obtain depth image information of the baffle. An image acquisition device obtains intensity image information of the baffle. A reconstructed image is determined based on the depth and intensity image information. The reconstructed image is compared with a standard image to determine whether the self-moving device has returned to a preset position. Because the reflectivity of the baffle is higher than the threshold, the laser radar can quickly distinguish between the pulse signals reflected from the baffle and the pulse signals reflected from the surrounding ambient light. Therefore, interference from ambient light can be minimized, improving the resolution of the laser radar and resulting in higher accuracy of the generated depth image information. This allows for a more accurate determination of whether the self-moving device has returned to the charging position, thereby improving the recharging accuracy and success rate of the self-moving device.

[0090] In some embodiments, the baffle can extend out of or retract into the self-moving device, and the lidar is connected to the base via a telescopic device. Before acquiring depth image information of the baffle via the lidar, method 300 further includes: when the self-moving device moves to a preset recharge range, the self-moving device controls the baffle to extend outward from the inside of the self-moving device; and drives the lidar to extend outward from the inside of the base by controlling the telescopic device.

[0091] A telescopic device refers to any device that can control the lidar to extend out of its base or retract into its base.

[0092] Understandably, depending on actual needs, the self-moving device can be controlled to extend out of the baffle or retract into the self-moving device.

[0093] The lidar is connected to the base via a telescopic device. The mobile device can send telescopic commands to the base to control the lidar to extend or retract from the base.

[0094] The recharge range can be understood as the area near a preset location where the charging dock can charge the mobile device. For example, the recharge range could be an area within 1 meter of the preset location where the charging dock can charge the mobile device.

[0095] The self-moving device recharging control method provided in this application embodiment allows the self-moving device to control a baffle to extend out of the self-moving device when it moves to a preset recharging range. It also drives a laser radar to extend from inside the base via a telescopic device to obtain depth image information of the baffle. This avoids the baffle and laser radar extending out of the self-moving device prematurely, reducing collisions between the baffle and objects in the environment during operation, and minimizing damage to the laser radar from collisions with other objects. Furthermore, it improves the aesthetics of the self-moving device and base without affecting their normal operation.

[0096] In other embodiments, the baffle can extend outwards from the self-moving device, and the lidar can extend outwards from the base. For example, the baffle can be fixed to the outside of the self-moving device so that it always extends outwards from the device, and the lidar can be fixed to the outside of the base so that it always extends outwards from the base. The self-moving device can reflect the pulse signal sent by the lidar through the baffle at any time.

[0097] In some embodiments, when the self-moving device determines the offset between the self-moving device and the base based on the first region and the second region, the offset can be obtained by acquiring the center position of the first region of the first region and the center position of the second region of the second region; and calculating the deviation between the center position of the first region and the center position of the second region.

[0098] The center point of the first region can be the center point of the baffle in the reconstructed image.

[0099] The center point of the second region can be the center point of the baffle in the standard image.

[0100] In practice, for example, when calculating the offset, the self-moving device can first determine the center position of the first region of the first region and the center position of the second region of the second region, then calculate the difference between the first center point position and the second center point position on the X and Y axes, and then convert the calculated difference into the offset to be adjusted by the self-moving device according to the transformation relationship between the world coordinate system and the standard camera coordinate system used to acquire the standard image, so that the self-moving device can move to a position that can be aligned with the base according to the offset.

[0101] The recharging control method for a self-moving device provided in this application embodiment determines the distance to be adjusted by the self-moving device by calculating the offset of the center position in the first region and the second region. This avoids the self-moving device comparing all position information in the first region and the second region, thereby improving the speed at which the self-moving device determines the distance to be adjusted.

[0102] In other embodiments, after the self-moving device converts the world coordinate system in the reconstructed image into the image coordinate system, it calculates the distances between the first region and the second region and the boundaries of other objects in the image, determines the difference between the calculated distances in the reconstructed image and the distances in the standard image, and adjusts the position of the self-moving device according to the calculated difference so that the self-moving device returns to the preset position.

[0103] In some embodiments, before the self-moving device acquires depth image information of the baffle via LiDAR, the self-moving device acquires environmental images around the base via an image acquisition device; if it is determined based on the environmental images that there are obstacles in the recharge range, a warning message is generated, which is used to indicate that there are obstacles around the base.

[0104] In practice, for example, after the image acquisition device extends outside the base via a telescopic device, the self-moving device can send image acquisition information to the base so that the base can acquire environmental images around the base through the image acquisition device. The base sends the acquired environmental images to the self-moving device, which performs target comparison based on the received environmental images to determine whether there are obstacles around the base. If there are obstacles around the base, the self-moving device issues a warning message to prompt the staff to remove the obstacles.

[0105] For example, the self-mobile device can send warning information to the staff's mobile device to prompt the staff to remove the obstacle.

[0106] The self-mobile device recharging control method provided in this application embodiment determines whether there are obstacles around the base. If there are obstacles, the device notifies staff to remove them through a warning message. This avoids the problem that the self-mobile device cannot return to the preset position due to the presence of obstacles on the recharging route, thus preventing recharging failure.

[0107] In other embodiments, the self-moving device can also acquire environmental images around the base by installing an image acquisition device to determine whether there are obstacles around the base. This application does not impose any limitations.

[0108] In some embodiments, after acquiring environmental images around the base via an image acquisition device, the self-moving device detects whether there is a slippage area within the recharge range. If a slippage area is determined to exist based on the environmental images, a recharge path for the self-moving device is generated based on the slippage area, the position of the base, and the position of the self-moving device; the self-moving device is then controlled to move towards the base along the recharge path. Specifically, historical position information of the self-moving device when slipping within the recharge range is recorded. This historical position information can be determined based on historical environmental images and is saved. When the self-moving device moves within the recharge range, it can be matched with the historical environmental images corresponding to the slippage area based on the environmental images. If no match is found, it indicates that there is no slippage area within the recharge range. Alternatively, if no historical position information exists within the range from the current position of the self-moving device to the base, it indicates that there is no slippage area within the recharge range. Alternatively, slippage area detection can be performed based on environmental images; for example, the slippage area could be an uneven terrain feature. If no slippage area is detected, it indicates that there is no slippage area within the recharge range. For example, as described above, after the self-moving device acquires an environmental image around the base via an image acquisition device, it determines whether there is a slippage area within the recharge range based on the acquired environmental image. If a slippage area is determined to exist within the recharge range, the self-moving device generates a recharge path based on the positional relationship between the slippage area, the base, and the self-moving device. The recharge path represents the path the self-moving device needs to move when aligned with the base. By controlling the self-moving device to move along the recharge path, the self-moving device avoids the slippage area.

[0109] The self-mobile device recharging control method provided in this application embodiment determines whether there is a slippage area in the recharging range by acquiring an environmental image. If there is, the self-mobile device generates a recharging path so that the self-mobile device can avoid entering the slippage area when moving towards the base, thereby ensuring the recharging safety of the self-mobile device and improving the recharging efficiency.

[0110] In other embodiments, similar to the examples above, the self-moving device is equipped with an image acquisition device to determine whether there is a slippage area in the recharge range.

[0111] The above, combined with Figures 1 to 3 This application provides a detailed description of a recharging method for a self-moving device, as illustrated in the embodiments below. Figures 4 to 5 This application provides a detailed description of the self-moving device and recharge control device provided according to embodiments thereof.

[0112] Figure 4 This is an exemplary block diagram of the recharge control device provided in this application embodiment. Exemplarily, the recharge control device 400 can be configured within a self-moving device or within a base. The recharge control device 400 includes:

[0113] The depth image acquisition unit 410 is used to acquire depth image information of the baffle through a laser radar when the self-moving device moves toward the base. The baffle is set on the self-moving device and the reflectivity of the baffle is greater than the reflection threshold. The base is equipped with a laser radar and an image acquisition device.

[0114] The intensity image acquisition unit 420 is used to acquire intensity image information of the baffle through the image acquisition device;

[0115] The reconstructed image acquisition unit 430 is used to generate a reconstructed image based on depth image information and intensity image information. The reconstructed image is used to indicate the first region of the baffle in the reconstructed image.

[0116] The standard image acquisition unit 440 is used to acquire a preset standard image, which is used to indicate the second region of the baffle in the standard image;

[0117] The intersection-union ratio calculation unit 450 is used to calculate the intersection-union ratio of the first region and the second region.

[0118] The offset determination unit 460 is used to determine the offset of the self-moving device relative to the base based on the first region and the second region when the crossover ratio is less than the preset crossover threshold.

[0119] The base alignment unit 470 is used to control the movement of the self-moving device based on the offset until the crossover ratio is greater than or equal to a preset crossover threshold, and then determine that the self-moving device is aligned with the base.

[0120] In some embodiments, the baffle can extend or retract within the self-moving device, and the lidar is connected to the base via a telescopic device; the recharge control device 400 further includes:

[0121] A control unit is configured to control a baffle to unfold from inside the self-moving device to the outside when the self-moving device moves into a preset recharge range; and,

[0122] The drive unit is used to drive the lidar to extend from inside the base by controlling the telescopic device.

[0123] In some implementations, the offset determination unit 460 includes:

[0124] Get sub-units, used to obtain the center position of the first region of the first region and the center position of the second region of the second region;

[0125] The calculation sub-unit is used to calculate the deviation between the center position of the first region and the center position of the second region to obtain the offset.

[0126] In some embodiments, the recharge control device 400 further includes:

[0127] An environmental image acquisition unit is used to acquire environmental images around the base through an image acquisition device.

[0128] The early warning information generation unit is used to generate early warning information if it is determined from the environmental image that there are obstacles in the recharge range. The early warning information is used to indicate that there are obstacles around the base.

[0129] In some embodiments, the recharge control device 400 further includes:

[0130] The recharge path generation unit is used to generate a recharge path for the self-moving device based on the slippage area, the position of the base, and the position of the self-moving device if a slippage area is determined to exist in the recharge range based on the environmental image.

[0131] The recharge path control unit is used to control the movement of the self-moving device along the recharge path towards the base.

[0132] In some embodiments, the recharge control device 400 further includes:

[0133] The signal sending unit is used to determine that the self-moving device is aligned with the base when the crossover ratio is greater than or equal to the preset crossover threshold, and to send an indication signal through the indicator light of the self-moving device, wherein the indication signal is used to indicate that the self-moving device has successfully returned to charging.

[0134] It should be understood that the above-mentioned recharge control device 400 is used to execute the corresponding steps and / or processes in the above method embodiments. For a detailed description, please refer to the relevant description above, which will not be repeated here.

[0135] In the embodiments of this application, Figure 4 The recharge control device in the chip can also be a chip or a chip system, such as a system on chip (SoC).

[0136] Figure 5 This is a schematic structural diagram of the self-moving device provided in an embodiment of this application. The self-moving device 500 is used to execute the corresponding steps and / or processes in the above method embodiments.

[0137] The self-moving device 500 includes a processor 510, a memory 520, and a transceiver 530. The processor 510, memory 520, and transceiver 530 communicate with each other via internal connections. The processor 510 can implement the functions of all processing units in various possible implementations of the recharge control device 400. The memory 520 is used to store instructions, and the processor 510 is used to execute the instructions stored in the memory 520. In other words, the processor 510 can call these stored instructions to implement the functions of all processing units in the recharge control device 400.

[0138] Optionally, the memory 520 may include read-only memory and random access memory, and provide instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information. The processor 510 may be used to execute instructions stored in the memory, and when the processor 510 executes instructions stored in the memory, the processor 510 is used to perform the various steps and / or processes of the method embodiments corresponding to the self-moving device described above.

[0139] It should be understood that, in the embodiments of this application, the processor of the aforementioned self-moving device can be a central processing unit (CPU), or it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0140] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware processor, or as a combination of hardware and software units within the processor. The software units can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor executes the instructions in the memory, combining them with its hardware to complete the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0141] This application provides a computer program product that, when run on a self-moving device, causes the self-moving device to execute the technical solutions described in the above embodiments. Its implementation principle and technical effects are similar to those of the related embodiments described above, and will not be repeated here.

[0142] This application provides a readable storage medium containing instructions that, when executed on a self-moving device, cause the self-moving device to perform the technical solution described in the above embodiments. The implementation principle and technical effects are similar and will not be repeated here.

[0143] This application provides a chip for executing instructions. When the chip is running, it performs the technical solutions described in the above embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0144] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0145] It should be understood that the term "embodiment" used throughout the specification means that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, various embodiments throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0146] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0147] It should also be understood that in this application, “when…”, “if” and “if” all refer to the UE or base station taking corresponding actions under certain objective circumstances, and are not time-limited, nor do they require the UE or base station to perform a judgment action, nor do they imply any other limitations.

[0148] Those skilled in the art will understand that the various numerical designations such as "first," "second," etc., involved in this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application, nor do they indicate the order of sequence.

[0149] In this application, the use of singular pronouns to denote "one or more" rather than "one and only one," unless otherwise specified. In this application, unless otherwise specified, "at least one" is intended to mean "one or more," and "more than" is intended to mean "two or more."

[0150] In this document, the terms "at least one of..." or "at least one of..." refer to all or any combination of the listed items. For example, "at least one of A, B, and C" can mean: A exists alone, B exists alone, C exists alone, A and B exist simultaneously, B and C exist simultaneously, and A, B, and C exist simultaneously. A can be singular or plural, B can be singular or plural, and C can be singular or plural.

[0151] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0152] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0153] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0154] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0155] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0156] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0157] The same or similar parts between the various embodiments in this application can be referred to mutually. In the various embodiments of this application, and in the various implementation methods / methods / implementations within each embodiment, unless otherwise specified or logically conflicting, the terminology and / or descriptions between different embodiments and between the various implementation methods / methods / implementations within each embodiment are consistent and can be mutually referenced. The technical features in different embodiments and the various implementation methods / methods / implementations within each embodiment can be combined according to their inherent logical relationships to form new embodiments, implementation methods, methods, or implementation approaches. The above embodiments of this application do not constitute a limitation on the scope of protection of this application.

[0158] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims. In conclusion, the above are merely preferred embodiments of the technical solution of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A recharging control method for a self-operated mobile device, characterized in that, The method includes: When the self-moving device moves toward the base, depth image information of the baffle is acquired by the LiDAR. The baffle is set on the self-moving device and the reflectivity of the baffle is greater than the reflection threshold. The base is equipped with the LiDAR and the image acquisition device. The reflection threshold is a value that is greater than the reflectivity of all objects in the environment in which the self-moving device is located. The intensity image information of the baffle is obtained through the image acquisition device; Based on the depth image information and the intensity image information, a reconstructed image is generated, wherein the reconstructed image is used to indicate a first region of the baffle in the reconstructed image; A preset standard image is obtained, which is an image when the self-moving device is aligned with the base, and the standard image is used to indicate the second region of the baffle in the standard image; Calculate the intersection-union ratio of the first region and the second region; When the crossover ratio is less than a preset crossover threshold, the offset of the self-moving device relative to the base is determined based on the first region and the second region; The self-moving device is controlled to move based on the offset until the intersection-to-union ratio is greater than or equal to the preset intersection-to-union threshold, at which point the self-moving device is determined to be aligned with the base.

2. The method according to claim 1, characterized in that, The baffle can extend out of or retract into the self-moving device from the inside or outside of the self-moving device, and the lidar is connected to the base via a telescopic device; Before acquiring the depth image information of the baffle using the lidar, the method further includes: When the self-moving device moves to the preset recharge range, the baffle is controlled to unfold from the inside of the self-moving device to the outside; as well as The lidar is extended from inside the base by controlling the telescopic device.

3. The method according to claim 1, characterized in that, Determining the offset of the self-moving device relative to the base based on the first region and the second region includes: Obtain the center position of the first region of the first region and the center position of the second region of the second region; The offset is obtained by calculating the deviation between the center position of the first region and the center position of the second region.

4. The method according to claim 2, characterized in that, The method further includes: The image acquisition device acquires environmental images around the base. If an obstacle is determined to exist within the recharge range based on the environmental image, a warning message is generated to indicate the presence of an obstacle around the base.

5. The method according to claim 4, characterized in that, After acquiring an image of the environment surrounding the base using the image acquisition device, the method further includes: If a slippage area is determined to exist within the recharge range based on the environmental image, a recharge path for the self-moving device is generated based on the slippage area, the position of the base, and the position of the self-moving device. Control the self-moving device to move towards the base along the recharge path.

6. The method according to any one of claims 1-5, characterized in that, After calculating the intersection-union ratio of the first region and the second region, the method further includes: When the crossover ratio is greater than or equal to the preset crossover threshold, it is determined that the self-moving device is aligned with the base, and an indication signal is sent through the indicator light of the self-moving device, wherein the indication signal is used to indicate that the self-moving device has successfully returned to charging.

7. A recharge control device for a self-moving device, characterized in that, The recharge control device includes: A depth image acquisition unit is used to acquire depth image information of a baffle via a lidar when the self-moving device moves toward the base. The baffle is disposed on the self-moving device and has a reflectivity greater than a reflection threshold. The lidar and the image acquisition device are disposed on the base. The reflection threshold represents a value greater than the reflectivity of all objects in the environment in which the self-moving device is located. An intensity image acquisition unit is used to acquire intensity image information of the baffle through the image acquisition device; The reconstructed image acquisition unit is used to generate a reconstructed image based on the depth image information and the intensity image information, wherein the reconstructed image is used to indicate a first region of the baffle in the reconstructed image; A standard image acquisition unit is used to acquire a preset standard image, wherein the standard image is an image when the self-moving device is aligned with the base, and the standard image is used to indicate the second region of the baffle in the standard image; An intersection-to-union ratio calculation unit is used to calculate the intersection-to-union ratio between the first region and the second region; An offset determination unit is used to determine the offset of the self-moving device relative to the base based on the first region and the second region when the crossover ratio is less than a preset crossover threshold. The base alignment unit is used to control the movement of the self-moving device based on the offset until the intersection-to-intersection ratio is greater than or equal to the preset intersection-to-intersection threshold, and then determine that the self-moving device is aligned with the base.

8. The recharge control device for a self-moving device according to claim 7, characterized in that, The baffle can extend or retract from the self-moving device and be housed within it; the lidar is connected to the base via a telescopic device; the recharge control device further includes: The control unit is configured to control the baffle to unfold from inside the self-moving device to the outside when the self-moving device moves into a preset recharge range; and A drive unit is used to drive the lidar to extend from inside the base by controlling the telescopic device.

9. A self-moving device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images