Method, apparatus and storage medium for recharging control of self-moving devices

Abstract

This application relates to the field of automatic control technology and provides a method, apparatus, and storage medium for controlling the recharge of a self-moving device. The method includes: acquiring an environmental image of the self-moving device during the recharge process; performing target detection on the environmental image to obtain a first detection frame including the charging dock; extracting a second detection frame corresponding to the charging dock features from the first detection frame; wherein the charging dock features are identifying features on the charging dock, and the image area of ​​the second detection frame is smaller than that of the first detection frame; determining the position information of the second detection frame; and controlling the self-moving device to move towards the charging dock based on the position information of the second detection frame. This application solves the problem of inaccurate control of the self-moving device returning to the charging dock. By narrowing the target detection range, the recharge control of the self-moving device can be made more precise.

Description

Technical Field

[0001] This application belongs to the field of self-moving device technology, and particularly relates to a recharging control method, device and storage medium for self-moving devices. Background Technology

[0002] With the development of charging technology, self-moving devices that can autonomously return to their charging docks are increasingly used in smart homes, smart factories, and other intelligent industries. When a self-moving device's battery is low or it has completed its task, it will automatically search for and return to its charging dock to recharge, ensuring it has enough power for its next task.

[0003] In related technologies, deep learning is generally used to identify the position of the charging dock and thus realize the recharging and positioning of the mobile device. However, as the mobile device moves towards the charging dock, the size of the detection field changes, which causes the positioning of the charging dock to deviate, resulting in the failure of docking between the mobile device and the charging dock. It can be seen that the inaccurate positioning of the charging dock leads to the problem of poor accuracy of recharging control. Summary of the Invention

[0004] This application provides a method, apparatus, and storage medium for controlling the return of a self-moving device to its charging dock, which can solve the problem of poor accuracy in controlling the return of a self-moving device to its charging dock in related technologies.

[0005] In a first aspect, embodiments of this application provide a recharging control method for a self-moving device, comprising the following steps:

[0006] Acquire environmental images during the recharging process of a mobile device;

[0007] Target detection is performed on the environmental image to obtain a first detection box including the charging dock;

[0008] Extract the second detection box corresponding to the charging seat feature from the first detection box; wherein, the charging seat feature is the identifying feature on the charging seat, and the image area of ​​the second detection box is smaller than the image area of ​​the first detection box;

[0009] Determine the position information of the second detection frame;

[0010] The self-moving device is controlled to move towards the charging dock based on the position information of the second detection frame.

[0011] Secondly, embodiments of this application provide a recharging control device for a self-moving device, the device comprising:

[0012] The image acquisition module is used to acquire environmental images during the recharging process of the mobile device;

[0013] The first detection module is used to perform target detection on the environmental image to obtain a first detection box including the charging dock;

[0014] The second detection module is used to extract the second detection box corresponding to the charging seat feature from the first detection box; wherein, the charging seat feature is the identification feature on the charging seat, and the image area of ​​the second detection box is smaller than the image area of ​​the first detection box;

[0015] The first positioning module is used to determine the position information of the second detection frame;

[0016] The movement control module controls the self-moving device to move towards the charging dock based on the position information of the second detection frame.

[0017] Thirdly, embodiments of this application provide a self-moving device, which includes an image acquisition unit, a memory, a processor, and a computer program stored in the memory and executable on the processor. The image acquisition unit is used to acquire images, and the processor executes the computer program to implement the recharging control method of the self-moving device described in any one of the first aspects.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the recharging control method for the self-moving device described in any one of the first aspects.

[0019] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the recharging control method for the self-moving device described in any of the first aspects.

[0020] The advantages of the embodiments in this application compared with related technologies are:

[0021] When a self-operated device returns to its charging dock after its battery is low or it has completed a task, the system identifies the environmental image during the recharging process. Target detection is performed within this image to obtain a first detection box. This first detection box provides the approximate location of the charging dock within the environmental image's field of view. To further improve the accuracy of the recharging location and avoid instability in charging dock positioning as the target detection distance decreases, a second detection box corresponding to the charging dock's features can be extracted from the first detection box. Since the second detection box is smaller than the first, the detection range of the charging dock can be reduced. The position information of the second detection box can then be used to adjust the self-operated device's direction of travel, resulting in more precise recharging control. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a recharging control method for a self-moving device according to an embodiment of this application;

[0024] Figure 2-a This is a schematic diagram of the structure of a charging dock provided in one embodiment of this application;

[0025] Figure 2-b This is a schematic diagram of a charging dock detection frame provided in an embodiment of this application;

[0026] Figure 3-a , 3-b Figures 3-c illustrate three application scenarios of the recharging control method for a self-moving device provided in an embodiment of this application.

[0027] Figure 4 This is a schematic diagram of the recharging control device for a self-moving device provided in an embodiment of this application;

[0028] Figure 5 This is a schematic diagram of the structure of a self-moving device provided in an embodiment of this application. Detailed Implementation

[0029] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0030] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0031] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0032] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0033] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0034] To address the problem of inaccurate control over the return of a self-moving device to its charging dock, embodiments of this application provide a method, apparatus, and storage medium for controlling the return of a self-moving device to its charging dock.

[0035] To ensure sufficient power to complete the task, the self-moving device will automatically return to the charging dock when it is finished working or when its power is low during the task, and then dock with the charging dock to recharge.

[0036] During the automatic return of the self-moving device to the charging dock, it needs to move towards the detected image area of ​​the charging dock. The heading adjustment during movement is based on the offset of the detected image area relative to the self-moving device. However, as the distance between the self-moving device and the charging dock decreases, the detected image area of ​​the charging dock increases, leading to a decrease in the accuracy of the self-moving device's positioning of the charging dock when turning, causing the self-moving device to fail to accurately dock with the charging dock.

[0037] This application provides a method, apparatus, and electronic device for controlling the recharge of a self-moving device. The method is applied during the process of the self-moving device returning to its charging dock. When the self-moving device approaches the charging dock, a distinctive feature on the charging dock is identified from the image area containing the charging dock. The image area containing this distinctive feature is used as a new target, reducing the size of the target to prevent the self-moving device from detecting an excessively large target due to proximity to the charging dock, thus ensuring accurate direction control. The self-moving device controls its direction based on the position of the target. If the direction deviates from the target, it is adjusted to return to the target direction. This application improves the accuracy of direction control for self-moving devices, enabling more precise recharge and solving the problem of inaccurate control over the return of self-moving devices to the charging dock.

[0038] Figure 1 This application provides a schematic flowchart of a recharging control method for a self-moving device, as illustrated in an embodiment of the present application. Figure 1 The recharging control method for this self-operated device includes the following steps:

[0039] In S101, an environmental image of the mobile device during the recharging process is acquired.

[0040] The aforementioned self-moving device can be a device that includes self-movement assistance functionality. This self-movement assistance functionality can be implemented through an in-vehicle terminal, and the corresponding self-moving device can be a vehicle equipped with that in-vehicle terminal. The self-moving device can also be a semi-self-moving device or a fully autonomous device. For example, it can include intelligent lawnmowers, robot vacuums, and robots with navigation functions.

[0041] Specifically, in the above steps, one possible implementation is that the self-moving device is equipped with an image acquisition device with image acquisition function to acquire environmental images around the self-moving device. This image acquisition device can be fixed or non-fixed and rotatable. The image acquisition device can be an RGB camera, a depth camera, or other devices with image acquisition function, etc., and this application embodiment does not limit this.

[0042] In some implementations, when the mobile device finishes its work or runs out of power during operation and returns to a preset recharge range, an image acquisition device is activated to capture environmental images during the recharge process. This reduces the need for image acquisition and processing outside of the recharge process. The recharge range can be an area centered on the charging dock with a preset radius, which can be set according to the specific scenario.

[0043] For example, Figure 2-a The diagram shown is a structural schematic of a charging dock provided in one embodiment of this application. Figure 2-a As shown, the charging stand 200 includes a charging base 210, a charging post 220, a lighting device 230, and a charging interface 250. The charging base 210 is typically used to support the device being charged, such as a mobile device. The charging interface 250 is disposed on the charging post 220, and the height of the charging interface 250 from the ground is the same as the height of the charging electrode of the mobile device from the ground, so that the charging interface 250 can be connected to the mobile device.

[0044] In step S102, target detection is performed on the environmental image to obtain a first detection box including the charging dock.

[0045] In the above steps, one possible implementation is that the self-moving device inputs the environmental image into a pre-trained deep learning model for object detection and outputs the detected image region including the entire charging dock and the charging dock head. The entire image region of the charging dock is marked using a first detection box. For example, as shown... Figure 2-b The diagram shown is a schematic representation of a charging dock detection frame according to an embodiment of this application. Figure 2-b The overall detection frame 201 of the charging dock shown is the first detection frame mentioned above.

[0046] For example, to build a deep learning model, images of charging docks from various different perspectives can be used as the training set for the deep learning model, and the deep learning model can be trained to recognize charging docks.

[0047] In S103, the second detection box corresponding to the charging dock feature is extracted from the first detection box.

[0048] In the above steps, one possible implementation is that the charging dock feature is an identifying feature on the charging dock, which includes, but is not limited to, lighting devices, color markings and / or other markings that can be obtained through image recognition; the image area in the second detection frame is smaller than the image area in the first detection frame.

[0049] In one embodiment, a possible implementation is that, since the first detection frame is the overall detection frame of the charging dock, it also includes the image area of ​​the charging dock head, which can be marked using a second detection frame. For example... Figure 2-b The charging dock head detection frame 221 shown is the second detection frame.

[0050] For example, the aforementioned charging dock features may include, but are not limited to, a lighting device mounted on the charging dock. This lighting device may be an incandescent lamp, a light-emitting diode, or other devices capable of producing a stable light source. The aforementioned second detection frame includes the image area of ​​the aforementioned lighting device corresponding to the charging dock head detection frame, such as... Figure 2-a or Figure 2-b The lighting device 230 shown.

[0051] In one possible implementation, the center position of the charging dock head detection frame is determined; if the center position is within the overall charging dock detection frame, the second detection frame corresponding to the lighting device is extracted from the charging dock head detection frame.

[0052] Specifically, the coordinates of the top-left and bottom-right corners of the charging dock head detection frame are obtained from a preset coordinate system, such as an image coordinate system. Then, these coordinates are divided by 2 to obtain the coordinates of the center position of the charging dock head detection frame. For example, if the top-left corner coordinates of the charging dock head detection frame are (1, 2) and the bottom-right corner coordinates are (2, 1), then the x-axis coordinate of the center point of the charging dock head detection frame is (1+2) / 2 = 1.5, and the y-axis coordinate is (2+1) / 2 = 1.5. Therefore, the coordinates of the center position are (1.5, 1.5). Figure 2-b The center position of the charging dock head detection frame is 240.

[0053] Based on the coordinates of the center position obtained from the above calculation, if the coordinates of the center position are within the coordinate range of the overall detection frame of the charging dock, it indicates that the second detection frame includes a lighting device, that is, the lighting device in the head detection frame of the charging dock is extracted.

[0054] In one possible implementation, the second detection box corresponding to the aforementioned lighting device is extracted. If it is nighttime and insufficient light is present, the lighting device's illumination will be more prominent, thus it can be used as a feature of the charging dock in low-light conditions. However, if the lighting device is not clearly visible in a well-lit environment, making it impossible to accurately capture the illuminated image area, other features of the charging dock can be used as positioning targets. For example, specific color markings can be applied to the charging dock, allowing its features to serve as positioning targets in various environments.

[0055] In another embodiment, one possible way to extract the second detection box corresponding to the lighting device from the head detection box of the charging dock is to perform a binarization operation on the image region corresponding to the head detection box of the charging dock to obtain a grayscale feature map.

[0056] For example, the above binarization operation is to perform BGR (blue, green, red) channel color separation on the image area. Since the light emitted by the lighting device is biased towards red, only the pixels of the R channel are left. That is, the pixel value of the red-biased pixels is set to 255, and the pixel value of the remaining pixels is set to 0, thus obtaining a grayscale feature map.

[0057] In one possible implementation, after obtaining the grayscale feature map, a filtering operation is performed on the grayscale feature map. Connected component calculation is then performed on the filtered grayscale feature map to obtain the image region within the second detection frame corresponding to the lighting device.

[0058] Specifically, one possible implementation of the above-mentioned filtering operation on the grayscale feature map is to calculate multiple connected components in the grayscale feature map, obtain the size of multiple connected components, set the connected components with a size greater than a preset size threshold as target connected components, retain the target connected components, and filter out the connected components other than the target connected components to obtain the filtered grayscale feature map.

[0059] In this context, image regions consisting of adjacent pixels with the same pixel value are defined as connected components. The preset size threshold can be set based on the size of the smallest connected component, or based on the ratio of the width w to the height h of the smallest connected component, such as w / h; no specific limitation is made here. The size of a connected component includes both width and height.

[0060] For example, a size threshold of 5 pixels can be set, and connected components whose width and / or height is less than 5 pixels can be filtered out. Alternatively, when the size threshold is set to w / h = 0.5, connected components with a width less than 0.5 can be filtered out.

[0061] Furthermore, filtering operations also include operations on the horizontal or vertical directions of the grayscale feature map, depending on the specific application scenario. For example, if the lighting device is installed horizontally, then the connected components in the vertical direction of the grayscale feature map are filtered; conversely, the connected components in the horizontal direction of the grayscale feature map are filtered.

[0062] Furthermore, the target connected components are merged, that is, the target connected components with the same pixel value are merged to obtain the image region in the second detection frame corresponding to the lighting device.

[0063] In S104, the position information of the second detection frame is determined.

[0064] In the above steps, one possible implementation is that the position information of the second detection frame is represented by obtaining the center coordinate point of the second detection frame. The calculation method is the same as the step of obtaining the center position of the charging seat head detection frame provided in an embodiment of S103 of this application, and will not be repeated here.

[0065] In S105, the self-moving device is controlled to move toward the charging dock based on the position information of the second detection frame.

[0066] In the above steps, one possible implementation is that, after obtaining the center coordinate point of the second detection frame in S104 of this application embodiment, the self-moving device moves. During the movement, the traveling direction control method is used to control the traveling direction of the self-moving device based on the center coordinate point.

[0067] In one embodiment, one possible implementation is that the self-moving device can also acquire the central axis of the environmental image, and use the central axis to determine whether the charging dock is located directly in front of the self-moving device in the direction of operation.

[0068] Specifically, depending on the specifications of the image acquisition device, environmental images of different sizes can be acquired. The central axis is determined based on the size of the acquired environmental images.

[0069] For example, when the image acquisition device is a camera with a resolution of 640*480, meaning the captured environmental image has a width of 640 and a height of 480, the x-axis coordinate of the central axis is 640 / 2 = 320, and the y-axis coordinate is in the range of greater than 0 and less than 480. Similarly, the camera's specifications include, but are not limited to, other specifications such as 720*1280 or 1080*1920.

[0070] In one possible implementation, after the self-moving device acquires the central axis of the environmental image, it can also determine the position of the charging dock relative to the self-moving device by determining the offset of the center coordinate point of the second detection frame acquired in the aforementioned S104 relative to the central axis.

[0071] Specifically, the aforementioned offset can be obtained by comparing the x-axis coordinate value of the center point with the x-axis coordinate value of the central axis. When the x-axis coordinate value is less than the x-axis coordinate value of the central axis, the center point is offset to the left relative to the central axis; when the x-axis coordinate value is greater than the x-axis coordinate value of the central axis, the center point is offset to the right relative to the central axis; when the x-axis coordinate value is equal to the x-axis coordinate value of the central axis, and the y-axis coordinate value is within the range of the y-axis coordinate value of the central axis, then the center point is located on the central axis.

[0072] For example, when the x-axis coordinate of the above-mentioned central axis is 320 and the y-axis coordinate range is greater than 0 and less than 480, if the above-mentioned center coordinate point is (200, 400), the x-axis coordinate value of the center coordinate point is 200 < 320. Therefore, the center coordinate is offset to the left relative to the central axis, and the offset is 200 - 320 = -120.

[0073] If the center coordinate point is (400, 400), the x-axis coordinate value of the center coordinate point is 400 > 320. Therefore, the center coordinate point is offset to the right relative to the central axis, and the offset is 400 - 320 = 80.

[0074] If the center coordinates are (320, 400), the x-axis coordinate value of the center coordinates is 320 = 320, and the y-axis coordinate value is 400, which is greater than 0 and less than 480. Therefore, the center coordinates are located on the central axis, and the offset is 400 - 320 = 80.

[0075] It should be noted that the sign of the above offset is not limited, and it is only used to determine the positional relationship between the above center coordinate point and the above central axis.

[0076] In one possible implementation, the self-moving device can also control itself to move toward the charging dock based on the acquired offset.

[0077] Specifically, if the center coordinate point is shifted to the left relative to the central axis based on the offset, then the mobile device is controlled to turn to the left.

[0078] If, based on the aforementioned offset, the center coordinate point is shifted to the right relative to the central axis, then turn to the right.

[0079] If the center coordinate point is determined to be located on the central axis based on the offset, then the self-moving device is controlled to move in a straight line.

[0080] Based on the above Figure 1 The provided method for recharging control of self-moving devices, combined with subsequent... Figure 3-a , 3-b The application scenario diagram of the recharge control method for self-moving devices provided by 3-c is used to illustrate this.

[0081] Figure 3-a , 3-b Figures 3-c illustrate three application scenarios of the recharging control method for self-moving devices provided in the embodiments of this application.

[0082] In one possible implementation, during the recharging control process, the self-moving device has an image acquisition device that can continuously acquire environmental images directly in front of the self-moving device's direction of travel, and perform target detection on the environmental images to obtain a first detection box including the charging dock. From the first detection box, a second detection box corresponding to the features of the charging dock is extracted, and the position information of the second detection box is determined.

[0083] Specifically, based on the position information of the acquired second detection frame, the position of the charging dock relative to the self-moving device is determined. For example... Figure 3-aAs shown, the position information of the second detection box obtained at this time is located in the front right of the self-moving device, which means that the charging dock is located in the front right of the current running direction of the self-moving device.

[0084] In one possible implementation, the self-moving device can be controlled to move toward the charging dock based on the already determined position information of the second detection frame.

[0085] Specifically, the driving direction of the self-moving device is controlled based on the determined position information of the second detection frame. For example... Figure 3-b As shown, at this time, based on the determined position information of the second detection frame, the self-moving device is controlled to adjust its driving direction to the right front and drive to the right front.

[0086] In one possible implementation, since the self-moving device is constantly in motion, it can continuously acquire the position information of the second detection frame to update the orientation of the charging dock relative to the self-moving device. When it is determined that the charging dock is once again directly in front of the self-moving device, the self-moving device will readjust its driving direction to move forward.

[0087] Specifically, the position information of the second detection frame is continuously acquired. When it is determined, based on this position information, that the charging dock is directly in front of the self-moving device, the driving direction of the self-moving device is adjusted to be directly in front. For example... Figure 3-c As shown, once the charging dock is determined to be directly in front of the environment image captured by the image acquisition device, the self-moving device readjusts its driving direction to be directly forward. At this time, the charging dock is directly in front of the self-moving device, and the self-moving device continues to drive forward.

[0088] In one possible implementation, the self-moving device continues to travel straight ahead. Once the self-moving device has successfully docked with the charging dock, the charging dock sends a charging signal. Upon receiving this charging signal, the self-moving device terminates the recharge control and stops moving.

[0089] Figure 4 A schematic diagram of the structure of a recharging control device for a self-moving device according to an embodiment of this application is shown. Figure 4 The device includes an image acquisition module 401, a first detection module 402, a second detection module 403, a first positioning module 404, a motion control module 405, a second positioning module 406, and a judgment and extraction module 407, wherein:

[0090] Image acquisition module 401 is used to acquire environmental images during the recharging process of the mobile device.

[0091] The first detection module 402 is used to perform target detection on the environmental image to obtain a first detection box including the charging dock.

[0092] The second detection module 403 is used to extract a second detection box corresponding to the charging seat feature from the first detection box, wherein the charging seat feature is the identification feature on the charging seat, and the image area of ​​the second detection box is smaller than the image area of ​​the first detection box.

[0093] The first positioning module 404 is used to determine the position information of the second detection frame.

[0094] The movement control module 405 controls the self-moving device to move toward the charging dock based on the position information of the second detection frame.

[0095] In one embodiment, the charging dock features include a lighting device mounted on the charging dock, and the first detection frame includes an overall charging dock detection frame and a charging dock head detection frame. Therefore, the recharging control device for the self-moving device further includes the following modules:

[0096] The second positioning module 406 is used to determine the center position of the charging dock head detection frame.

[0097] The judgment and extraction module 407 is used to extract the second detection frame corresponding to the lighting device from the head detection frame of the charging dock if the center position is within the overall detection frame of the charging dock.

[0098] The interaction between the modules in the aforementioned recharging control device for self-moving devices, and the detailed process for achieving the functions, are described in the foregoing embodiments of this application. Figure 1 The relevant descriptions will not be repeated here.

[0099] Figure 5 This is a schematic diagram of the structure of a self-moving device provided in an embodiment of this application. Figure 5 As shown, the self-moving device 5 in this embodiment includes: an image acquisition unit 50, and at least one processor 51. Figure 5 (Only one is shown) a processor, a memory 52, and a computer program 53 stored in the memory 52 and capable of running on at least one processor 51. The image acquisition device 50 is used to acquire images, and the processor 51 executes the computer program 53 to implement the steps in any of the above-described self-moving device method embodiments.

[0100] The self-moving device 5 can be a device with self-movement assistance functions, or it can be a semi-self-moving or fully autonomous mobile device such as a smart lawnmower, a sweeper, or a robot with navigation functions. It can also be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The self-moving device 5 may include, but is not limited to, an image acquisition unit 50, a processor 51, and a memory 52. ​​Those skilled in the art will understand that… Figure 5This is merely an example of the self-moving device 5 and does not constitute a limitation on the self-moving device 5. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0101] The image acquisition device 50 can be a camera, RGB camera, depth camera or scanning device or other devices with image acquisition function.

[0102] The processor 51 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0103] In some embodiments, memory 52 may be an internal storage unit of the self-moving device, such as a hard disk or memory of the self-moving device 5. In other embodiments, memory 52 may be an external storage device of the self-moving device 5, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the self-moving device 5. Furthermore, memory 52 may include both internal and external storage units of the self-moving device 5. Memory 52 is used to store operating systems, applications, bootloaders, data, and other programs, such as program code for computer programs. Memory 52 may also be used to temporarily store data that has been output or will be output.

[0104] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0105] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0106] This application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor. When the processor executes the computer program, it implements the steps in any of the above method embodiments.

[0107] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0108] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0109] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0110] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0111] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0112] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0114] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A recharging control method for a self-operated mobile device, characterized in that, The method includes: Acquire environmental images during the recharging process of a mobile device; The environmental image is subjected to target detection to obtain a first detection box including the charging seat, which includes: inputting the environmental image into a pre-trained deep learning model for target detection, and outputting the detected image region including the entire charging seat and the image region of the charging seat head, and marking the image region of the entire charging seat using the first detection box. Extracting a second detection frame corresponding to the charging seat feature from the first detection frame; wherein, the charging seat feature is an identifying feature on the charging seat, and the image area of ​​the second detection frame is smaller than the image area of ​​the first detection frame; the charging seat feature includes a lighting device installed on the charging seat, and the first detection frame includes a charging seat overall detection frame and a charging seat head detection frame, the step of extracting the second detection frame corresponding to the charging seat feature from the first detection frame includes: determining the center position of the charging seat head detection frame; if the center position is within the charging seat overall detection frame, then extracting the second detection frame corresponding to the lighting device from the charging seat head detection frame; Determine the position information of the second detection frame; The self-moving device is controlled to move toward the charging dock based on the position information of the second detection frame.

2. The method of claim 1, wherein, The step of extracting the second detection frame corresponding to the lighting device from the detection frame at the head of the charging dock includes: Binarize the image region corresponding to the detection frame at the head of the charging dock to obtain a grayscale feature map. Perform a filtering operation on the grayscale feature map; Connectivity calculation is performed on the filtered grayscale feature map to obtain the image region corresponding to the second detection box of the lighting device.

3. The method of claim 2, wherein, The filtering operation on the grayscale feature map includes: Calculate multiple connected components of the grayscale feature map; Obtain the size of each of the connected components; The connected components whose size is greater than a preset size threshold are identified as target connected components; The target connected component is retained, and connected components other than the target connected component are filtered out to obtain the filtered grayscale feature map.

4. The method of claim 1, wherein, Determining the position information of the second detection frame includes: Determine the coordinates of the top left and bottom right corners of the second detection frame; Based on the coordinates of the upper left corner and the lower right corner, the coordinates of the center point of the second detection box are determined, and the coordinates of the center point are used as the position information of the second detection box.

5. The method of claim 4, wherein, The step of controlling the self-moving device to move toward the charging dock based on the position information of the second detection frame includes: Obtain the centerline of the environmental image; Determine the offset of the center point coordinates relative to the central axis; based on the offset, control the self-moving device to move towards the charging dock.

6. A recharging control apparatus of a self-moving device, characterized by comprising: include: The image acquisition module is used to acquire environmental images during the recharging process of the mobile device; The first detection module is used to perform target detection on the environmental image to obtain a first detection box including the charging seat. The module includes: inputting the environmental image into a pre-trained deep learning model for target detection, and outputting the detected image region including the entire charging seat and the image region of the charging seat head, and marking the image region of the entire charging seat using the first detection box. The second detection module is used to extract a second detection box corresponding to the charging seat feature from the first detection box, wherein the charging seat feature is an identifying feature on the charging seat, and the image area of ​​the second detection box is smaller than the image area of ​​the first detection box. The first positioning module is used to determine the position information of the second detection box; The movement control module controls the self-moving device to move toward the charging dock based on the position information of the second detection frame; The charging dock features a lighting device mounted on it, the first detection frame includes a charging dock overall detection frame and a charging dock head detection frame, and the device further includes: The second positioning module is used to determine the center position of the charging dock head detection frame; The determination and extraction module is used to extract the second detection frame corresponding to the lighting device from the head detection frame of the charging dock if the center position is within the overall detection frame of the charging dock.

7. A self-moving device comprising an image collector, a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The image acquisition device is used to acquire images, and the processor executes the computer program to implement the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 7. When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 5.

Images