Pool robot control method and apparatus, storage medium, and pool robot

Abstract

Embodiments of the present disclosure provide a pool robot control method and apparatus, a storage medium, and a pool robot. The method comprises: acquiring image information of a target water area captured by a pool robot, wherein the pool robot runs in the target water area; analyzing the image information to determine an analysis result; and if the analysis result is set to indicate the presence of a target object in the target water area, determining a target position of the target object, controlling the pool robot to run to the target position, and executing a target operation corresponding to the type of the target object.

Description

Control method, device, storage medium and pool robot

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure is based on Chinese patent application CN202310768678.3, filed on June 27, 2023, entitled “Control method, device, storage medium and pool robot for pool robot”, and claims the priority of the patent application, and all the contents disclosed therein are incorporated into this disclosure by reference. Technical Field

[0003] The embodiments of the present disclosure relate to the field of computers, and in particular, to a control method, device, storage medium, and pool robot. Background Art

[0004] The pool robot in the related art can perform one or more work tasks in the pool (for example, disinfection, cleaning, etc.), but when the pool robot performs an operation on a specific object, it cannot quickly determine the location of the object, thereby affecting the efficiency of performing the operation.

[0005] Regarding the above problems, relevant technologies have not yet proposed effective solutions.

[0006] Summary of the Invention

[0007] The embodiments of the present disclosure provide a control method, device, storage medium, and pool robot for a pool robot, so as to at least solve the problem of low efficiency of the pool robot in operating specific objects in the related art.

[0008] According to one embodiment of the present disclosure, a control method for a pool robot is provided, comprising: acquiring image information of the target water area photographed by the pool robot, wherein the pool robot operates in the target water area; analyzing the image information to determine an analysis result; when the analysis result is set to indicate that a target object exists in the target water area, determining a target position of the target object, and controlling the pool robot to operate to the target position, and performing a target operation corresponding to the type of the target object.

[0009] In an exemplary embodiment, obtaining image information of the target water area photographed by the pool robot includes: obtaining image information of the target water area photographed by the pool robot when operating in a first operating mode, wherein the first operating mode includes a mode in which the pool robot operates in a random route between multiple preset positions according to a preset time period.

[0010] In an exemplary embodiment, when the above-mentioned analysis result is set to indicate that there is a target object in the above-mentioned target water area, the target position of the above-mentioned target object is determined, and the above-mentioned pool robot is controlled to run to the above-mentioned target position, and the target operation corresponding to the type of the above-mentioned target object is performed, including: when the above-mentioned analysis result is set to indicate that there is a target object in the above-mentioned target water area, the above-mentioned target position is determined according to the above-mentioned image information; the above-mentioned pool robot is controlled to run to the above-mentioned target position, and the target operation corresponding to the type of the above-mentioned target object is performed.

[0011] In an exemplary embodiment, after obtaining the image information of the target water area captured by the pool robot, the method further includes: determining the pixel coordinates of the target object in multiple frames of images to obtain multiple pixel coordinates, wherein the image information includes multiple frames of the image; using the first configuration parameters of the camera device in the pool robot to transform and map the multiple pixel coordinates into the camera device coordinate system to obtain multiple mapping coordinates, wherein the first configuration parameters include the parameters when capturing the image; determining the target position of the target object in the target water area using the multiple mapping coordinates and the second configuration parameters of the camera device, wherein the second configuration parameters include the posture information when the camera device captured the image, and the posture information includes a rotation matrix and a translation vector.

[0012] In an exemplary embodiment, the first configuration parameters of the camera device in the above-mentioned pool robot are used to transform and map the multiple pixel coordinates into the camera device coordinate system to obtain multiple mapping coordinates, including: transforming and mapping the multiple pixel coordinates into the normalized coordinates in the above-mentioned camera device coordinate system to obtain multiple mapping coordinates; or transforming and mapping the multiple pixel coordinates into the imaging surface coordinates in the above-mentioned camera device coordinate system to obtain multiple mapping coordinates.

[0013] In an exemplary embodiment, obtaining image information of the target water area photographed by the pool robot includes: obtaining image information of the target water area photographed by the pool robot when operating in a second operating mode, wherein the second operating mode includes a mode in which the pool robot operates in a fixed route between multiple preset positions according to a preset time period.

[0014] In an exemplary embodiment, when the above-mentioned analysis result is set to indicate that the above-mentioned target water area includes a target object, the target position of the above-mentioned target object is determined, and the above-mentioned pool robot is controlled to run to the above-mentioned target position, and a target operation corresponding to the type of the above-mentioned target object is performed, including: when the above-mentioned pool robot is operating in accordance with the second operating mode, when it is determined from the above-mentioned analysis result that the above-mentioned target water area includes the above-mentioned target object, the target position of the above-mentioned target object is determined, and the above-mentioned pool robot is controlled to run to the above-mentioned target position, and a target operation corresponding to the type of the above-mentioned target object is performed.

[0015] In an exemplary embodiment, when the above-mentioned analysis result is set to indicate that there is a target object in the above-mentioned target water area, the target position of the above-mentioned target object is determined, and the above-mentioned pool robot is controlled to run to the above-mentioned target position, and the target operation corresponding to the type of the above-mentioned target object is performed, including: after the above-mentioned pool robot operates in accordance with the second operating mode, when it is determined from the above-mentioned analysis result that the above-mentioned target water area includes multiple target objects, the positions of the multiple target objects are marked in the above-mentioned map, wherein the above-mentioned map is constructed during the operation of the above-mentioned pool robot in the above-mentioned target water area; the operation route is determined according to the positions of the multiple target objects; the above-mentioned pool robot is controlled to run to multiple positions of the multiple target objects according to the operation route, and the target operation corresponding to the type of each of the above-mentioned target objects is performed.

[0016] In an exemplary embodiment, determining a running route according to the positions of multiple target objects includes: inputting the positions of the multiple target objects into a target shortest path model to obtain a running route, wherein the running route is the shortest running route passing through the positions of the multiple target objects.

[0017] In an exemplary embodiment, the target shortest path model is determined by: defining the objective function as the length of the running route; taking a plurality of the running routes randomly combined as a parent generation, and taking a plurality of the running routes randomly combined again as a child generation; and determining the target shortest path model using the parent generation and the child generation.

[0018] In an exemplary embodiment, before obtaining the image information of the target water area captured by the pool robot, the method further includes: controlling the pool robot to perform the target operation along the edge of the target water area; after determining that the pool robot performs the target operation along the edge, controlling the pool robot to perform a patrol operation in the target water area, wherein the patrol operation includes controlling the pool robot to capture the image information in the target water area.

[0019] In an exemplary embodiment, before obtaining the image information of the target water area captured by the pool robot, the method further includes: controlling the pool robot to perform a work task in the target water area; wherein the work task includes performing the target operation along the boundary of the target water area, performing the target operation in the target water area according to a planned route, and performing the target operation again along the boundary of the target water area; after determining that the pool robot has completed the work task, controlling the pool robot to perform a patrol operation in the target water area, wherein the patrol operation includes controlling the pool robot to capture the image information in the target water area.

[0020] In an exemplary embodiment, the target object is determined in the following manner: image information is input into a target network model to obtain feature information of the object included in the image information output by the target network model, wherein the feature information object includes spatial features and semantic features; and the target object is determined from the feature information of the object.

[0021] In an exemplary embodiment, the target network model is trained in the following manner: M frames of sample images are obtained, wherein each frame of the sample image includes a preset object, wherein M is a natural number greater than or equal to 1; the preset object is marked in each frame of the sample image to obtain M frames of marked sample images; and the original network model is trained using the M frames of marked sample images to obtain a target network model.

[0022] According to another embodiment of the present disclosure, a control device for a pool robot is provided, comprising: an acquisition module, configured to acquire image information of the target water area photographed by the pool robot, wherein the pool robot operates in the target water area; an analysis module, configured to analyze the image information to determine an analysis result; a determination module, configured to determine a target position of the target object when the analysis result is configured to indicate that a target object exists in the target water area, and to control the pool robot to operate to the target position and perform a target operation corresponding to the type of the target object.

[0023] In an exemplary embodiment, the acquisition module includes: a first acquisition sub-module, configured to acquire image information of the target water area captured by the pool robot when the pool robot operates in a first operating mode, wherein the first operating mode includes a mode in which the pool robot operates in a random route between multiple preset positions according to a preset time period.

[0024] In an exemplary embodiment, the above-mentioned determination module includes: a first determination submodule, which is configured to determine the above-mentioned target position based on the above-mentioned image information when the above-mentioned analysis result is set to indicate that there is a target object in the above-mentioned target water area; a control submodule, which is configured to control the above-mentioned pool robot to run to the above-mentioned target position and perform a target operation corresponding to the type of the above-mentioned target object.

[0025] In an exemplary embodiment, the acquisition module includes: a second acquisition sub-module, configured to acquire image information of the target water area captured by the pool robot when the pool robot operates in a second operating mode, wherein the second operating mode includes a mode in which the pool robot operates in a fixed route between multiple preset positions according to a preset time period.

[0026] In an exemplary embodiment, the above-mentioned determination module includes: a second determination sub-module, which is configured to determine the target position of the above-mentioned target object when the above-mentioned pool robot is operating in accordance with the second operating mode, and control the above-mentioned pool robot to operate to the above-mentioned target position and perform the target operation corresponding to the type of the above-mentioned target object when it is determined from the above-mentioned analysis results that the above-mentioned target water area includes the above-mentioned target object.

[0027] In an exemplary embodiment, the above-mentioned device is also used to obtain the image information of the above-mentioned target water area photographed by the pool robot, determine the pixel coordinates of the above-mentioned target object in multiple frames of images, and obtain multiple pixel coordinates, wherein the above-mentioned image information includes multiple frames of the above-mentioned images; use the first configuration parameters of the camera device in the above-mentioned pool robot to transform and map the multiple above-mentioned pixel coordinates into the camera device coordinate system to obtain multiple mapping coordinates, wherein the above-mentioned first configuration parameters include the parameters when the above-mentioned images are photographed; use the multiple above-mentioned mapping coordinates and the second configuration parameters of the above-mentioned camera device to determine the target position of the above-mentioned target object in the above-mentioned target water area, wherein the above-mentioned second configuration parameters include the posture information of the above-mentioned camera device when photographing the above-mentioned images, and the above-mentioned posture information includes the rotation matrix and the translation vector.

[0028] In an exemplary embodiment, the first configuration parameters of the camera device in the above-mentioned pool robot are used to transform and map the multiple pixel coordinates into the camera device coordinate system to obtain multiple mapping coordinates in the following manner: the multiple pixel coordinates are transformed and mapped into the normalized coordinates in the camera device coordinate system to obtain multiple mapping coordinates; or the multiple pixel coordinates are transformed and mapped into the imaging surface coordinates in the camera device coordinate system to obtain multiple mapping coordinates.

[0029] In an exemplary embodiment, the above-mentioned determination module also includes: a third determination submodule, which is configured to mark the positions of the multiple target objects in the above-mentioned map when it is determined from the above-mentioned analysis results that the target water area includes multiple target objects after the above-mentioned pool robot operates in the second operating mode, wherein the above-mentioned map is constructed during the operation of the above-mentioned pool robot in the above-mentioned target water area; a fourth determination submodule, which is configured to determine the operating route according to the positions of the multiple target objects; and a control submodule, which is configured to control the above-mentioned pool robot to operate to multiple positions of the multiple target objects according to the operating route, and perform target operations corresponding to the type of each of the above-mentioned target objects.

[0030] In an exemplary embodiment, the running route is determined according to the positions of multiple target objects in the following manner: the positions of the multiple target objects are input into a target shortest path model to obtain a running route, wherein the running route is the shortest running route passing through the positions of the multiple target objects.

[0031] In an exemplary embodiment, the target shortest path model is determined by: defining the objective function as the length of the running route; taking a plurality of the running routes randomly combined as a parent generation, and taking a plurality of the running routes randomly combined again as a child generation; and determining the target shortest path model using the parent generation and the child generation.

[0032] In an exemplary embodiment, the above-mentioned device also includes: a first control module, which is configured to control the above-mentioned pool robot to perform the above-mentioned target operation along the edge of the above-mentioned target water area before obtaining the image information of the above-mentioned target water area photographed by the pool robot; a second control module, which is configured to control the above-mentioned pool robot to perform a patrol operation in the above-mentioned target water area after determining that the above-mentioned pool robot performs the above-mentioned target operation along the edge, wherein the above-mentioned patrol operation includes controlling the above-mentioned pool robot to photograph the above-mentioned image information in the above-mentioned target water area.

[0033] In an exemplary embodiment, the above-mentioned device is also used to control the above-mentioned pool robot to perform a work task in the above-mentioned target water area before obtaining the image information of the above-mentioned target water area photographed by the pool robot; wherein, the work task includes performing the above-mentioned target operation along the boundary of the above-mentioned target water area, performing the above-mentioned target operation in the above-mentioned target water area according to the planned route, and performing the above-mentioned target operation again along the boundary of the above-mentioned target water area; after determining that the above-mentioned pool robot has completed the above-mentioned work task, the above-mentioned pool robot is controlled to perform a patrol operation in the above-mentioned target water area, wherein the above-mentioned patrol operation includes controlling the above-mentioned pool robot to photograph the above-mentioned image information in the above-mentioned target water area.

[0034] In an exemplary embodiment, the target object is determined in the following manner: image information is input into a target network model to obtain feature information of the object included in the image information output by the target network model, wherein the feature information includes spatial features and semantic features of the object; and the target object is determined from the feature information of the object.

[0035] In an exemplary embodiment, the target network model is trained in the following manner: M frames of sample images are obtained, wherein each frame of the sample image includes a preset object, wherein M is a natural number greater than or equal to 1; the preset object is marked in each frame of the sample image to obtain M frames of marked sample images; and the original network model is trained using the M frames of marked sample images to obtain a target network model.

[0036] According to another embodiment of the present disclosure, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when run.

[0037] According to another embodiment of the present disclosure, a pool robot is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0038] Through the present disclosure, by analyzing the image information of the target water area taken by the pool robot, the target object can be identified and the position of the target object can be confirmed, and then the robot can be controlled to go directly to the position of the target object and perform the target operation on the target object, avoiding the situation where the pool robot cannot locate the target object and can only re-clean the entire target water area according to the preset route. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] FIG1 is a hardware structure block diagram of a mobile terminal of a method for controlling a pool robot according to an embodiment of the present disclosure;

[0040] FIG2 is a flow chart of a control method of a pool robot according to an embodiment of the present disclosure;

[0041] FIG3 is a structural block diagram of a control device of a pool robot according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings and in conjunction with the embodiments.

[0043] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are set to distinguish similar objects, and are not necessarily set to describe a specific order or sequence.

[0044] The method embodiments provided in the embodiments of the present disclosure can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, FIG1 is a hardware structure block diagram of a mobile terminal of a pool robot control method according to an embodiment of the present disclosure. As shown in FIG1 , the mobile terminal may include one or more (only one is shown in FIG1 ) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 configured to store data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 configured to have a communication function. It will be understood by those skilled in the art that the structure shown in FIG1 is for illustration only and does not limit the structure of the mobile terminal. For example, the mobile terminal may also include more or fewer components than those shown in FIG1 , or have a configuration different from that shown in FIG1 .

[0045] The memory 104 can be configured to store computer programs, such as software programs and modules of application software, such as the computer program corresponding to the pool robot control method in the embodiment of the present disclosure. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-mentioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0046] The transmission device 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the mobile terminal's telecommunications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module configured to communicate with the Internet wirelessly.

[0047] In this embodiment, a method for controlling a pool robot is provided. FIG2 is a flow chart according to an embodiment of the present disclosure. As shown in FIG2 , the flow chart includes the following steps:

[0048] Step S202, obtaining image information of the target water area photographed by the pool robot, wherein the pool robot operates in the target water area;

[0049] Step S204, analyzing the image information to determine an analysis result;

[0050] Step S206, when the above analysis result is set to indicate that there is a target object in the above target water area, determine the target position of the above target object, control the above pool robot to move to the above target position, and perform a target operation corresponding to the type of the above target object.

[0051] The execution subject of the above steps may be the controller of the pool robot, or a processor with data processing and signal interaction capabilities, or other processing devices or processing units with similar processing capabilities, but not limited thereto.

[0052] In the above embodiment, the device used by the pool robot for shooting is a camera with a high waterproof grade. The camera can be placed at the front, side, top, etc. of the pool robot, and the specific location is not limited. The target water area includes but is not limited to swimming pools, ornamental reservoirs, outdoor pools, etc. The pool robot can be located on the surface of the target water area, or at the bottom of the target water area, or suspended in the target water area. If the target water area is a swimming pool, it can also be located on the wall of the swimming pool. The image information can be analyzed by recognizing the image information through a pre-trained network model, or by using other image analysis equipment to recognize the image information. The target objects include but are not limited to various types of water garbage, such as plastic bottle caps, leaves, etc. The image information can be a picture, or a video including multiple frames of images. The image information is obtained in real time while the robot is patrolling the target water area. The above-mentioned target operations include but are not limited to cleaning operations, disinfection operations, alarm operations, etc. For example, the pool robot can clean various types of water garbage (such as plastic bottle caps, leaves, etc.). The pool robot can also disinfect the area where the target object that needs to be disinfected is located. The pool robot can also send an alarm signal when it detects that the target object is a person or animal that accidentally falls into the water.

[0053] Optionally, analyzing the image information to determine the analysis results includes: inputting the image information into a target network model to obtain feature information of the objects included in the image information output by the target network model, wherein the feature information includes spatial features and semantic features of the objects included in each frame of the image; and determining the target object from the feature information of the objects included in each frame of the image. In this embodiment, the spatial features are shallow visual features, including texture, color, shape, spatial relationships, etc., and the semantic features include deep relationships between objects.

[0054] Optionally, a training method for the target network model includes: obtaining M frames of sample images, wherein each frame of the sample image includes a preset object, wherein the M is a natural number greater than or equal to 1; marking the preset object in each frame of the sample image to obtain M frames of marked sample images; using the M frames of marked sample images to train the original network model to obtain the target network model. In this embodiment, the original network model can be yolov3 (the third version of the YOLO algorithm) or FasterRcnn (an optimized version of the FastRcnn algorithm, in which the region of interest is generated using a RegionProposal network). Of course, the above algorithm is only an illustrative example, and the original network model can be any model that can meet the recognition requirements. The sample image can be a network resource, or manually shot data, or AI-generated training material. The target objects in the sample image can be marked manually, and each sample image can include multiple target objects.

[0055] Through the embodiments of the present disclosure, by analyzing the image information of the target water area captured by the pool robot, the target object can be identified and the position of the target object can be confirmed, and then the robot can be controlled to go directly to the position of the target object and perform the target operation on the target object, avoiding the situation where the pool robot cannot locate the target object and can only re-clean the entire target water area according to the preset route.

[0056] In one exemplary embodiment, obtaining image information of the target water area captured by the pool robot includes obtaining image information of the target water area captured by the pool robot while operating in a first operating mode, wherein the first operating mode comprises a mode in which the pool robot operates along a random route between a plurality of preset locations according to a preset time period. In the above embodiment, the first operating mode comprises a mode of patrolling along a random route, and the preset time period may be 5 minutes, 6 minutes, or 10 minutes. Of course, these times are merely illustrative examples, and the preset time period may be any duration that satisfies the patrol requirements of the pool robot. For example, if the target water area includes three preset locations, A, B, and C, the first operating mode may comprise the pool robot randomly patrolling between the three preset locations for 5 minutes. Through the above embodiment, by identifying individual target objects within the target water area through random route patrolling, the pool robot can effectively improve its efficiency in processing specific objects.

[0057] In an exemplary embodiment, when the above-mentioned analysis result is set to indicate that there is a target object in the above-mentioned target water area, the target position of the above-mentioned target object is determined, and the above-mentioned pool robot is controlled to run to the above-mentioned target position, and the target operation corresponding to the type of the above-mentioned target object is performed, including: when the above-mentioned analysis result is set to indicate that there is a target object in the above-mentioned target water area, the above-mentioned target position is determined according to the above-mentioned image information; the above-mentioned pool robot is controlled to run to the above-mentioned target position, and the target operation corresponding to the type of the above-mentioned target object is performed.

[0058] Optionally, the method for determining the target position based on image information includes: determining the pixel coordinates of the target object in each frame of the image to obtain N pixel coordinates, wherein the image information includes the image; using the first configuration parameters of the pool robot camera device to convert and map the N pixel coordinates into the camera device coordinate system to obtain N mapping coordinates, wherein the first configuration parameters include the parameters when the image information was captured; using the N mapping coordinates and the second configuration parameters of the camera device to determine the target position, wherein the second configuration parameters include the posture information of the camera device when the image information was captured, and the posture information includes a rotation matrix and a translation vector. In this embodiment, in the above embodiment, the above conversion and mapping of the pixel coordinates into the camera device coordinate system can be converting the pixel coordinates into normalized coordinates in the camera device coordinate system (converting to normalized coordinates can simplify the subsequent calculation of the target position), or converting the pixel coordinates into imaging surface coordinates in the camera device coordinate system (that is, normalized to focal length coordinates, in which case the distance between the imaging surface coordinates is the distance between the corresponding pixel coordinates). Through the above embodiments, the position information of the target object is determined by image information, and individual target objects in the target water area are quickly located, which can effectively improve the processing efficiency of the pool robot for specific objects.

[0059] In one exemplary embodiment, obtaining image information of the target water area captured by the pool robot includes obtaining image information of the target water area captured by the pool robot while operating in a second operating mode, wherein the second operating mode includes a mode in which the pool robot operates along a fixed route between a plurality of preset locations according to a preset time period. In the above embodiment, the second operating mode includes a mode of patrolling along a fixed route, and the preset time period may be 5 minutes, 6 minutes, or 10 minutes. Of course, these times are merely illustrative examples, and the preset time period may be any duration that satisfies the patrol requirements of the pool robot. For example, if the target water area includes three preset locations, A, B, and C, the second operating mode may include the pool robot rapidly swimming along a preset fixed route from A→B→C for 10 minutes. Through the above embodiment, by patrolling along a fixed route, the pool robot can effectively improve its efficiency in addressing specific objects.

[0060] In an exemplary embodiment, when the analysis result indicates that a target object exists in the target water area, the target position of the target object is determined, the pool robot is controlled to move to the target position, and a target operation corresponding to the type of the target object is performed, including: when the pool robot is operating in the second operating mode, when it is determined from the analysis result that the target water area includes the target object, the target position of the target object is determined, the pool robot is controlled to move to the target position, and a target operation corresponding to the type of the target object is performed. Through the above embodiment, the target object position information is determined through image information, and individual target objects in the target water area can be quickly located, which can effectively improve the pool robot's processing efficiency for specific objects.

[0061] In an exemplary embodiment, when the analysis result indicates that a target object exists in the target water area, determining a target location of the target object, controlling the pool robot to move to the target location, and performing a target operation corresponding to the type of the target object comprises: after the pool robot operates in the second operating mode, if it is determined from the analysis result that the target water area includes multiple target objects, marking the locations of the multiple target objects in the map, wherein the map is constructed during the operation of the pool robot in the target water area; determining an operation route according to the locations of the multiple target objects; and controlling the pool robot to move to multiple locations of the multiple target objects according to the operation route, and performing a target operation corresponding to the type of each target object. In the above embodiment, optionally, determining the operation route according to the locations of the multiple target objects comprises: inputting the locations of the multiple target objects into a target shortest path model to obtain an operation route, wherein the operation route is the shortest operation route passing through the locations of the multiple target objects. The target shortest path model can calculate the shortest operation route traversing the multiple locations using a heuristic algorithm (e.g., a genetic algorithm, an ant colony algorithm, etc.). For example, a method for determining the shortest path using a genetic algorithm includes defining the objective function as the length of a route, using a random combination of multiple routes as the parent generation, and a random combination of multiple routes as the offspring generation. The offspring with the best objective function value is selected as the new parent generation, and the iteration is continued. If the offspring's objective function value does not significantly exceed that of the parent generation, the offspring generation is determined to be the aforementioned route. By recording the location of target objects during fixed-path inspections and planning the route processing after the inspection is completed, processing distance can be reduced, effectively improving the pool robot's cleaning efficiency.

[0062] In an exemplary embodiment, before obtaining the image information of the target water area captured by the pool robot, the method further includes: controlling the pool robot to perform the target operation along the edge of the target water area; after determining that the pool robot performs the target operation along the edge, controlling the pool robot to perform a patrol operation in the target water area, wherein the patrol operation includes controlling the pool robot to capture the image information in the target water area. In the above embodiment, the pool robot's work tasks include performing target operations along the boundary of the target water area, performing target operations in the target water area according to a planned route, and performing target operations again along the boundary of the target water area. The above-mentioned execution of target operations along the edge can correspond to performing target operations again along the boundary of the target water area in the work task, that is, the patrol operation can be performed after the pool robot completes the work task. Through the above embodiment, patrolling after the pool robot performs the work task can supplement and clean the target water area, effectively improving the work performance of the pool robot.

[0063] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the above methods of each embodiment of the present disclosure.

[0064] This embodiment also provides a control device for a pool robot, which is configured to implement the above-described embodiments and alternative implementations. Details already described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0065] Figure 3 is a structural block diagram of the control device of the pool robot according to an embodiment of the present disclosure. As shown in Figure 3, the device includes: an acquisition module 32, configured to acquire image information of the above-mentioned target water area photographed by the pool robot, wherein the above-mentioned pool robot operates in the above-mentioned target water area; an analysis module 34, configured to analyze the above-mentioned image information to determine the analysis result; a determination module 36, configured to determine the target position of the above-mentioned target object when the above-mentioned analysis result is configured to indicate that there is a target object in the above-mentioned target water area, and control the above-mentioned pool robot to operate to the above-mentioned target position and perform a target operation corresponding to the type of the above-mentioned target object.

[0066] In an exemplary embodiment, the acquisition module includes: a first acquisition sub-module, configured to acquire image information of the target water area captured by the pool robot when the pool robot operates in a first operating mode, wherein the first operating mode includes a mode in which the pool robot operates in a random route between multiple preset positions according to a preset time period.

[0067] In an exemplary embodiment, the above-mentioned determination module includes: a first determination submodule, which is configured to determine the above-mentioned target position based on the above-mentioned image information when the above-mentioned analysis result is set to indicate that there is a target object in the above-mentioned target water area; a control submodule, which is configured to control the above-mentioned pool robot to run to the above-mentioned target position and perform a target operation corresponding to the type of the above-mentioned target object.

[0068] In an exemplary embodiment, the acquisition module includes: a second acquisition sub-module, configured to acquire image information of the target water area captured by the pool robot when the pool robot operates in a second operating mode, wherein the second operating mode includes a mode in which the pool robot operates in a fixed route between multiple preset positions according to a preset time period.

[0069] In an exemplary embodiment, the above-mentioned determination module includes: a second determination sub-module, which is configured to determine the target position of the above-mentioned target object when the above-mentioned pool robot is operating in accordance with the second operating mode, and control the above-mentioned pool robot to operate to the above-mentioned target position and perform the target operation corresponding to the type of the above-mentioned target object when it is determined from the above-mentioned analysis results that the above-mentioned target water area includes the above-mentioned target object.

[0070] In an exemplary embodiment, the above-mentioned device is also used to obtain the image information of the above-mentioned target water area photographed by the pool robot, determine the pixel coordinates of the above-mentioned target object in multiple frames of images, and obtain multiple pixel coordinates, wherein the above-mentioned image information includes multiple frames of the above-mentioned images; use the first configuration parameters of the camera device in the above-mentioned pool robot to transform and map the multiple above-mentioned pixel coordinates into the camera device coordinate system to obtain multiple mapping coordinates, wherein the above-mentioned first configuration parameters include the parameters when the above-mentioned images are photographed; use the multiple above-mentioned mapping coordinates and the second configuration parameters of the above-mentioned camera device to determine the target position of the above-mentioned target object in the above-mentioned target water area, wherein the above-mentioned second configuration parameters include the posture information of the above-mentioned camera device when photographing the above-mentioned images, and the above-mentioned posture information includes the rotation matrix and the translation vector.

[0071] In an exemplary embodiment, the first configuration parameters of the camera device in the above-mentioned pool robot are used to transform and map the multiple pixel coordinates into the camera device coordinate system to obtain multiple mapping coordinates in the following manner: the multiple pixel coordinates are transformed and mapped into the normalized coordinates in the above-mentioned camera device coordinate system to obtain multiple mapping coordinates; or the multiple pixel coordinates are transformed and mapped into the imaging surface coordinates in the above-mentioned camera device coordinate system to obtain multiple mapping coordinates.

[0072] In an exemplary embodiment, the above-mentioned determination module also includes: a third determination submodule, which is configured to mark the positions of the multiple target objects in the above-mentioned map when it is determined from the above-mentioned analysis results that the target water area includes multiple target objects after the above-mentioned pool robot operates in the second operating mode, wherein the above-mentioned map is constructed during the operation of the above-mentioned pool robot in the above-mentioned target water area; a fourth determination submodule, which is configured to determine the operating route according to the positions of the multiple target objects; and a control submodule, which is configured to control the above-mentioned pool robot to operate to multiple positions of the multiple target objects according to the operating route, and perform target operations corresponding to the type of each of the above-mentioned target objects.

[0073] In an exemplary embodiment, the running route is determined according to the positions of multiple target objects in the following manner: the positions of the multiple target objects are input into a target shortest path model to obtain a running route, wherein the running route is the shortest running route passing through the positions of the multiple target objects.

[0074] In an exemplary embodiment, the target shortest path model is determined by: defining the objective function as the length of the running route; taking a plurality of the running routes randomly combined as a parent generation, and taking a plurality of the running routes randomly combined again as a child generation; and determining the target shortest path model using the parent generation and the child generation.

[0075] In an exemplary embodiment, the above-mentioned device also includes: a first control module, which is configured to control the above-mentioned pool robot to perform the above-mentioned target operation along the edge of the above-mentioned target water area before obtaining the image information of the above-mentioned target water area photographed by the pool robot; a second control module, which is configured to control the above-mentioned pool robot to perform a patrol operation in the above-mentioned target water area after determining that the above-mentioned pool robot performs the above-mentioned target operation along the edge, wherein the above-mentioned patrol operation includes controlling the above-mentioned pool robot to photograph the above-mentioned image information in the above-mentioned target water area.

[0076] In an exemplary embodiment, the apparatus is further configured to control the pool robot to perform a work task in the target water area before acquiring image information of the target water area captured by the pool robot; wherein the work task includes performing the target operation along the boundary of the target water area, performing the target operation in the target water area according to a planned route, and performing the target operation again along the boundary of the target water area;

[0077] After determining that the pool robot completes the work task, the pool robot is controlled to perform a patrol operation in the target water area, wherein the patrol operation includes controlling the pool robot to capture the image information in the target water area.

[0078] In an exemplary embodiment, the target object is determined in the following manner: image information is input into a target network model to obtain feature information of the object included in the image information output by the target network model, wherein the feature information object includes spatial features and semantic features; and the target object is determined from the feature information of the object.

[0079] In an exemplary embodiment, the target network model is trained in the following manner: M frames of sample images are obtained, wherein each frame of the sample image includes a preset object, wherein M is a natural number greater than or equal to 1; the preset object is marked in each frame of the sample image to obtain M frames of marked sample images; and the original network model is trained using the M frames of marked sample images to obtain a target network model.

[0080] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0081] An embodiment of the present disclosure further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when run.

[0082] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0083] An embodiment of the present disclosure further provides a pool robot, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0084] In an exemplary embodiment, the pool robot may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0085] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0086] Obviously, those skilled in the art should understand that the modules or steps of the present disclosure described above can be implemented using a general-purpose computing device, they can be concentrated on a single computing device, or distributed across a network composed of multiple computing devices, they can be implemented using program code executable by the computing device, and thus, they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be performed in a different order than herein, or they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. Thus, the present disclosure is not limited to any particular combination of hardware and software.

[0087] The above are merely optional embodiments of the present disclosure and are not intended to limit the present disclosure. Those skilled in the art will readily appreciate that the present disclosure may be modified and varied in various ways. Any modifications, equivalent substitutions, improvements, and the like made within the principles of the present disclosure shall be included within the scope of protection of the present disclosure.

Claims

1. A control method for a pool robot, comprising: Acquiring image information of a target water area photographed by a pool robot, wherein the pool robot operates in the target water area; analyzing the image information to determine an analysis result; In a case where the analysis result is set to indicate that a target object exists in the target water area, a target position of the target object is determined, and the pool robot is controlled to move to the target position to perform a target operation corresponding to the type of the target object.

2. The method according to claim 1, wherein: Obtaining image information of the target water area photographed by the pool robot, including: Image information of the target water area captured by the pool robot when operating in a first operating mode is obtained, wherein the first operating mode includes a mode in which the pool robot operates in a random route between a plurality of preset positions in a preset time period.

3. The method according to claim 2, wherein: In the case where the analysis result is set to indicate that a target object exists in the target water area, determining a target position of the target object, controlling the pool robot to move to the target position, and performing a target operation corresponding to the type of the target object, including: In a case where the analysis result is set to indicate that a target object exists in the target water area, determining the target position according to the image information; The pool robot is controlled to move to the target position and perform a target operation corresponding to the type of the target object.

4. The method according to claim 1, wherein: After acquiring the image information of the target water area photographed by the pool robot, the method further includes: Determine pixel coordinates of the target object in multiple frames of images to obtain multiple pixel coordinates, wherein the image information includes multiple frames of the image; The plurality of pixel coordinates are converted and mapped into a coordinate system of the camera device using a first configuration parameter of the camera device in the pool robot to obtain a plurality of mapping coordinates, wherein the first configuration parameter includes a parameter when the image is captured; The target position of the target object in the target water area is determined using the plurality of mapping coordinates and second configuration parameters of the camera device, wherein the second configuration parameters include posture information when the camera device captures the image, and the posture information includes a rotation matrix and a translation vector.

5. The method according to claim 4, wherein: The plurality of pixel coordinates are transformed and mapped into a coordinate system of the camera device using the first configuration parameters of the camera device in the pool robot to obtain a plurality of mapping coordinates, including: Transform and map the plurality of pixel coordinates into normalized coordinates in the camera coordinate system to obtain a plurality of mapping coordinates; or, The plurality of pixel coordinates are transformed and mapped into imaging surface coordinates in the camera device coordinate system to obtain a plurality of mapping coordinates.

6. The method according to claim 1, wherein: Obtaining image information of the target water area photographed by the pool robot, including: Image information of the target water area captured by the pool robot when the pool robot operates in a second operation mode is obtained, wherein the second operation mode includes a mode in which the pool robot operates in a fixed route between multiple preset positions according to a preset time period.

7. The method according to claim 6, wherein: In the case where the analysis result is set to indicate that a target object exists in the target water area, determining a target position of the target object, controlling the pool robot to move to the target position, and performing a target operation corresponding to the type of the target object, including: When the pool robot is operating in the second operating mode, if it is determined from the analysis result that the target water area includes the target object, the target position of the target object is determined, and the pool robot is controlled to operate to the target position to perform a target operation corresponding to the type of the target object.

8. The method according to claim 1, wherein: In the case where the analysis result is set to indicate that a target object exists in the target water area, determining a target position of the target object, controlling the pool robot to move to the target position, and performing a target operation corresponding to the type of the target object, including: After the pool robot operates in the second operation mode, if it is determined from the analysis result that the target water area includes a plurality of the target objects, marking the positions of the plurality of the target objects in a map, wherein the map is constructed during the operation of the pool robot in the target water area; Determine the running route according to the positions of the plurality of target objects; The pool robot is controlled to run to multiple positions of multiple target objects according to a running route, and execute a target operation corresponding to the type of each target object.

9. The method according to claim 8, wherein: Determining a running route according to the positions of the plurality of target objects includes: The positions of the plurality of target objects are input into a target shortest path model to obtain the running route, wherein the running route is the shortest running route passing through the positions of the plurality of target objects.

10. The method according to claim 9, wherein: The target shortest path model is determined by: The objective function is defined as the length of the running route; The plurality of randomly combined running routes are used as a parent generation, and the plurality of randomly combined running routes are used as a child generation; The target shortest path model is determined using the parent generation and the child generation.

11. The method according to any one of claims 1 to 10, wherein: Before obtaining the image information of the target water area photographed by the pool robot, the method further includes: Controlling the pool robot to perform the target operation along the edge of the target water area; After determining that the pool robot performs the target operation along the edge, the pool robot is controlled to perform a patrol operation in the target water area, wherein the patrol operation includes controlling the pool robot to capture the image information in the target water area.

12. The method according to any one of claims 1 to 10, wherein: Before obtaining the image information of the target water area photographed by the pool robot, the method further includes: Controlling the pool robot to perform a work task in the target water area; wherein the work task includes performing the target operation along the boundary of the target water area, performing the target operation in the target water area according to a planned route, and performing the target operation again along the boundary of the target water area; After determining that the pool robot completes the work task, the pool robot is controlled to perform a patrol operation in the target water area, wherein the patrol operation includes controlling the pool robot to capture the image information in the target water area.

13. The method according to claim 1, wherein: The target object is determined by: Inputting the image information into a target network model to obtain feature information of an object included in the image information output by the target network model, wherein the feature information includes spatial features and semantic features of the object; The target object is determined from the feature information of the object.

14. The method according to claim 13, wherein: The target network model is trained in the following way: Acquire M frames of sample images, wherein each frame of the sample images includes a preset object, wherein M is a natural number greater than or equal to 1; Marking the preset object in each frame of the sample image to obtain M frames of marked sample images; The original network model is trained using the M frames of labeled sample images to obtain the target network model.

15. A control device for a pool robot, comprising: An acquisition module, configured to acquire image information of a target water area photographed by the pool robot, wherein the pool robot operates in the target water area; An analysis module, configured to analyze the image information to determine an analysis result; The determination module is configured to determine the target position of the target object when the analysis result is configured to indicate that the target object exists in the target water area, and control the pool robot to run to the target position and perform a target operation corresponding to the type of the target object.

16. A computer-readable storage medium having a computer program stored therein, wherein: When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 14 are implemented.

17. A pool robot, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of claims 1 to 14 when executing the computer program.

Images