Automatic pool cleaning device and control method thereof

Abstract

The invention provides an automatic pool cleaning device and a control method thereof, the control method is used for controlling the automatic pool cleaning device to clean a pool, the automatic pool cleaning device comprises a sensor unit and a phased array ultrasonic unit, and the control method comprises the steps that first information of an object is acquired through the sensor unit; second information of the object is obtained through the phased array ultrasonic unit; and based on the first information and the second information, controlling the automatic cleaning device of the water pool to carry out path planning. According to the control method of the automatic pool cleaning device, different functions of different sensors can be combined, the functional defect of a single sensor can be made up, the attribute information of the obstacle can be accurately obtained, and a better obstacle avoidance effect can be achieved.

Description

Technical Field

[0001] The present application relates to the technical field of automatic pool cleaning, and in particular to an automatic pool cleaning device and a control method thereof. Background Art

[0002] Current automatic pool cleaning devices usually use cameras, ultrasonic sensors, laser sensors, etc. to measure obstacles and then perform path planning to avoid obstacles. However, these sensors have a single function and cannot solve all possible obstacle avoidance problems in the pool. For example, ultrasonic sensors have blind spots in close-range obstacle avoidance. Cameras can identify objects, but it is difficult to obtain depth information of objects. In addition, these sensors are easily affected by environmental factors in water, have poor detection accuracy, and cannot effectively avoid obstacles, which directly affects the cleaning efficiency of the pool cleaning robot.

[0003] Application Contents

[0004] In view of the above problems, the present application provides an automatic pool cleaning device and a control method thereof, which can combine the different functions of different sensors, make up for the functional defects of a single sensor, accurately obtain the attribute information of obstacles, and achieve better obstacle avoidance effect.

[0005] In a first aspect, a control method for an automatic pool cleaning device is provided, which is used to control the automatic pool cleaning device to clean a pool. The automatic pool cleaning device includes a sensor unit and a phased array ultrasonic unit. The control method includes: acquiring first information of an object through the sensor unit; acquiring second information of the object through the phased array ultrasonic unit; and controlling the automatic pool cleaning device to perform path planning based on the first information and the second information.

[0006] In one example, after acquiring the first information of the object through the sensor unit, the control method further includes: identifying the first information to acquire semantic information of the object, wherein controlling the automatic pool cleaning device to perform path planning based on the first information and the second information includes: controlling the automatic pool cleaning device to perform path planning based on the semantic information and the second information.

[0007] In one example, controlling the automatic pool cleaning device to perform path planning based on the semantic information and the second information includes: forming a data set with the semantic information and the second information; and controlling the cleaning device to perform path planning according to the data set.

[0008] In one example, the second information includes size information of the object, wherein obtaining the second information through the phased array ultrasonic unit includes: obtaining distance information of the object; obtaining a detection angle of the phased array ultrasonic unit to the object; and determining the size information of the object based on the distance information and the detection angle.

[0009] In one example, the second information is image information. After acquiring the second information through the phased array ultrasonic unit, the control method further includes: identifying the image information to acquire size information of the object, and controlling the automatic pool cleaning device to perform path planning based on the first information and the second information includes: controlling the automatic pool cleaning device to perform path planning based on the first information and the size information.

[0010] In one example, controlling the cleaning device to perform path planning includes: performing image fusion of the first information and the second information to form third information, wherein the third information is image information; acquiring semantic information and size information of the object based on the third information; and controlling the cleaning device to perform path planning based on the semantic information and the size information.

[0011] In one example, the sensor unit and the phased array ultrasonic unit are controlled separately to obtain the first information and the second information.

[0012] In one example, the sensor unit and the phased array ultrasonic unit are synchronously controlled to acquire the first information and the second information.

[0013] In one example, the first information includes image information or point cloud data information.

[0014] In a second aspect, an automatic pool cleaning device is also provided, comprising: a sensor unit for acquiring first information of an object; a phased array ultrasonic unit for acquiring second information of the object; and a processor for controlling the automatic pool cleaning device to perform path planning based on the first information and the second information.

[0015] In one example, the sensor unit and the phased array ultrasound unit are integrated into the same module.

[0016] In one example, the first information includes semantic information.

[0017] According to a third aspect, a non-volatile storage medium is provided, which stores computer program instructions. When the instructions are executed by a processor, the control method in any of the above examples is executed.

[0018] The control method of the automatic pool cleaning device and the automatic pool cleaning device provided in the embodiments of the present application can combine the different functions of different sensors to accurately determine the properties of obstacles, select obstacle avoidance paths according to the properties of the obstacles, achieve better obstacle avoidance effects, and improve the cleaning efficiency of the automatic cleaning device.

[0019] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0021] Figure 1 is a schematic structural diagram of an automatic pool cleaning device according to at least one embodiment of the present application;

[0022] Figure 2 is a flow chart of a control method of an automatic pool cleaning device according to at least one embodiment of the present application;

[0023] Figure 3 is a flow chart of a control method of an automatic pool cleaning device according to another embodiment of the present application;

[0024] Figure 4 is a flow chart of a control method of an automatic pool cleaning device according to another embodiment of the present application;

[0025] Figure 5 This is a flow chart of a control method of an automatic pool cleaning device according to another embodiment of the present application. DETAILED DESCRIPTION

[0026] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to enable the scope of the present application to be fully communicated to those skilled in the art. In addition, in the following description, the description of known structures and technologies is omitted to avoid unnecessary confusion of the concepts of the present application.

[0027] Various structural schematic diagrams according to embodiments of the present application are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may further design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0028] The present application provides a control method for an automatic pool cleaning device. The automatic pool cleaning device can be mobile and clean in a pool-shaped building, and the pool-shaped building can be, for example, a swimming pool, a water storage pool, a hydrotherapy pool, a water storage tank, a water storage tank, etc. The present application does not limit the specific presentation of the automatic pool cleaning device and the pool-shaped building, as long as the principle of the present application can be realized.

[0029] Figure 1 is a schematic structural diagram of an automatic pool cleaning device according to at least one embodiment of the present application; Figure 2 1 is a flow chart of a control method of an automatic pool cleaning device according to at least one embodiment of the present application. Figure 1 and Figure 2 , a control method for an automatic pool cleaning device provided in an embodiment of the present application is introduced.

[0030] Figure 1 The figure shows the structure of an automatic pool cleaning device. Figure 1 , the automatic pool cleaning device 100 may include a sensor unit 101 and a phased array ultrasonic unit 102. The sensor unit 101 and the phased array ultrasonic unit 102 may be integrated in the same module, which can reduce the space occupied by the two units, and thus appropriately optimize the wiring connected to the two units, thereby improving the space utilization in the automatic pool cleaning device, and at the same time ensure that the fields of view of the sensor unit and the phased array ultrasonic unit overlap and can sense the same target object. The sensor unit 101 can obtain first information about the object. The phased array ultrasonic unit 102 can obtain second information about the object.

[0031] In one embodiment, the sensor unit 101 may be various types of sensors. For example, the sensor unit 101 may be a single-point ultrasonic sensor. The single-point ultrasonic sensor is mainly used for distance measurement and obstacle detection. The single-point ultrasonic sensor can convert ultrasonic signals into other energy signals (such as electrical signals). Ultrasonic waves are mechanical waves with a vibration frequency higher than 20kHz. It has the characteristics of high frequency, short wavelength, small diffraction phenomenon, especially good directionality, and the ability to become rays and propagate in a directional manner. The single-point ultrasonic sensor can be composed of one or more fixed ultrasonic array elements, and the distance is calculated by measuring the time from the emission to the reception of the ultrasonic wave, thereby detecting whether there is an obstacle and the position of the obstacle. The single-point ultrasonic sensor may also include: drive circuit, signal processor and other components, and the detailed description of these components is omitted here.

[0032] In addition, the sensor unit 101 may also include one or more of a camera, a single-photon imaging sensor DTOF, an infrared sensor or a radar.

[0033] Cameras can be used for underwater distance measurement and image acquisition. For example, a camera can be used to capture images underwater and extract feature points from the captured images, such as using edge detection, corner detection and other methods to extract feature points in the image. The extracted feature points can be used for subsequent distance measurement and image recognition.

[0034] Single-photon imaging sensors (DTOF) can determine the distance of an object by measuring the time it takes for a light signal to be emitted and returned. The core principle of this technology is to use a laser or other light source to emit a single photon and time its return time to calculate the distance of the target object.

[0035] Infrared sensor is a sensor that uses infrared rays to process data. It has the advantages of high sensitivity. Infrared sensor can control the operation of the drive device.

[0036] Radar uses radio methods to detect targets and determine their spatial positions. Therefore, radar is also called "radio positioning". Radar is an electronic device that uses electromagnetic waves to detect targets. Radar emits electromagnetic waves to illuminate the target and receives its echo, thereby obtaining information such as the distance from the target to the electromagnetic wave emission point, the rate of change of distance (radial velocity), direction, altitude, etc.

[0037] It should be understood that the above description of the components and functions of various sensors is merely exemplary and does not constitute a limitation on the various parameters and functions of the above components. Those skilled in the art can select and set the above sensors and their components, parameters and functions according to actual needs, as long as the principles of the present application can be implemented.

[0038] In one embodiment, the phased array ultrasonic unit 102 may also be referred to as a phased array ultrasonic probe. The phased array ultrasonic probe generally includes the following components:

[0039] (1) A phased array composed of multiple independent chip units. The chip unit is made of piezoelectric material. Each chip unit is called an ultrasonic array element (abbreviated as array element). Each array element has an independent connector and excitation circuit, and can transmit and receive ultrasonic signals through piezoelectric effect and inverse piezoelectric effect, and the array elements are insulated from each other. Therefore, functionally, each array element can be regarded as an ultrasonic probe. For example, a phased array probe with 32 chip units is equivalent to 32 conventional ultrasonic probes. In practice, the common size of each array element can be 0.5*10mm, and the spacing between array elements is, for example, 0.4~1mm.

[0040] (2) Driving circuit: The driving circuit is used to control the excitation signal of each array element to ensure that the array element works according to the set time and phase.

[0041] (3) Signal processor: The signal processor is used to amplify, filter and digitize the reflected ultrasonic signal.

[0042] The phased array ultrasonic unit 102 can form a synthetic ultrasonic beam by controlling the transmission timing and phase of each array element, and realize the direction control of the ultrasonic beam, for example, it can selectively focus on a certain point or a specific detection area in space. The phased array ultrasonic unit 102 can also scan from multiple angles at the same time, and use efficient algorithms (such as time domain reflection method, frequency domain analysis, etc.) to process the received ultrasonic signals, extract useful information, and generate images. Due to the presence of multiple array elements, the phased array ultrasonic unit 102 can use multi-channel acquisition technology to synchronously receive multiple signals and improve imaging accuracy.

[0043] It should be understood that the above description of the various components in the phased array ultrasonic unit is merely exemplary and does not constitute a limitation on the various parameters and functions of the above components. Those skilled in the art can select and set the phased array ultrasonic probe and its components, parameters, and functions according to actual needs, as long as the principles of the present application can be implemented.

[0044] Figure 2 is a flow chart of a control method of an automatic pool cleaning device according to the present application. Figure 2The control method of the automatic pool cleaning device includes steps S201-S203. In step S201, the first information of the object is obtained by the sensor unit. In step S202, the second information of the object is obtained by the phased array ultrasonic unit. In step S203, the automatic pool cleaning device is controlled to perform path planning based on the first information and the second information.

[0045] Each step in steps S201-S203 will be introduced below.

[0046] In step S201, first information of an object is acquired through a sensor unit.

[0047] In one example, the first information may include image information or point cloud data information.

[0048] In one example, the sensor unit may be one or more of an ultrasonic sensor, a camera, a single-photon imaging sensor DTOF, an infrared sensor, or a lidar, and thus the object may be imaged by one or more of these sensor units to obtain image information of the object.

[0049] In another example, point cloud data information of an object can be obtained through a laser radar.

[0050] The point cloud data information of the target object obtained by the laser radar can refer to the spatial point data set of the object obtained by scanning the laser radar. Each point cloud contains three-dimensional coordinate information, namely the three elements of X, Y, and Z. The three-dimensional structure information of the measured target object can be obtained through these coordinate information. Each point cloud also reflects the intensity information, which is related to the surface material and roughness of the target object, the laser incident angle, the laser wavelength, and the energy density of the laser radar. In addition, the point cloud data also includes the number of echoes. The number of echoes is the total number of echoes for a given pulse, which can provide information about the reflection characteristics of the target object.

[0051] In one example, a laser radar can acquire point cloud data of a target object by emitting a laser signal to the target object and collecting the laser signal reflected from the target object. The distance information of the target object is measured by using the speed of light and the time information from the emission to the return of the laser, and then the three-dimensional coordinate information and distance information of the target object are calculated by combining the information of the inertial measurement unit IMU, odometer, global navigation satellite system GNSS, etc.

[0052] After acquiring the point cloud data information of the object, the point cloud data information can be processed. In one example, point cloud data processing can include a preprocessing process and a data parsing process. The directly collected point cloud data is usually full of noise and isolated points. In the preprocessing, preprocessing steps such as filtering, registration, and segmentation are required to obtain a smoother surface. In the data parsing process, the original point cloud data is parsed from the data packet and converted into a data format that can be further processed, such as PCD format, which contains Cartesian coordinates (x, y, z) and intensity values. In this way, the point cloud data of the target object that has been partially processed can be obtained.

[0053] The above describes an example of obtaining first information of a detected object, such as image information or point cloud data information, through a sensor unit. Through the above example, the characteristics and functions of the sensor unit can be effectively utilized to obtain image information or point cloud data information of a detected object (such as an obstacle, garbage to be cleaned, etc.). After obtaining this information, the first information of the detected object can be used to control the automatic pool cleaning device for path planning.

[0054] In step S202, second information of the object is acquired by the phased array ultrasonic unit.

[0055] In one example, the second information may include size information of the object.

[0056] That is, the size information of the object can be obtained by the phased array ultrasonic unit. The size information can be obtained by the distance information of the object detected by the phased array ultrasonic unit. For example, the distance information of the object is first obtained by the phased array ultrasonic unit, and the detection angle of the phased array ultrasonic unit to the object is determined, and then the size information of the object is determined based on the distance information and the detection angle.

[0057] For example, by utilizing an array composed of multiple transmitters and receivers in a phased array ultrasonic unit, the phase and amplitude of each transmitter can be controlled to achieve focusing and directional emission of ultrasound. By controlling the emission time of each transmitting transducer in the phased array ultrasonic unit, the ultrasound can be focused at a certain point in space. The spatial distance is then calculated by analyzing and calculating the time of the reflected echo, thereby obtaining the distance information between the phased array ultrasonic unit and the target object.

[0058] In addition, the phased array ultrasonic unit can use multiple detection angles when detecting an object. By controlling the phase settings of different chips, different chips can emit waves in different directions. When the phased array ultrasonic unit detects the target object at different angles, the amplitude distribution of the particles in the axial direction of the sound field will change, and the amplitude will change with the change of the detection angle. Different detection angles can be recorded each time the target object is detected.

[0059] After obtaining the distance information and detection angle information of the phased array ultrasonic unit to the target object, the distance between each end point of the target object can be determined based on the above distance information and detection angle information using mathematical methods such as triangulation and trigonometric functions, thereby determining the size information of the target object.

[0060] In addition, in another example, the size of the target object can also be detected by the ultrasonic propagation technology of the phased array ultrasonic unit. For example, the ultrasonic wave in the phased array ultrasonic wave can propagate in the material. When the ultrasonic wave passes through the object material, it interacts with the interface, defects or other features in the material. The propagation speed of the ultrasonic wave in different media will change, and this change causes the ultrasonic wave to be reflected, scattered, diffracted and transmitted. By measuring the propagation time and amplitude change of the ultrasonic wave, information such as the size and internal structure of the target object can be obtained.

[0061] In addition, in another example, the size of an object can also be detected through the beam control principle of a phased array ultrasonic unit. The beam refers to the propagation path and shape of ultrasound in space. Traditional unit ultrasonic probes can only emit ultrasound in a fixed direction, while phased array ultrasonic probes can change the direction and shape of the beam by controlling the emission time and amplitude of each unit. By adjusting the combination of emission time and amplitude, an oblique, electronically scanned ultrasonic beam can be formed to achieve accurate measurement of the size, orientation, posture and other parameters of the object being measured.

[0062] In addition, the size of the object can be detected by phase adjustment technology based on the wave physics of the phased array ultrasonic unit. For example, by changing the time between a series of output ultrasonic pulses, the individual waves generated by each chip in the array are combined with each other to increase or eliminate energy in a predictable way, thereby controlling and shaping the sound beam. This example can dynamically deflect the sound beam at different angles, different focal lengths, and different focal sizes to complete the detection of the entire object being tested.

[0063] The above describes a variety of ways to obtain the size information of the object through the phased array ultrasonic unit. Through one or more of the above methods, the size information, distance information, etc. of the object can be obtained. Those skilled in the art can select one or more of the methods, or other undescribed methods, according to actual needs to obtain the size information of the object. This application does not limit this.

[0064] In addition, in one embodiment, the sensor unit and the phased array ultrasonic unit are controlled separately to obtain the first information and the second information. For example, two processors may be provided to control the sensor unit 101 and the phased array ultrasonic unit 102 respectively. Thus, the cleaning device may control the sensor unit 101 to obtain the first information when the first information needs to be obtained, and control the phased array ultrasonic unit 102 to obtain the second information when the second information needs to be obtained.

[0065] In another embodiment, the sensor unit and the phased array ultrasonic unit are synchronously controlled to obtain the first information and the second information. For example, the sensor unit and the phased array ultrasonic unit are synchronously controlled using the same processor to obtain the first information and the second information at the same time.

[0066] It should be noted that although step S201 and step S202 are described in sequence above, that is, step S201 is first executed to obtain the first information, and then step S202 is executed to obtain the second information, the acquisition of the first information and the second information can be performed in the opposite order, that is, step S202 is first executed to obtain the second information, and then step S201 is executed to obtain the first information. In addition, as described above, the first information and the second information can be acquired at the same time, that is, step S201 and step S202 are executed at the same time.

[0067] The above describes a variety of ways to obtain the second information of the detected object, such as the size information of the detected object, through the phased array ultrasonic unit. Through the above methods, the characteristics and functions of the phased array ultrasonic unit can be effectively utilized to obtain the size, orientation, posture, structure and other attribute information of the detected object. After obtaining this information, it is convenient to use this information of the detected object to determine whether the detected object is an obstacle and its various attribute information, thereby providing conditions for accurately controlling the automatic pool cleaning device to perform path planning.

[0068] In step S203, the automatic pool cleaning device is controlled to perform path planning based on the first information and the second information.

[0069] After the first information and the second information have been acquired through the above-mentioned steps S201 and S202, in step S203, the automatic pool cleaning device may be controlled to perform path planning based on the first information and the second information.

[0070] For example, after the first information is identified according to the aforementioned embodiment or example to obtain the semantic information of the object, the automatic pool cleaning device can be controlled to perform path planning based on the semantic information and the second information.

[0071] For example, according to the aforementioned embodiments or examples, when the second information is size information, the automatic pool cleaning device may be controlled to perform path planning based on the first information and the size information.

[0072] In one example, the semantic information obtained in step S201 and the second information obtained in step S202 may be formed into a data set, and then the cleaning device may be controlled to perform path planning according to the data set.

[0073] For example, in step S201, after obtaining the first information of the object through the sensor unit, the first information can be added to the first information database corresponding to the object. For example, when the first information is image information or point cloud data information, the image information of the object obtained is added to the image information database, and when the first information is point cloud data information, the point cloud data information of the object obtained is added to the point cloud data information database. In step S202, after obtaining the second information of the object through the phased array ultrasonic unit, the second information can be added to the second information database corresponding to the object. In this way, in step S203, the automatic pool cleaning device can be controlled to perform path planning according to the data set formed by combining the first information database and the second information database. For example, the first information of the detected object is obtained from the first information database of the data set, and the second information of the detected object is obtained from the second information database of the data set. The processor analyzes the first information and the second information to determine the properties of the detected object, such as type, size, distance, orientation, posture, structure, etc., so as to determine whether the object is an obstacle, what kind of obstacle it is, whether it can be avoided, how passable the obstacle is, and how to plan a path to bypass the obstacle, etc.

[0074] For example, if there is a first object (such as a drain) on the cleaning path of the automatic pool cleaning device, in step S201, the sensor unit 101 is used to obtain the first information of the first object, that is, image information or point cloud data, and the first information of the object obtained is added to the corresponding first information database. The processor identifies the image information and point cloud data in the information database, and determines the first features of the first object such as basic shape, color, and second features such as function, attribute, relationship and type through the acquired semantic information. The automatic pool cleaning device can determine the first object as a drain based on these features. Afterwards or at the same time, in step S202, the first object is further detected using a phased array ultrasonic unit to obtain second information of the first object, such as size, orientation, posture and other information. The second information is added to the second information database corresponding to the object.

[0075] In step S203, the processor determines that the first object is a drain according to the first information in the first information database, determines the size information of the first object according to the second information in the second information database, and combines these two aspects of information to determine whether the drain of this size is an obstacle that needs to be avoided. For example, it is determined whether the size of the drain is larger than the preset size (for example, the preset size corresponds to the maximum size of the obstacle when the automatic pool cleaning device can travel along a predetermined path and pass through the obstacle smoothly). If it is larger than the preset size, the automatic pool cleaning device cannot pass directly, and it is determined that the drain needs to be avoided. Afterwards, a path can be further planned to bypass the drain based on the information of the drain (such as size, orientation, etc.). If it is determined that the size of the drain is smaller than the preset size, the automatic pool cleaning device can pass directly, so there is no need to plan an obstacle avoidance path.

[0076] In another example, if there is a first object (e.g., a branch) on the cleaning path of the automatic pool cleaning device, in step S201, the processor determines that the first object is a branch based on the first information in the first information database, and in step S202, the size information of the first object is determined based on the second information in the second information database, and these two pieces of information are combined to determine whether a branch of this size is an obstacle that needs to be avoided. For example, it is determined whether the size of the branch is larger than a preset size (e.g., the preset size corresponds to the maximum size of the branch when the automatic pool cleaning device can travel along a predetermined path and successfully suck the branch into its internal trash basket / dust box). If it is larger than the preset size, the automatic pool cleaning device cannot pass directly, and it is determined that the branch needs to be avoided, thereby planning an obstacle avoidance path. If branches of any size are not considered obstacles and the cleaning device can pass directly and clean, it is determined that there is no need to plan an obstacle avoidance path.

[0077] The above describes an example of controlling the automatic pool cleaning device to perform path planning based on the first information and the second information. Through these examples, the first information and the second information can be effectively combined to obtain comprehensive information such as the type, structure, size, orientation, and posture of the detected object, so as to accurately determine whether the detected object is an obstacle that needs to be avoided. This makes the obstacle identification information richer and more complete, improves the recognition rate and the accuracy of path planning, and further improves the cleaning efficiency of the cleaning device.

[0078] Figure 3 FIG. 1 is a flow chart of a control method of an automatic pool cleaning device according to another embodiment of the present application. Figure 3 , the control method of the automatic pool cleaning device includes steps S301-S304. In step S301, the first information of the object is obtained by the sensor unit. In step S302, the first information is identified to obtain semantic information of the object. In step S303, the second information of the object is obtained by the phased array ultrasonic unit. In step S304, the automatic pool cleaning device is controlled to perform path planning based on the semantic information and the second information.

[0079] That is to say, Figure 2 Based on the embodiment shown in FIG. 1 , after the first information of the object is obtained by the sensor unit, the control method of the automatic pool cleaning device further includes S302: identifying the first information to obtain semantic information of the object. Steps S301, S303, and S304 are respectively Figure 2 Steps S201, S202 and S203 of the illustrated embodiment are the same or similar and are not described in detail here. Only step S302 is described in detail below.

[0080] In S302, the first information is identified to obtain semantic information of the object. When the first information is image information or point cloud data, both the image information and the point cloud data may contain semantic information, and the image information or the point cloud data may be analyzed and identified to obtain the semantic information. Semantic information recognition may, for example, be to identify objects in the image or point cloud data that carry rich semantic information and can be directly associated with the language description. These objects not only contain low-level features such as the basic shape and color of the object, but also contain higher-level semantic information, such as function, attribute, relationship, type, etc.

[0081] In one example, image semantic recognition enables computers to understand the content of an image and give the image deeper semantic information. In image semantic recognition, feature extraction can be performed first, such as converting image information into a form that can be understood by a computer, such as edges, textures, colors, and shapes. Then image recognition is performed, that is, mapping image features to predefined categories, including tasks such as object recognition, scene recognition, and action recognition. After that, semantic understanding can be performed to convert image information into higher-level knowledge, such as identifying the association between various objects in the image, the association between objects and scenes, etc.

[0082] In one example, semantic recognition of point cloud data involves analyzing and understanding three-dimensional point cloud data to identify and classify objects and scenes therein. For example, point cloud semantic recognition can use semantic segmentation algorithms, which can be divided into traditional methods and deep learning-based methods. Deep learning methods such as PointNet, PointNet++, and KPConv. These models can effectively extract features of point clouds and assign corresponding semantic labels to each point. In one example, point cloud data can be used together with other sensor data, such as images, radar, GPS, etc., to provide more comprehensive information and obtain more accurate three-dimensional scene understanding and perception.

[0083] Figure 4 FIG. 1 is a flow chart of a control method of an automatic pool cleaning device according to another embodiment of the present application. Figure 4 The control method of the automatic pool cleaning device includes steps S401 to S404. In step S401, the first information of the object is obtained by the sensor unit. In step S402, the second information of the object is obtained by the phased array ultrasonic unit, and the second information is image information. In step S403, the image information is recognized to obtain the size information of the object. In step S404, the automatic pool cleaning device is controlled to perform path planning based on the first information and the size information.

[0084] That is to say, Figure 2 Based on the embodiment shown, if the second information of the object obtained by the phased array ultrasonic unit is image information, after the image information is obtained by the phased array ultrasonic unit, the image information can be identified to obtain the size information of the object. Step S401, step S402, and step S404 are respectively Figure 2 Step S201, step S202, and step S203 of the illustrated embodiment are the same or similar, and are not described in detail here. Only step S403 is described in detail below.

[0085] Since the second information is image information, in step S403, the image information is recognized to obtain the size information of the object.

[0086] In one example, the object is imaged by a phased array ultrasonic unit to obtain image information of the object. For example, the electronic scanning and focusing of the ultrasonic beam can be achieved by controlling the transmitting elements and receiving elements of multiple independent phased array ultrasonic units, thereby obtaining image information of the object. For example, each time the focusing unit of the phased array ultrasonic unit generates a scan line for scanning, each scan line is finally assembled into a two-dimensional image.

[0087] In one example, the phased array ultrasonic unit can image the object using a full-focus imaging method. For example, a virtual focusing point is selected in the object to be imaged. For a one-dimensional phased array, a coordinate system is established with the geometric center of the array transducer as the origin, and the imaging area is divided into a number of pixels. The sound pressure amplitude of each pixel is calculated based on the collected data, and then normalized and imaged for display, thereby obtaining image information of the object.

[0088] The above-mentioned imaging method enables the phased array ultrasonic unit to provide detailed internal and external images of the object being detected, and can efficiently obtain information such as the type, property, structure, etc. of the object being detected.

[0089] After imaging an object using a phased array ultrasonic unit, the size information of the object can be further identified based on the image information obtained after imaging. In one example, the imaged image can be pre-processed such as denoising, enhancement, and correction to improve the accuracy of subsequent recognition. After that, feature extraction is performed on the image information to identify key features in the image, such as edges, corners, textures, etc. Machine learning or deep learning methods, such as convolutional neural networks (CNNs), can then be used to identify objects in the image.

[0090] After an object is identified, its size can be estimated by one of the following methods:

[0091] Pixel measurement: Measure the pixel size of the object directly on the image and then convert it to the actual size based on the camera's calibration parameters.

[0092] Scale: Include a scale of known size in the image and then estimate the size of the object based on the scale.

[0093] Deep Learning Model: Use a trained deep learning model to directly predict the size of an object.

[0094] Calibration and calibration: To ensure the accuracy of measurement, the camera needs to be calibrated to determine the conversion relationship between pixel size and actual physical size, so as to obtain the size information of the object, such as the length, width, height, diameter and other parameters of the object.

[0095] Figure 5 is a flow chart of controlling the cleaning device to perform path planning according to an embodiment of the present application. Figure 5 , the control method of the automatic pool cleaning device includes steps S501 to S504. In step S501, the first information of the object is obtained by the sensor unit. In step S502, the second information of the object is obtained by the phased array ultrasonic unit. In step S503, the first information and the second information are image fused to form third information, and the third information is image information; in step S504, the semantic information and size information of the object are obtained based on the third information; in step S505, the cleaning device is controlled to perform path planning based on the semantic information and the size information.

[0096] That is to say, Figure 2 The specific implementation of step S103 in the embodiment shown (i.e., controlling the automatic pool cleaning device to perform path planning based on the first information and the second information) includes: Figure 5 Step S503, step S504 and step S505 are shown. Figure 5 Steps S501 and S502 of the embodiment shown correspond to Figure 2 Step S201 and step S202 of the illustrated embodiment are not described in detail here. Only step S503, step S504 and step S505 are described in detail below.

[0097] In step S503, the first image information acquired by the sensor unit and the second image information acquired by the phased array ultrasonic unit are fused to form third information, which may be image information, for example. In step S504, semantic information and size information of the object are acquired based on the third information; in step S505, the cleaning device is controlled to perform path planning based on the semantic information and the size information.

[0098] When image fusion is performed in step S503, feature points of the first image information and the second image information may be matched, and then fused according to the matching result to form image information. Feature points of an image may include, for example, edges, textures, colors, and the like.

[0099] Feature point extraction algorithms can use SIFT (Scale Invariant Feature Transform) and SURF (Speeded Up Robust Features). SIFT algorithm can extract feature points that remain invariant to rotation, scale transformation and brightness changes. SURF algorithm is an enhanced version of SIFT, which extracts feature points faster and more robustly.

[0100] After the feature points are extracted, feature point matching can be performed to determine the location of the same object between different images. The matching method can, for example, adopt a KNN algorithm based on a KD tree, which can quickly find matching pairs of feature points. Homography transformation matrix calculation can also be used. In the homography transformation matrix calculation, the homography transformation matrix (Homography) between the first image information and the second image information can be calculated through the matched feature point pairs. Using the homography transformation matrix, the image can be transformed from one view to another, so that the feature points are aligned in space, thereby performing image fusion on the first image information and the second image information.

[0101] In step S504, the semantic information and the size information of the object are acquired based on the fused image information.

[0102] Similar to the previous semantic recognition method, in one example, semantic recognition can be performed based on the fused image information to obtain the semantic information of the object. For example, feature extraction can be performed first, such as converting the image information into a computer-understandable form, such as edges, textures, colors, and shapes. Then image recognition is performed, that is, mapping image features to predefined categories, including tasks such as object recognition, scene recognition, and action recognition. After that, semantic understanding can be performed to convert image information into higher-level knowledge, such as identifying the association between objects in the image, the association between objects and scenes, etc.

[0103] Similar to the previous embodiment, in one example, when obtaining the size information of an object based on the fused image information, the imaged image can be pre-processed such as denoising, enhancement, and correction to improve the accuracy of subsequent recognition. Afterwards, feature extraction is performed on the image information to identify key features in the image, such as edges, corners, textures, etc. Machine learning or deep learning methods, such as convolutional neural networks (CNNs), can then be used to identify objects in the image. After the object is identified, the size of the object can be calculated by the pixel measurement, scale, deep learning model, calibration, and calibration methods described above, thereby obtaining the size information of the object, which may include parameters such as the length, width, height, and diameter of the object.

[0104] In step S505, the cleaning device is controlled to perform path planning based on the semantic information and the size information.

[0105] Similar to the previous embodiment, by analyzing the semantic information and size information obtained from the third information, various parameters such as the category, structure, size, distance, and attributes of the target object are obtained. Based on these parameters, it is determined whether the target object is an obstacle that needs to be avoided. If so, the cleaning device is controlled to re-plan the path based on parameters such as the attributes and volume of the obstacle.

[0106] The above introduces various embodiments of the control method of the automatic pool cleaning device of the present application, which can combine the different functions of different sensors to detect objects, make up for the functional defects of a single sensor, accurately obtain the attribute information of the object, and thus determine whether the object is an obstacle, and then perform reasonable path planning and effective obstacle avoidance.

[0107] The following further introduces the automatic pool cleaning device in the control method of the automatic pool cleaning device of the present application.

[0108] Figure 1 Schematic diagram of the structure of the automatic pool cleaning device according to the present application. Figure 1 , the automatic pool cleaning device 100 comprises:

[0109] The sensor unit 101 is used to obtain first information of the object;

[0110] A phased array ultrasonic unit 102, configured to obtain second information of the object;

[0111] The processor 103 controls the automatic pool cleaning device to perform path planning based on the first information and the second information.

[0112] Among them, the specific embodiments and examples of the sensor unit 101 for obtaining the first information of the object can refer to the steps corresponding to how to use the sensor unit 101 to obtain the first information of the object in the control method of the aforementioned automatic pool cleaning device, which will not be repeated here.

[0113] Among them, the specific embodiments and examples of the phased array ultrasonic unit 102 for obtaining the second information of the object can refer to the steps corresponding to how to use the phased array ultrasonic unit 102 to obtain the second information of the object in the control method of the aforementioned automatic pool cleaning device, which will not be repeated here.

[0114] The processor 103 can control the sensor unit 101 and the phased array ultrasonic unit 102 to obtain the first information or the second information, and control the automatic pool cleaning device to perform path planning based on the first information and the second information. For specific embodiments and examples of how to use the processor 103 to control the automatic pool cleaning device to perform path planning based on the first information and the second information, reference can be made to the steps corresponding to how to use the processor 103 to control the automatic pool cleaning device to perform path planning based on the first information and the second information in the aforementioned control method of the automatic pool cleaning device, which will not be repeated here.

[0115] According to another embodiment of the present application, a non-volatile storage medium is provided, which stores computer program instructions. When the instructions are executed by a processor, the control method of the automatic pool cleaning device in any of the aforementioned embodiments is executed.

[0116] It should be understood that the above description of the components of the automatic pool cleaning device is merely exemplary and does not constitute a limitation on the various parameters and functions of the above components. Those skilled in the art can select and set the sensor module and its components, parameters, and functions according to actual needs, as long as the principles of the present application can be implemented.

[0117] The algorithm and description provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the application is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the application described herein, and the description made to specific languages ​​above is for the purpose of disclosing the best mode of implementation of the application.

[0118] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0119] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various aspects of the application, in the above description of exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof.

[0120] Those skilled in the art will appreciate that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. The modules or units or components in the embodiment can be combined into one module or unit or component, and in addition they can be divided into multiple submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including abstracts and drawings) and all processes or units of any method or device disclosed in this manner can be combined in any combination. Unless otherwise explicitly stated, each feature disclosed in this specification (including accompanying abstracts and drawings) can be replaced by alternative features that provide the same, equivalent or similar purposes.

[0121] Furthermore, those skilled in the art will appreciate that although some embodiments herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present application and to form different embodiments.

[0122] It should be noted that the above embodiments illustrate the present application rather than limit the present application. Any reference symbol between brackets should not be constructed as a limitation to the present application. The word "comprising" does not exclude the existence of components or steps not listed in the present application. The word "one" or "an" before a component does not exclude the existence of multiple such components. The present application can be implemented by means of hardware including several different components and by means of a suitably programmed computer. In the embodiments in which several devices are listed, several of these devices can be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names.

Claims

1. A control method for an automatic pool cleaning device, used to control the automatic pool cleaning device to clean a pool, the automatic pool cleaning device comprising a sensor unit and a phased array ultrasonic unit, the control method comprising: Acquire first information of the object through the sensor unit; Acquiring second information of the object by using the phased array ultrasonic unit; as well as The automatic pool cleaning device is controlled to perform path planning based on the first information and the second information.

2. The control method according to claim 1, wherein: After acquiring the first information of the object through the sensor unit, the control method further includes: identifying the first information to acquire semantic information of the object, wherein: The controlling the automatic pool cleaning device to perform path planning based on the first information and the second information includes: The automatic pool cleaning device is controlled to perform path planning based on the semantic information and the second information.

3. The control method according to claim 2, wherein: Controlling the automatic pool cleaning device to perform path planning based on the semantic information and the second information includes: Forming a data set from the semantic information and the second information; and The cleaning device is controlled to perform path planning according to the data set.

4. The control method according to claim 1, wherein: The second information includes size information of the object, wherein acquiring the second information by using the phased array ultrasonic unit includes: Acquiring distance information of the object; Acquiring a detection angle of the phased array ultrasonic unit to the object; The size information of the object is determined based on the distance information and the detection angle.

5. The control method according to claim 1, wherein: The second information is image information. After the second information is acquired by the phased array ultrasonic unit, the control method further includes: Recognize the image information to obtain the size information of the object, and The controlling the automatic pool cleaning device to perform path planning based on the first information and the second information includes: controlling the automatic pool cleaning device to perform path planning based on the first information and the size information.

6. The control method according to claim 1, wherein: Controlling the cleaning device to perform path planning includes: Performing image fusion on the first information and the second information to form third information, wherein the third information is image information; Acquire semantic information and size information of the object based on the third information; The cleaning device is controlled to perform path planning based on the semantic information and the size information.

7. The control method according to claim 1, wherein: The sensor unit and the phased array ultrasonic unit are controlled separately to obtain the first information and the second information.

8. The control method according to claim 1, wherein: The sensor unit and the phased array ultrasonic unit are synchronously controlled to acquire the first information and the second information.

9. The control method according to any one of claims 1 to 8, wherein: The first information includes image information or point cloud data information.

10. An automatic pool cleaning device, comprising: A sensor unit, used for acquiring first information of the object; A phased array ultrasonic unit, used for acquiring second information of the object; The processor controls the automatic pool cleaning device to perform path planning based on the first information and the second information.

11. The automatic pool cleaning device according to claim 10, wherein: The sensor unit and the phased array ultrasonic unit are integrated in the same module.

12. The automatic pool cleaning device according to claim 10 or 11, wherein: The first information includes semantic information.

13. A non-volatile storage medium storing computer program instructions, wherein when the instructions are executed by a processor, the method according to any one of claims 1 to 9 is executed.

Images