Control method, cleaning robot, and storage medium

Abstract

This application relates to the field of cleaning robot technology, disclosing a control method, a cleaning robot, and a storage medium. The cleaning robot includes a camera and a light source. The camera is mounted on the chassis of the cleaning robot, and the light source is positioned near the camera, so that the light source can illuminate the ground in the direction the cleaning robot is moving. The image captured by the camera includes the area illuminated by the light source. The method includes: acquiring the current image captured by the camera; identifying the ground material based on the current image to obtain an identification result; and detecting a cliff based on the current image to obtain a detection result. Finally, the cleaning robot is controlled to operate based on the identification result and / or the detection result. In this embodiment, with fewer sensors, the environment can be accurately perceived, which is beneficial for the intelligent operation of the cleaning robot. In addition, the sensor integration effectively improves the problem of single sensor function and helps to meet the needs of circuit integration and robot miniaturization.

Description

Technical Field

[0001] This application relates to the field of cleaning robot technology, and in particular to a control method, a cleaning robot, and a storage medium. Background Technology

[0002] Cleaning robots are an important branch of modern robotics. They are robots capable of autonomously performing cleaning tasks in environments such as homes and large spaces. Common types include robotic vacuum cleaners, robotic mops, and combined vacuum and mop robots.

[0003] Cleaning robots are typically equipped with multiple sensors, including LiDAR, cliff detection sensors, and ultrasonic sensors. The LiDAR scans the robot's environment and creates a map based on the scan results. The robot then plans its path and moves according to this map. The cliff detection sensor detects sinking steps, helping the robot avoid them. The ultrasonic sensor detects the surface material (e.g., flooring or carpet), helping the robot avoid carpets or employing different cleaning modes based on the surface material.

[0004] It is evident that most cleaning robots rely on a large number of sensors for autonomous cleaning. These sensors have limited functionality, high cost, large space requirements, and require a high degree of data processing and integration. Summary of the Invention

[0005] In view of this, some embodiments of this application provide a control method, a cleaning robot, and a storage medium. By employing an integrated vision sensor (including a camera and a light source), accurate ground material recognition and cliff detection can be achieved, enabling accurate environmental perception and facilitating the intelligent operation of the cleaning robot. Furthermore, the integration of the ground material recognition and cliff detection sensors effectively addresses the issue of single-function sensors, helping to meet the demands of circuit integration and robot miniaturization.

[0006] In a first aspect, some embodiments of this application provide a control method applied to a cleaning robot, the cleaning robot including a camera and a light source, the camera being mounted on the chassis of the cleaning robot, and the light source being positioned near the camera, the method comprising:

[0007] Acquire the current image captured by the camera;

[0008] Based on the current image, ground material is identified, and the identification result is obtained;

[0009] Cliff detection is performed based on the current image, and the detection results are obtained;

[0010] Control the operation of the cleaning robot based on the identification and / or detection results.

[0011] In some embodiments, the aforementioned cliff detection based on the image, to obtain detection results, includes:

[0012] The initial height is obtained by measuring the distance based on the current image;

[0013] The power of the light source is adjusted according to the initial height, where the power of the light source is positively correlated with the initial height.

[0014] After adjusting the power of the light source, the current image is acquired and updated, and the distance is measured based on the current image to obtain the cliff height;

[0015] The test results were determined based on the cliff height.

[0016] In some embodiments, the camera is a monocular camera, the current image includes the current frame image and the previous frame image, and the cliff height is obtained by ranging based on the current image, including:

[0017] Divide the current frame image and the previous frame image into multiple image blocks respectively;

[0018] The first distance is determined based on two corresponding image blocks from consecutive frames;

[0019] The cliff height is determined based on multiple first distances.

[0020] In some embodiments, determining the first distance based on two corresponding image blocks from consecutive frames includes:

[0021] Extract the corresponding target points from the two image patches;

[0022] The first distance is determined based on the image positions of the target point in the two image blocks and the camera parameters.

[0023] In some embodiments, the aforementioned extraction of corresponding target points in two image patches includes:

[0024] Identify objects in two image patches and perform object matching;

[0025] If a matching object exists, a feature point of the matching object will be used as the target point.

[0026] If no matching object exists, pixel feature recognition and matching are performed on the two image blocks. Pixel features include corner features or contour features.

[0027] If a matching pixel exists, use that pixel as the target point.

[0028] In some embodiments, the method further includes:

[0029] If no matching pixel is found, the two image blocks are merged with at least one of their adjacent image blocks.

[0030] The two merged image blocks are first matched for objects, and then matched for pixel features to extract the target points.

[0031] In some embodiments, the method further includes:

[0032] If the target point is still not matched and extracted after merging all image blocks, the camera is rotated by a preset angle to obtain a new current frame image and a previous frame image, so as to match and extract the corresponding target point in the current frame image and the previous frame image.

[0033] In some embodiments, the method further includes:

[0034] If the camera rotates a preset number of times and still fails to extract the target point, cliff detection at the current location will stop.

[0035] In some embodiments, the light source has at least two preset light shapes, and the method further includes:

[0036] The initial default light source has the first shape as its light shape;

[0037] If the confidence level of the material identification result is less than or equal to the first threshold, the light shape of the light source is adjusted to the second shape, and the illumination area of ​​the second shape is greater than the illumination area of ​​the first shape.

[0038] In some embodiments, the method further includes:

[0039] Given that the shape of the light source is a first shape, identify the length or width of the light shape in the image;

[0040] If the length of the light shape is less than or equal to the second threshold, or the width is less than or equal to the third threshold, adjust the light shape of the light source to the second shape.

[0041] In some embodiments, the method further includes:

[0042] If the cliff height is found to be abnormal in the test results, the cliff test should be repeated.

[0043] If the detected cliff height is still abnormal, stop cliff detection using the current image, and acquire and update the current image.

[0044] Secondly, some embodiments of this application provide a cleaning robot, including:

[0045] The camera and light source are mounted on the chassis of the cleaning robot, and the light source is located near the camera.

[0046] At least one processor is connected in communication with both the camera and the light source.

[0047] A memory communicatively connected to at least one processor; wherein the memory stores instructions executable by at least one processor, the instructions being executed by at least one processor to enable at least one processor to perform the control method as described in the first aspect.

[0048] Thirdly, some embodiments of this application provide a computer-readable storage medium storing computer-executable instructions for causing a computer device to perform the control method as described in the first aspect.

[0049] The beneficial effects of this application's embodiments: Unlike existing technologies, the control method provided in this application is applied to a cleaning robot. The cleaning robot includes a camera and a light source. The camera is mounted on the robot's chassis, and the light source is positioned near the camera. Thus, the light source illuminates the ground in the direction the cleaning robot is moving, and the image captured by the camera includes the area illuminated by the light source. The method includes: acquiring a current image captured by the camera; performing ground material identification based on the current image to obtain an identification result; performing cliff detection based on the current image to obtain a detection result; and finally, controlling the cleaning robot to operate based on the identification result and / or the detection result.

[0050] In this embodiment, images are acquired using an integrated visual sensor that includes a camera and a light source. By combining image recognition and visual ranging, accurate ground material identification and cliff detection can be achieved. In other words, with fewer sensors, the environment can be accurately perceived, facilitating the intelligent control of the cleaning robot. Furthermore, the integration of the sensors for ground material identification and cliff detection effectively addresses the issue of single-function sensors, helping to meet the demands of circuit integration and robot miniaturization. Attached Figure Description

[0051] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0052] Figure 1 This is a schematic diagram illustrating the application environment of the control method for a cleaning robot in some embodiments of this application;

[0053] Figure 2 This is a schematic diagram of the structure of the cleaning robot in some embodiments of this application;

[0054] Figure 3 This is a flowchart illustrating a control method applied to a cleaning robot in some embodiments of this application;

[0055] Figure 4 This is a schematic diagram illustrating the principle of triangulation in some embodiments of this application;

[0056] Figure 5 This is a schematic diagram illustrating the single-frame image ranging principle in some embodiments of this application. Detailed Implementation

[0057] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. In addition, the terms "first," "second," and "third" used herein do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0060] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0061] Furthermore, the technical features involved in the various embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0062] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating the application environment of the control method provided in the embodiments of this application. For example... Figure 1 As shown, the cleaning robot 100 is located on the ground, which can be the ground in a living room, office, or outdoors. The location of the cleaning robot 100 includes objects such as the base station 200, desk, flower pot, sofa, cabinet, or bed.

[0063] The free movement of the cleaning robot 100 mainly relies on modules such as mapping, localization, navigation, and obstacle avoidance. The operational control of the cleaning robot 100 is a crucial factor affecting its normal movement. Understandably, after sensing its surroundings, the cleaning robot 100 plans one or more paths that avoid collisions with environmental obstacles and achieve full area coverage, following a certain area cleaning sequence or planning method. Understandably, these modules are implemented by sensors and corresponding control programs.

[0064] In some embodiments, the cleaning robot 100 is equipped with a LiDAR and / or a visible light camera. The LiDAR scans the surrounding environment of the cleaning robot 100 to obtain a laser point cloud. The visible light camera captures images of the surrounding environment. The LiDAR and the visible light camera are communicatively connected to a controller, sending the laser point cloud and the images to the controller. The controller then calls a pre-installed map-building program in the cleaning robot 100's memory to construct a map based on the laser point cloud and / or the images. The map-building program may include a program corresponding to the SLAM (Simultaneous Localization and Mapping) algorithm, which will not be described in detail here. In some embodiments, the map is a raster map. The map is stored in the cleaning robot 100's memory. When the robot is moving and operating, the controller uses the map as the basis for autonomous localization, path planning, and obstacle avoidance.

[0065] Understandably, SLAM algorithms possess both localization and navigation capabilities. During localization, a high-speed rotating lidar emits laser light to measure the distance between the cleaning robot and obstacles. This distance is then combined with a map to determine the relative position of the cleaning robot to the obstacles, thus achieving localization. In some embodiments, the cleaning robot 100 can perform visual localization using a visible light camera. During navigation, cleaning control is performed based on localization and the cleaning task. Each area to be cleaned is cleaned sequentially, and a full-coverage cleaning path is planned within each area to complete the corresponding cleaning task.

[0066] The cleaning robot 100 can be configured in any suitable shape to perform specific business functions. For example, in some embodiments, the cleaning robot 100 can be a mobile robot based on a SLAM system. The cleaning robot 100 includes, but is not limited to, sweeping robots, vacuuming robots, mopping robots, or floor washing robots.

[0067] In some embodiments, the cleaning robot 100 may include a robot body, a lidar, a controller, and a locomotion mechanism. The robot body is the main structure of the cleaning robot 100, and can be selected according to the actual needs of the cleaning robot 100, with appropriate shape, structure, and manufacturing material (such as hard plastic or metals such as aluminum or iron), for example, it can be set to a relatively flat cylindrical shape commonly used in cleaning robots 100.

[0068] The locomotion mechanism is a structural device mounted on the robot body that provides the cleaning robot 100 with mobility. This locomotion mechanism can be implemented using any type of mobility device, such as rollers, tracks, etc.

[0069] A lidar sensor is mounted on the body of the cleaning robot 100 to sense obstacles in the surrounding environment, scan and obtain laser point cloud data, and send it to the controller so that the controller can build a map based on the laser point cloud data and perform tasks such as walking and obstacle avoidance. In some embodiments, the lidar sensor includes pulse lidar, continuous wave lidar, and other types of radar.

[0070] The controller is an electronic computing core built into the robot's body, used to execute logical operations to achieve intelligent control of the cleaning robot. For example, the controller communicates with a LiDAR sensor to create a map based on LiDAR point cloud data and plan the cleaning path for the robot. For example, the controller also communicates with other sensors (such as cliff detection sensors or ultrasonic sensors) to detect the ground material (such as flooring or carpet) based on data signals collected by these sensors, controlling the cleaning robot to avoid carpets or adopt different cleaning modes based on the ground material, and to detect sinking steps and control the cleaning robot to avoid them.

[0071] Understandably, a controller can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a microcontroller, an ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. A controller can also be any conventional processor, controller, microcontroller, or state machine. A controller can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration, or one or more combinations of a microcontroller unit (MCU), a field-programmable gate array (FPGA), or a system-on-a-chip (SoC).

[0072] In some embodiments, the robot body may further include a clean water tank, a wastewater tank, a detergent dispenser, and a dust collection box. In this embodiment, the base station 200 includes a base, a cleaning device, a water supply device, a dust collection device, a power supply device, a detergent replenishment device, and a wastewater collection device (not shown). The base, cleaning device, water supply device, dust collection device, power supply device, detergent replenishment device, and wastewater collection device are not shown in the figures. The base is used to dock the cleaning robot, the cleaning device is used to clean the cleaning robot, the water supply device is used to supply water to the cleaning robot and / or to the cleaning device, and the dust collection device is used to collect dust from the cleaning robot's dust collection box. The power supply device is used to charge the cleaning robot. The wastewater collection device is used to collect wastewater from the cleaning robot's wastewater tank.

[0073] Understandably, the surface of the base station 200 used for approaching the cleaning robot 100 matches the shape of the cleaning robot. For example, the cleaning robot 100 is typically a relatively flat cylindrical shape; accordingly, the base station 200 is provided with an open docking cavity for accommodating the cleaning robot 100. The docking cavity is semi-cylindrical with a curved surface. In some embodiments, the radius of the docking cavity is 5-10 cm larger than the radius of the cleaning robot, so that when the cleaning robot 100 docks with the base station 200, the cleaning robot 100 is accommodated in the docking cavity.

[0074] It should be noted that, depending on the task to be completed, in addition to the above-mentioned functional modules, the main body of the cleaning robot can also be equipped with one or more other different functional modules (such as water tank, mop, etc.) to work together to perform the corresponding tasks.

[0075] Some control methods for cleaning robots known to the inventors of this application rely on a large number of sensors. These sensors have limited functionality, high cost, large space requirements, and high data processing integration requirements.

[0076] For example, when a cleaning robot is mopping on carpets, problems may arise such as the robot becoming stuck on the carpet or the mop getting stuck. Some solutions incorporate ultrasonic sensors on the bottom of the robot. These sensors work by recognizing that sound waves are absorbed more on carpets than on the floor, allowing the robot to determine the surface material based on the intensity of the echoes. Other solutions use the current of the bottom roller brush motor; the greater resistance on carpets causes a change in the motor's current, which can then be used to determine whether the robot is operating on a carpet.

[0077] In addition, the cleaning robot is equipped with cliff sensors to detect steps on the floor and control its movement based on the detection results. Typically, cliff sensors consist of an infrared transmitter and an infrared receiver.

[0078] It is evident that the control methods of most cleaning robots rely on a large number of sensors. These sensors have limited functionality, are costly, and occupy a lot of space, which is not conducive to miniaturization design.

[0079] To address the aforementioned problems, some embodiments of this application provide a control method, a cleaning robot, and a storage medium. The control method is applied to a cleaning robot, which includes a camera and a light source. The camera is mounted on the robot's chassis, and the light source is positioned near the camera, allowing the light source to illuminate the ground in the robot's forward direction. The image captured by the camera includes the area illuminated by the light source. The method includes: acquiring a current image captured by the camera; performing ground material identification based on the current image to obtain an identification result; performing cliff detection based on the current image to obtain a detection result; and finally, controlling the cleaning robot to operate based on the identification result and / or the detection result.

[0080] In this embodiment, images are acquired using an integrated visual sensor that includes a camera and a light source. By combining image recognition and visual ranging, accurate ground material identification and cliff detection can be achieved. In other words, with fewer sensors, the environment can be accurately perceived, facilitating the intelligent control of the cleaning robot. Furthermore, the integration of the sensors for ground material identification and cliff detection effectively addresses the issue of single-function sensors, helping to meet the demands of circuit integration and robot miniaturization.

[0081] Some embodiments of this application provide a cleaning robot; please refer to [link / reference]. Figure 2 The cleaning robot 100 includes a camera 101, a light source 102, at least one processor 103, and a memory 104. Figure 2 (Taking a bus connection and a single processor as an example).

[0082] The camera is mounted on the chassis (not shown) of the cleaning robot 100. For example, the camera 101 is positioned 2cm-6cm in front of the casters (not shown) on the chassis, where "in front" refers to the direction of travel of the cleaning robot. The angle of the camera 101 can be flexibly adjusted according to the scene to adjust the field of view. A light source 102 is positioned near the camera, thereby illuminating the ground below the cleaning robot or the ground in its direction of travel. The image captured by the camera 101 includes the area illuminated by the light source 102. In one case, the illumination range of the light source 102 is smaller than the field of view of the camera 101. In another case, the illumination range of the light source 102 is larger than the field of view of the camera 101.

[0083] In some implementations, the camera 101 can be a visible light camera or an invisible light camera (e.g., an infrared camera). Correspondingly, the light source 102 can be a visible light source or an invisible light source (e.g., an infrared light source). In some implementations, the shape of the light emitted by the light source 102 can be changed or dynamically adjusted according to the actual scene. For example, the light source has at least two preset light shapes, such as stripes, squares, circles, or rings. When the light shape is stripe, the light emitted by the light source projects a stripe shape onto the ground. When the light shape is square, the light emitted by the light source projects a square shape onto the ground, with the square area larger than the stripe area. When the light shape is ring, the light emitted by the light source projects a ring shape onto the ground.

[0084] It is understood that the processor 103 is used to provide computing and control capabilities to control the cleaning robot to perform any of the control methods provided in the following embodiments.

[0085] It is understood that the processor 103 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0086] Memory 104, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the control methods in the embodiments of this application. Processor 103 can implement any of the control methods provided in the following embodiments by running the non-transitory software programs, instructions, and modules stored in memory 104. Specifically, memory 104 may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 104 may also include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0087] In some embodiments, the cleaning robot 100 may also include other sensors such as lidar, gyroscope, odometry, magnetometer, accelerometer, or speedometer. It is understood that the structure illustrated in this embodiment does not constitute a limitation on the cleaning robot 100. In some implementations, the cleaning robot 100 may include more or fewer components than illustrated, or combine some components, or separate some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of both.

[0088] As can be understood from the above, the control method provided in this application embodiment can be implemented by a cleaning robot, for example, by one or more processors of the cleaning robot. In some embodiments, the control method provided in this application embodiment can also be implemented by other devices with computing power. Other devices with computing power can be intelligent devices that are communicatively connected to the cleaning robot, such as servers.

[0089] The control method provided in this application is described below with reference to exemplary applications and implementations of the cleaning robot provided in the embodiments of this application. Please refer to... Figure 3 , Figure 3 This is a flowchart illustrating the control method provided in an embodiment of this application. It is understood that the execution entity of this control method can be one or more processors of a cleaning robot.

[0090] like Figure 3 As shown, the method S100 includes, but is not limited to, the following steps:

[0091] S10: Acquire the current image captured by the camera.

[0092] Understandably, because the light source shines towards the ground, the camera can clearly capture visual data of the ground below the cleaning robot or the ground in its direction of movement, generating images. In some embodiments, the camera captures data at a certain frequency, such as 90 frames per second. While the cleaning robot is moving, at least one processor in the robot controls the light source to turn on, and the camera captures images at a certain frequency, thus capturing a temporal image sequence. The image acquired at the current moment is the current image; it is understood that the current image is continuously updated as the data acquisition process continues.

[0093] The current image reflects the ground conditions at the current location of the cleaning robot. These conditions include the ground material (floor or carpet, etc.), sunken steps, or objects on the ground (such as table and chair legs, tangled wires, shoes, trash cans, etc.). Understandably, when there are few obstacles on the ground where the cleaning robot is located, the current image may also show a uniform floor.

[0094] After the camera captures the current image, it sends the current image to the processor, so that the processor can obtain the current image.

[0095] S20: Perform ground material identification based on the current image to obtain the identification result.

[0096] In some embodiments, the floor material includes categories such as flooring, carpet, or cobblestone flooring. Optionally, carpets can be further subdivided into rugs or smooth mats, and flooring can be further subdivided into tiles or wood flooring. Those skilled in the art can define the floor material category based on the actual conditions of the floor in the real-world application scenario and the operation of the cleaning robot on different floor materials; here, no limitation is made on the category of floor material. For example, in a home setting, common floor materials include flooring or carpet. Considering that the cleaning robot is prone to getting its wheels or suction cups stuck on carpets, and therefore needs to avoid carpets, in this scenario, the floor material can be set to include both flooring and carpet to guide subsequent training data collection.

[0097] The cleaning robot has a pre-trained recognition model built into its memory. This recognition model is obtained by those skilled in the art by training a neural network using collected training data. In some embodiments, the neural network may be a network such as RCNN, YOLO, or SSD.

[0098] Understandably, the training data covers the object categories that the recognition model can identify. For example, data is collected on common objects in the application scenarios of cleaning robots, and category labels are assigned. For instance, several images are collected, including floors, carpets, cabling, shoes, pet cats and dogs, trash cans, table and chair legs, etc. These images are used to train a neural network. The neural network and the trained model parameters are then deployed to the cleaning robot's memory to obtain the recognition model, which is then used by the processor for real-time inference and detection.

[0099] The processor invokes the recognition model, inputs the current image into the model, and outputs the category of each object in the current image, such as the ground material category. Thus, the ground material category can be used as the recognition result.

[0100] S30: Perform cliff detection based on the current image and obtain the detection results.

[0101] Those skilled in the art will understand that in the field of cleaning robots, cliff detection refers to detecting changes in ground height, such as the height of stairs, sunken steps, or other cliff-like objects, to prevent the cleaning robot from falling or plunging off a cliff. When the cleaning robot approaches the edge of a cliff, it determines the detection result by measuring the height from the ground to its base (i.e., the cliff height). For example, if the cliff height is greater than or equal to a preset height, the detection result is determined to be a cliff; if the cliff height is less than the preset height, the detection result is determined to be a non-cliff. This detection result is helpful in controlling the operation of the cleaning robot; for example, when the detection result is a cliff, the robot can be controlled to stop immediately to prevent a fall.

[0102] In this embodiment, cliff detection is performed based on the current image, that is, the cliff height is determined based on the current image, and the detection result is determined. The specific process of image-based ranging will be described in detail below; please refer to the specific implementation process of step S33 below.

[0103] In some embodiments, in order to reduce the problem that the brightness of the light source affects the clarity of the current image and thus affects the ranging accuracy, step S30 specifically includes the following steps:

[0104] S31: Measure the distance based on the current image to obtain the initial height.

[0105] The current image here was acquired before adjusting the light source power. At this point, due to changes in ground conditions, the light source power may no longer be suitable for the height from the ground to the chassis of the cleaning robot. Therefore, the height obtained by ranging based on this current image is used as the initial height. The specific process of ranging based on the current image will be described in detail below, and you can refer to the specific implementation process of step S33 below.

[0106] S32: Adjust the power of the light source according to the initial height, wherein the power of the light source is positively correlated with the initial height.

[0107] If the detected initial height is large, the power of the light source is increased; if the detected initial height is small, the power of the light source is decreased, so that the power of the light source is adapted to the initial height (distance), and high ranging accuracy can be obtained at different distances.

[0108] In some embodiments, the power of the light source is related to the initial height as follows:

[0109]

[0110] Where Power is the power of the light source, x is the initial height, and a, b, c are model parameters.

[0111] In this embodiment, the power of the light source is positively correlated with the initial height, that is, the power of the light source is positively correlated with the distance from the ground to the chassis of the cleaning robot. As the detected initial height increases, it indicates that there may be a cliff ahead, so the power of the light source is increased for further refinement; as the detected initial height decreases, it indicates that there may not be a cliff ahead, so the power of the light source is decreased for further refinement. On the one hand, this can reduce the error caused by insufficient light source power; on the other hand, dynamically adjusting the power of the light source can save energy.

[0112] S33: After adjusting the power of the light source, acquire and update the current image, and measure the distance based on the current image to obtain the cliff height.

[0113] Understandably, the camera continuously captures images at a certain frequency (e.g., 90 frames per second). After adjusting the light source power, the newly acquired images are used as the current image to update the current image. Then, distance is measured based on the updated current image, and the resulting height is taken as the cliff height. The specific process of distance measurement based on the current image will be described in detail below.

[0114] In this embodiment, since the adjusted light source power is matched with the initial height, the initial height can initially reflect the cliff height. Therefore, with the adjusted light source power, the cliff height can be further accurately detected, reducing the problem of large cliff height errors caused by insufficient light source power, and making the cliff height accurate.

[0115] In some embodiments, the ground material can also be identified using the current image obtained after adjusting the power. For the specific implementation process, please refer to step S20, which will not be described again here.

[0116] In some embodiments, the camera can be a binocular camera, which, based on the ranging principle of a binocular camera, can measure distances based on the current image. The ranging principle of a binocular camera is existing technology and will not be described in detail here.

[0117] However, in binocular ranging, issues such as inconsistent focal lengths can lead to system instability, and the hardware costs are high while the algorithm integration is complex. Therefore, in some embodiments, a monocular camera is used, requiring only one camera, resulting in a simpler structure and easier setup.

[0118] In monocular ranging, the distance (cliff height) is determined based on two consecutive image frames. In this embodiment, the current image includes the current frame image and the previous frame image. It can be understood that the current frame image refers to the image captured by the cleaning robot at the current moment during its movement, and the previous frame image is the image preceding the current frame image in the time-series data.

[0119] In this embodiment, step S33, "measuring the distance based on the current image to obtain the cliff height," specifically includes:

[0120] S331: Divide the current frame image and the previous frame image into multiple image blocks respectively.

[0121] For example, for the current frame image or the previous frame image, it is divided into H image blocks of size M*N according to their dimensions. The division method is the same for both, and image blocks at the same position are corresponding image blocks. For example, the first image block in the upper left corner of the current image frame corresponds to the first image block in the upper left corner of the previous image frame, which is equivalent to two corresponding image blocks in the previous and next frames.

[0122] S332: Determine the first distance based on two corresponding image blocks from the preceding and following frames.

[0123] Here, the two image blocks corresponding to consecutive frames refer to two image blocks located at the same position in consecutive frames. It can be understood that, due to the high frame rate of the camera, the cleaning robot's movement distance between consecutive frames is small. Therefore, the current frame image and the previous frame image contain the same content with minimal difference. In other words, the two image blocks corresponding to consecutive frames contain roughly the same image content.

[0124] Since the two image patches are highly likely to contain the same object, a first distance can be determined based on the corresponding image patches from consecutive frames. This first distance is the distance determined based on the image patches.

[0125] In some embodiments, the aforementioned step S332 specifically includes:

[0126] S3321: Extract the corresponding target points from the two image patches.

[0127] In this context, the target point is a feature point of an object in the world coordinate system. Since the content of two image patches is similar, the corresponding target points in the two image patches can be identified and extracted through matching. For example, if both image patches include table legs, the target point can be a point where the table leg contacts the ground. If both image patches include wall corners, the target point can be the vertex of the wall corner. If both image patches include carpet edges, the target point can be a point on the carpet edge.

[0128] In some embodiments, the aforementioned step S3321 specifically includes:

[0129] (1) Identify objects in two image blocks and perform object matching.

[0130] Here, a recognition model can be used to identify objects in the two image patches and determine the object categories. The recognition model has already been described in detail in step S20 and will not be repeated here.

[0131] Object matching refers to determining whether two image patches contain the same object. For example, if both image patches are found to contain table legs, the features of the table legs in the two image patches are compared. If the features are similar or identical, it is determined that the two image patches contain the same table leg, i.e., a matching object exists.

[0132] (2) If a matching object exists, then a feature point of the matching object is taken as the target point.

[0133] For example, if the object to be matched is a table leg, then a feature point where the table leg contacts the ground is taken as the target point.

[0134] (3) If there is no matching object, pixel feature recognition and pixel feature matching are performed on the two image blocks.

[0135] In other words, when no matching object can be found, pixel feature matching is performed. Pixel features include corner features or contour features. Pixel feature matching refers to determining whether two image patches contain the same pixel features. In some embodiments, corner detection algorithms can be used to identify corner features in image patches. These algorithms may include existing corner detection algorithms such as the Kitchen-Rosenfeld corner detection algorithm, the Harris corner detection algorithm, the KLT corner detection algorithm, or the SUSAN corner detection algorithm. In some embodiments, algorithms such as Sobe I and Canny can be used to extract contour features from image patches.

[0136] After identifying pixel features (including corner features and contour features) in two image patches, it is determined whether the two image patches contain the same pixel features in order to perform pixel feature matching. If they contain the same pixel features, then there are matching pixels.

[0137] (4) If a matching pixel exists, the matching pixel will be used as the target point.

[0138] For example, if a matching pixel is a corner point, then the corner point is used as the target point.

[0139] In this embodiment, for two corresponding image blocks, object matching is performed first, followed by pixel feature matching. On the one hand, this can accurately extract the target point, which is beneficial to the accuracy of subsequent distance calculation; on the other hand, it can quickly determine the target point, saving computing power.

[0140] In some embodiments, the aforementioned step S3321 further includes:

[0141] (5) If there are no matching pixels, the two image blocks are merged with at least one of their adjacent image blocks.

[0142] Understandably, if no matching object or matching pixel is detected in two image patches, it's possible that both image patches contain homogeneous ground planes, thus increasing the detection range. The image patch in the current frame is merged with at least one adjacent image patch (e.g., two or more), and the image patch in the previous frame is merged with at least one adjacent image patch (e.g., two or more) in the same way, resulting in two merged image patches. Merging image patches increases the detection range, which is beneficial for identifying target points.

[0143] (6) Perform object matching on the two merged image blocks first, and then perform pixel feature matching to extract the target point.

[0144] Similarly, for the two merged image blocks, according to the above steps (1)-(4), object matching is performed first, and then pixel feature matching is performed, so as to extract the target point.

[0145] It is understandable that if the target point cannot be extracted from the two image blocks after merging once, it can continue to be merged with at least one surrounding image block. Following the steps (1)-(4) above, object matching is performed first, followed by pixel feature matching, until the target point is extracted or all image blocks are merged.

[0146] Compared to extracting target points from the entire image, in this embodiment, for multiple image blocks, target points are extracted separately first. If the target points cannot be extracted, they are then merged and extracted. Each time object features or pixel features are identified, the image content is simple and there is little interference, which helps to improve the accuracy of target points.

[0147] In some embodiments, the method S100 further includes:

[0148] (7) If the target point is still not matched and extracted after merging all image blocks, the camera is rotated by a preset angle to obtain a new current frame image and a previous frame image, and the target point corresponding to the current frame image and the previous frame image is matched and extracted.

[0149] Even if all image blocks are merged into one, the target point cannot be determined. This means that no identical object, corner, or contour exists in the current frame or the previous frame. In this case, the processor controls the camera to rotate by a preset angle and then captures images, obtaining a new current frame image and a previous frame image. The preset angle can be 20°, 30°, 40°, 50°, or 60°, etc.

[0150] For example, the camera is controlled to rotate 60° clockwise or counterclockwise to capture images and update the current frame image and the previous frame image. For the updated current frame image and the previous frame image, object matching can be performed first, and then pixel matching can be performed according to the above steps (1)-(6). If the target point is not extracted, the matching is performed after merging (object matching first, then pixel matching) until the target point is extracted or all image blocks are merged.

[0151] If the target point still cannot be determined after merging all image blocks, step (7) can be repeated until the target point is extracted.

[0152] In this embodiment, by adjusting the angle of the camera and changing its field of view, the probability of matching the target point between the current frame image and the previous frame image is increased. This helps the cleaning robot find the target point at the current location and reduces the interference caused by the camera's perspective in cliff detection.

[0153] In some embodiments, the method S100 further includes:

[0154] (8) If the camera rotates a preset number of times and still fails to extract the target point, then stop cliff detection at the current location.

[0155] For example, the preset number of attempts can be 6. If the camera rotates 6 times and still fails to find the target point, cliff detection at the current location stops. As the cleaning robot moves, cliff detection is performed again after the camera captures the image of the next location and updates the current image.

[0156] In this embodiment, setting a limit on the number of rotations can reduce errors caused by a small number of image defects at the current position, and timely use of the image at the next position for detection, so that cliff detection can be carried out smoothly, which is conducive to timely detection of cliffs and reduces the risk of the cleaning robot falling off a cliff.

[0157] S3322: Determine the first distance based on the image positions of the target point in the two image blocks and the camera parameters.

[0158] As mentioned above, the target point is a feature point of an object in the world coordinate system. The target point is presented in two image patches. Therefore, using the principle of triangulation, the first distance can be determined based on the image positions of the target point in the two image patches and the camera parameters.

[0159] Please see Figure 4Point P is the target point, point P1 is the image position of target point P in the previous frame, point P2 is the image position of target point P in the current frame, point O is the optical center position of the camera when capturing the current frame, and point O' is the optical center position of the camera when capturing the previous frame. The distance b between points O and O' is the baseline, which can be understood as the translation distance of the camera when capturing the previous and current frames.

[0160] According to the principle of triangulation, the first distance from the target point P to the baseline b can be calculated, that is, the first measured distance is: z = f * b / d

[0161] Where x is the first distance, f is the focal length of the camera, b is the baseline, and d is the disparity value of the same feature falling in the preceding and following frames, d=|uL-uR|.

[0162] In this embodiment, the first distance can be accurately determined by using the triangulation principle based on the two consecutive frames of images captured by a monocular camera.

[0163] It is understandable that for image blocks containing target points, the first distance is calculated according to the above step S3322.

[0164] S333: Determine the cliff height based on multiple first distances.

[0165] Understandably, a single image patch theoretically yields a first distance, thus H image patches correspond to H first distances. In some possible cases, an image patch may only contain a floor of uniform material, making it impossible to determine the first distance. In other words, the number of these multiple first distances is less than or equal to H.

[0166] Understandably, each first distance reflects the distance between the ground and the camera, and these first distances are not exactly the same due to calculation errors. Therefore, the cliff height is determined based on multiple first distances. For example, the average of the multiple first distances can be used as the cliff height, or the mode of the multiple first distances can be used as the cliff height.

[0167] In this way, by dividing the current frame image and the previous frame image into regions, calculating the first distance in each region, and determining the cliff height based on multiple first distances, on the one hand, real-time performance can be improved and recognition computing resources can be saved; on the other hand, the accuracy of cliff height can be improved.

[0168] S34: Determine the test results based on the cliff height.

[0169] For example, when the cliff height is greater than or equal to a preset height, the detection result is determined to be a cliff; when the cliff height is less than the preset height, the detection result is determined to be a non-cliff. This detection result is helpful in controlling the operation of the cleaning robot. For instance, when the detection result is a cliff, the cleaning robot can be controlled to stop immediately to prevent it from falling.

[0170] S40: Control the operation of the cleaning robot based on the identification and / or detection results.

[0171] Understandably, if a cleaning robot encounters carpets or cliffs while moving, it needs to stop promptly to reduce the risk of its wheels or suction cups getting stuck or falling off the cliff. Controlling the cleaning robot's operation based on recognition and / or detection results enables its intelligent operation.

[0172] In this embodiment, images are acquired using an integrated visual sensor that includes a camera and a light source. By combining image recognition and visual ranging, accurate ground material identification and cliff detection can be achieved. In other words, with fewer sensors, the environment can be accurately perceived, facilitating the intelligent control of the cleaning robot. Furthermore, the integration of the sensors for ground material identification and cliff detection effectively addresses the issue of single-function sensors, helping to meet the demands of circuit integration and robot miniaturization.

[0173] In some embodiments, the light source has at least two preset light shapes. These at least two preset light shapes have already been described above and will not be repeated here.

[0174] The method S100 further includes:

[0175] S50: The initial default light source shape is the first shape.

[0176] For example, the first shape is a strip shape. The light in the image captured by the camera is a long strip shape.

[0177] S60: If the confidence level of the recognition result is less than or equal to the first threshold, adjust the light shape of the light source to the second shape, and the illumination area of ​​the second shape is greater than the illumination area of ​​the first shape.

[0178] Understandably, when the recognition model outputs a recognition result (ground material category), it also outputs a confidence score. In the field of machine learning, confidence score reflects the veracity of the predicted output.

[0179] When the confidence level of the identification result is less than or equal to a first threshold (e.g., 80%), it indicates that the identified ground material category lacks accuracy. Therefore, the shape of the light source is adjusted to a second shape with a larger illumination area; exemplarily, the second shape is a rectangle or a square.

[0180] When the light beam is in the second shape, its area in the image is larger than when it is in the first shape. This allows the image to include more ground features, making subsequent ground material recognition and cliff detection based on the current image more accurate. Furthermore, the dynamic adjustment of the light beam shape helps save energy.

[0181] In some embodiments, the method S100 further includes:

[0182] S70: If the shape of the light source is a first shape, identify the length or width of the light shape in the current image.

[0183] S80: If the length of the light shape is less than or equal to the second threshold, or the width is less than or equal to the third threshold, adjust the light shape of the light source to the second shape.

[0184] When the light source has a first shape, the light shape in the image is a long strip. In this embodiment, the camera's field of view is larger than the light range of the first-shape light source, thus the image can reflect the light shape. The length and width of the light shape (long strip) can be determined from the image through pixel recognition.

[0185] When the length of the light source is less than or equal to a second threshold, or the width is less than or equal to a third threshold, a cliff may be preliminarily identified. In this case, the light source shape is adjusted to the second shape to acquire a new current image for further cliff detection. The second and third thresholds can be set based on the actual situation of the strip light. If the cliff height detected under the second-shape light source is greater than or equal to a preset height, the robot immediately stops and reverses. If the cliff height detected under the second-shape light source is less than the preset height, the robot continues to operate, and the light source shape is adjusted to the first shape.

[0186] In this embodiment, the second-shaped light source has a large illumination area, which makes the features of the camera's effective field of view more obvious, thereby enabling more accurate measurement of cliff height and making cliff detection more accurate. Furthermore, using the first-shaped light source for preliminary detection and then the second-shaped light source for detailed detection allows the light source shape to be dynamically adjusted between the first and second shapes, which helps save energy.

[0187] In some embodiments, the method S100 further includes:

[0188] S101: If the cliff height is abnormal in the test results, the cliff test shall be carried out again.

[0189] Understandably, if the cliff height is less than the chassis height of the cleaning robot, or if the cliff height is greater than the height of a typical step, it indicates a possible anomaly.

[0190] Cliff detection anomalies can occur for various reasons, such as system errors, calculation errors due to unclear feature points, or distortion errors generated during operation. Therefore, at this location, the processor re-measures the distance based on the current image to detect the cliff height. In some embodiments, the light source can illuminate at the currently adjusted power, or at a preset fixed value.

[0191] S102: If the detected cliff height is still abnormal, stop cliff detection using the current image, and acquire and update the current image.

[0192] Understandably, due to the high frame rate of the camera, if an anomaly is detected during cliff detection using the current image, the cleaning robot continues to move, covering a very small distance, ensuring safety even if there is a cliff ahead. Furthermore, stopping cliff detection using the current image, acquiring and updating the current image, and then starting cliff detection using the updated current image helps ensure smoother, more timely, and accurate cliff detection.

[0193] To prevent specific objects (such as tangled wires) from getting caught in the wheels or brushes of the cleaning robot and causing it to become trapped, in some embodiments, the distance to specific objects is identified based on images captured by a camera, so as to control the cleaning robot to maintain a certain distance from specific objects during operation.

[0194] like Figure 5 As shown, X is the length of a specific object, which is stored in advance in the cleaning robot. X' is the length of the specific object on the sensor imaging plane through the optical center of the camera, f is the distance between the objects, and z is the object distance.

[0195] Therefore, the expression for the object distance Z is as follows:

[0196]

[0197] In this embodiment, the length of a specific object is known, allowing the distance from the object to the camera to be obtained based on imaging principles. This distance is then used to control the cleaning robot to maintain a certain distance from the specific object during operation, preventing the robot from becoming trapped due to excessive proximity.

[0198] In summary, the control method provided in this application is applied to a cleaning robot. The cleaning robot includes a camera and a light source. The camera is mounted on the chassis of the cleaning robot, and the light source is positioned near the camera. Thus, the light source can illuminate the ground in the direction the cleaning robot is moving, and the image captured by the camera includes the area illuminated by the light source. The method includes: acquiring a current image captured by the camera; performing ground material recognition based on the current image to obtain a recognition result; performing cliff detection based on the current image to obtain a detection result; and finally, controlling the cleaning robot to operate based on the recognition result and / or the detection result.

[0199] In this embodiment, images are acquired using an integrated visual sensor that includes a camera and a light source. By combining image recognition and visual ranging, accurate ground material identification and cliff detection can be achieved. In other words, with fewer sensors, the environment can be accurately perceived, facilitating the intelligent control of the cleaning robot. Furthermore, the integration of the sensors for ground material identification and cliff detection effectively addresses the issue of single-function sensors, helping to meet the demands of circuit integration and robot miniaturization.

[0200] This application also provides a computer-readable storage medium storing computer-executable instructions for causing an electronic device to perform the control method provided in this application.

[0201] In some embodiments, the storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0202] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0203] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0204] As an example, executable instructions can be deployed to execute on a single computing device (including devices such as smart terminals and servers), or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0205] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the control method as described in the foregoing embodiments.

[0206] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0207] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software and a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0208] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of this application as described above, which are not provided in detail for the sake of brevity; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A control method applied to a cleaning robot, characterized in that, The cleaning robot includes a camera and a light source. The camera is mounted on the chassis of the cleaning robot, and the light source is positioned near the camera. The method includes: Acquire the current image captured by the camera; Ground material identification is performed based on the current image to obtain the identification result; The initial height is obtained by measuring the distance based on the current image; The power of the light source is adjusted according to the initial height, wherein the power of the light source is positively correlated with the initial height; After adjusting the power of the light source, the current image is acquired and updated, and the distance is measured based on the current image to obtain the cliff height; The test results are determined based on the cliff height. The cleaning robot is controlled to operate based on the identification results and / or the detection results; Wherein, the camera is a monocular camera, the current image includes the current frame image and the previous frame image, and the step of measuring the distance based on the current image to obtain the cliff height includes: The current frame image and the previous frame image are each divided into multiple image blocks; The first distance is determined based on two corresponding image blocks from consecutive frames; The cliff height is determined based on multiple first distances.

2. The method according to claim 1, characterized in that, The step of determining the first distance based on two corresponding image blocks from consecutive frames includes: Extract the corresponding target points from the two image blocks; The first distance is determined based on the image position of the target point in the two image blocks and the camera parameters.

3. The method according to claim 2, characterized in that, Extracting the corresponding target points from the two image blocks includes: Identify objects in the two image blocks and perform object matching; If a matching object exists, a feature point of the matching object is taken as the target point. If no matching object exists, pixel feature recognition and pixel feature matching are performed on the two image blocks. The pixel features include corner features or contour features. If a matching pixel exists, the matching pixel is used as the target point.

4. The method according to claim 3, characterized in that, The method further includes: If no matching pixel is found, the two image blocks are merged with at least one of their adjacent image blocks. The object matching is performed on the two merged image blocks first, and then the pixel feature matching is performed to extract the target point.

5. The method according to claim 4, characterized in that, The method further includes: If the target point is still not matched and extracted after merging all image blocks, the camera is rotated by a preset angle to obtain a new current frame image and a previous frame image, and the target point corresponding to the current frame image and the previous frame image is matched and extracted.

6. The method according to claim 5, characterized in that, The method further includes: If the camera rotates a preset number of times and still fails to extract the target point, then cliff detection at the current location will stop.

7. The method according to any one of claims 1-6, characterized in that, The light source has at least two preset light shapes, and the method further includes: The light shape of the light source is initially assumed to be the first shape; If the confidence level of the recognition result is less than or equal to the first threshold, the light shape of the light source is adjusted to a second shape, and the illumination area of ​​the second shape is greater than the illumination area of ​​the first shape.

8. The method according to claim 7, characterized in that, The method further includes: When the shape of the light source is the first shape, identify the length or width of the light shape in the current image; If the length of the light shape is less than or equal to a second threshold, or the width is less than or equal to a third threshold, the light shape of the light source is adjusted to the second shape.

9. The method according to claim 1, characterized in that, The method further includes: If the cliff height is abnormal in the test results, the cliff test should be repeated. If the detected cliff height is still abnormal, stop cliff detection using the current image, and acquire and update the current image.

10. A cleaning robot, characterized in that, include: A camera and a light source, wherein the camera is mounted on the chassis of the cleaning robot, and the light source is located near the camera; At least one processor is communicatively connected to both the camera and the light source. A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the control method as described in any one of claims 1-9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer device to perform the control method as described in any one of claims 1-9.

Images